2024-04-21T08:49:28.517 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-21T08:49:28.518 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538 --name leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --description fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} -- /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/orig.config.yaml 2024-04-21T08:49:28.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T08:49:28.680 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538 branch: wip-lusov-quiescer description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} email: null first_in_suite: false job_id: '7666538' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4f6bc5273643aaf5660909e85400647a8951c10b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7599 sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e sleep_before_teardown: 0 subset: 111/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa suite_relpath: qa suite_repo: https://github.com/leonid-s-usov/ceph-ceph suite_sha1: 4f6bc5273643aaf5660909e85400647a8951c10b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNWDD6NXJaM0r1cQ6hz827U7hnauLGZej8cHcsYfO/sxrBHiZ9jxxYlemwuvuQv9qtzhAQTb7TEc3IAUMSKVc9A= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUXBiHT4qAE/mo7YvPRoA788R9+B6KJzvCWntbMNXc854vuuq3/mC66GojIy4Xr/+nZZZ/zicQHArNkb40NPjY= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtX7ERHlIMZyJNMv7GKaPSMdQwgvQGrc8HgYRDdEoDc7GklfjHkGaBoM7r4nbTBsnHB87xg+chHSCL+9eM6fFE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-21_08:12:55 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-21T08:49:28.680 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa; will attempt to use it 2024-04-21T08:49:28.681 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks 2024-04-21T08:49:28.681 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-21T08:49:28.682 INFO:teuthology.task.internal:Checking packages... 2024-04-21T08:49:28.705 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a630765c1f93c6e61d9b40c4b76437611d5ef29e' 2024-04-21T08:49:28.705 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-21T08:49:28.705 INFO:teuthology.packaging:ref: None 2024-04-21T08:49:28.705 INFO:teuthology.packaging:tag: None 2024-04-21T08:49:28.706 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-21T08:49:28.706 INFO:teuthology.packaging:sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:49:28.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-21T08:49:28.864 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.ga630765c 2024-04-21T08:49:28.866 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-21T08:49:28.872 INFO:teuthology.task.internal:no buildpackages task found 2024-04-21T08:49:28.872 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-21T08:49:28.908 INFO:teuthology.task.internal:Saving configuration 2024-04-21T08:49:28.929 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-21T08:49:28.947 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:49:28.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:43:38.123882', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNWDD6NXJaM0r1cQ6hz827U7hnauLGZej8cHcsYfO/sxrBHiZ9jxxYlemwuvuQv9qtzhAQTb7TEc3IAUMSKVc9A='} 2024-04-21T08:49:28.995 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:43:38.121229', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUXBiHT4qAE/mo7YvPRoA788R9+B6KJzvCWntbMNXc854vuuq3/mC66GojIy4Xr/+nZZZ/zicQHArNkb40NPjY='} 2024-04-21T08:49:29.016 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:43:38.122744', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtX7ERHlIMZyJNMv7GKaPSMdQwgvQGrc8HgYRDdEoDc7GklfjHkGaBoM7r4nbTBsnHB87xg+chHSCL+9eM6fFE='} 2024-04-21T08:49:29.016 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-21T08:49:29.021 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-21T08:49:29.021 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-21T08:49:29.021 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-21T08:49:29.021 INFO:teuthology.run_tasks:Running task console_log... 2024-04-21T08:49:29.143 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1dfefa90d0>, signals=[15]) 2024-04-21T08:49:29.143 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-21T08:49:29.149 INFO:teuthology.task.internal:Opening connections... 2024-04-21T08:49:29.149 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T08:49:29.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:49:29.227 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-21T08:49:29.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:49:29.300 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-04-21T08:49:29.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:49:29.375 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-21T08:49:29.382 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-04-21T08:49:29.401 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-04-21T08:49:29.401 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-04-21T08:49:29.459 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:49:29.460 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:49:29.460 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:49:29.460 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:49:29.460 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:49:29.460 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:49:29.460 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-04-21T08:49:29.480 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-21T08:49:29.498 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-21T08:49:29.498 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:49:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-04-21T08:49:29.556 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:49:29.556 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:49:29.556 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:49:29.556 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:49:29.556 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:49:29.556 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:49:29.556 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-21T08:49:29.572 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-04-21T08:49:29.589 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-04-21T08:49:29.590 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:49:29.647 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:49:29.648 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:49:29.648 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:49:29.648 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:49:29.648 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-04-21T08:49:29.672 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-21T08:49:29.685 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-21T08:49:29.707 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-21T08:49:29.707 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:49:29.710 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:49:29.712 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:49:29.729 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-21T08:49:29.740 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-21T08:49:29.740 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:49:29.767 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:49:29.770 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:49:29.847 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-21T08:49:29.913 INFO:teuthology.run_tasks:Running task kernel... 2024-04-21T08:49:29.928 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-21T08:49:29.928 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-21T08:49:29.928 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-21T08:49:29.929 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:49:29.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:49:29.946 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-04-21T08:49:30.005 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-437.el9.x86_64 2024-04-21T08:49:30.005 INFO:teuthology.task.kernel:Running kernel on smithi066: 5.14.0-437.el9.x86_64 2024-04-21T08:49:30.005 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum install -y kernel 2024-04-21T08:49:33.307 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-04-21T08:49:35.304 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-21T08:49:40.644 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-04-21T08:49:42.312 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-21T08:49:45.321 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 21 MB 00:02 2024-04-21T08:49:50.495 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-21T08:49:52.460 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-21T08:49:52.460 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-21T08:49:52.551 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-21T08:49:52.552 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-21T08:49:52.552 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-21T08:49:52.662 DEBUG:teuthology.orchestra.run.smithi066:> echo no | sudo yum reinstall kernel || true 2024-04-21T08:49:53.224 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 08:49:50 AM UTC. 2024-04-21T08:49:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-21T08:49:53.367 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-21T08:49:53.367 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-04-21T08:49:53.367 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-21T08:49:53.367 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalling: 2024-04-21T08:49:53.367 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:49:53.367 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:49:53.367 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-21T08:49:53.368 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-21T08:49:53.368 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:49:53.368 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 255 k 2024-04-21T08:49:53.368 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 0 2024-04-21T08:49:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:Operation aborted. 2024-04-21T08:49:53.422 INFO:teuthology.orchestra.run.smithi066.stdout:Is this ok [y/N]: 2024-04-21T08:49:53.422 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum reinstall -y kernel || true 2024-04-21T08:49:53.938 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 08:49:50 AM UTC. 2024-04-21T08:49:54.058 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalling: 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-21T08:49:54.059 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:49:54.060 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 255 k 2024-04-21T08:49:54.060 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 0 2024-04-21T08:49:54.060 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-21T08:49:54.214 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.9 MB/s | 255 kB 00:00 2024-04-21T08:49:54.215 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:49:54.215 INFO:teuthology.orchestra.run.smithi066.stdout:Total 1.6 MB/s | 255 kB 00:00 2024-04-21T08:49:54.237 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-21T08:49:54.428 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-21T08:49:54.429 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-21T08:49:54.434 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-21T08:49:54.434 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-21T08:49:54.564 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-21T08:49:54.657 INFO:teuthology.orchestra.run.smithi066.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:49:54.658 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:49:55.422 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:49:55.422 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:49:56.059 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:49:56.060 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:49:56.060 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalled: 2024-04-21T08:49:56.060 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-21T08:49:56.060 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:49:56.060 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-21T08:49:56.245 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-04-21T08:49:56.293 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-21T08:49:56.293 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-21T08:49:56.293 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-21T08:49:56.293 DEBUG:teuthology.parallel:result is None 2024-04-21T08:49:56.294 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-21T08:49:56.300 INFO:teuthology.task.internal:Creating test directory... 2024-04-21T08:49:56.301 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:49:56.303 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:49:56.305 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:49:56.323 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-21T08:49:56.378 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-21T08:49:56.413 INFO:teuthology.task.internal:Creating archive directory... 2024-04-21T08:49:56.414 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:49:56.416 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:49:56.418 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:49:56.539 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-21T08:49:56.547 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-21T08:49:56.548 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:49:56.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:49:56.564 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:49:56.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:49:56.581 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:49:56.584 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:49:56.607 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:49:56.613 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:49:56.625 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:49:56.641 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:49:56.659 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:49:56.664 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:49:56.679 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:49:56.681 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-21T08:49:56.688 INFO:teuthology.task.internal:Configuring sudo... 2024-04-21T08:49:56.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:49:56.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:49:56.704 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:49:56.748 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-21T08:49:56.757 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-21T08:49:56.757 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:49:56.759 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:49:56.771 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:49:56.805 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:49:56.848 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:49:56.902 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:49:56.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:49:57.023 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:49:57.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:49:57.087 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:49:57.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:49:57.185 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:49:57.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:49:57.310 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:49:57.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:49:57.376 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:49:57.420 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:49:57.475 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:49:57.561 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:49:57.600 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:49:57.600 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:49:57.667 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-04-21T08:49:57.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-21T08:49:57.674 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-04-21T08:49:57.706 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:49:57.707 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:49:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:49:58.160 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-21T08:49:58.168 INFO:teuthology.task.internal:Starting timer... 2024-04-21T08:49:58.168 INFO:teuthology.run_tasks:Running task pcp... 2024-04-21T08:49:58.208 INFO:teuthology.run_tasks:Running task selinux... 2024-04-21T08:49:58.240 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-21T08:49:58.240 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-04-21T08:49:58.286 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-04-21T08:49:58.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-04-21T08:49:58.353 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-04-21T08:49:58.355 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-04-21T08:49:58.435 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-04-21T08:49:58.438 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-21T08:49:58.438 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-04-21T08:49:58.460 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-04-21T08:49:58.461 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-04-21T08:49:58.503 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-04-21T08:49:58.503 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-04-21T08:49:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-04-21T08:49:58.530 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-04-21T08:49:58.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:49:58.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:49:58.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:49:58.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:49:58.583 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:49:58.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:49:58.609 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-21T08:49:58.609 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:49:58.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:49:58.659 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:49:58.687 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-21T08:49:58.697 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-21T08:49:58.702 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-21T08:49:58.844 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-21T08:49:58.859 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-21T08:49:58.881 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi083.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-21T08:54:25.908 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-04-21T08:54:25.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-21T08:54:25.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:54:25.989 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-04-21T08:54:26.073 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-21T08:54:26.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-21T08:54:26.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:54:26.149 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-21T08:54:26.227 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-21T08:54:26.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-21T08:54:26.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:54:26.354 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-04-21T08:54:26.429 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-21T08:54:26.429 INFO:teuthology.run_tasks:Running task clock... 2024-04-21T08:54:26.442 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-21T08:54:26.442 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:54:26.442 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:54:26.445 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:54:26.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:54:26.447 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:54:26.448 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:54:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:54:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:54:26.487 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:54:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:54:26.497 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:54:26.511 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:54:26.511 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-04-21T08:54:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-04-21T08:54:26.521 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-04-21T08:54:26.522 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-04-21T08:54:26.533 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-04-21T08:54:26.534 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:54:26.536 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:54:26.543 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-04-21T08:54:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:54:26.550 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:54:26.556 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:54:26.569 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:54:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:54:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:54:26.609 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:54:26.609 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-21T08:54:26.609 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.610 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.610 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.610 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.611 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:54:26.611 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-21T08:54:26.611 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.612 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.612 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.612 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.622 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:54:26.625 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:54:26.625 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-21T08:54:26.625 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.625 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.626 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.626 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:26.627 INFO:teuthology.run_tasks:Running task install... 2024-04-21T08:54:26.637 DEBUG:teuthology.task.install:project ceph 2024-04-21T08:54:26.637 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'}} 2024-04-21T08:54:26.637 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'} 2024-04-21T08:54:26.638 INFO:teuthology.task.install:Using flavor: default 2024-04-21T08:54:26.643 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-21T08:54:26.643 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-21T08:54:26.644 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:54:26.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:54:26.646 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:54:26.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:54:26.647 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:54:26.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:54:26.788 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:54:26.789 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:54:26.859 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:54:26.860 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:54:26.865 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:54:26.865 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:54:26.962 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:54:26.962 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:54:26.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:54:26.967 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:54:26.967 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:54:26.967 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:54:26.978 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:54:26.978 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:54:26.978 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:54:26.987 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:54:26.987 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:54:26.993 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:54:26.993 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:54:27.002 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:54:27.003 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:54:27.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:54:27.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:54:27.067 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:54:27.124 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:54:27.132 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:54:27.141 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:54:27.187 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-04-21T08:54:27.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-04-21T08:54:27.196 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-04-21T08:54:27.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-04-21T08:54:27.204 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-04-21T08:54:27.205 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-04-21T08:54:27.411 INFO:teuthology.orchestra.run.smithi083.stdout:48 files removed 2024-04-21T08:54:27.431 INFO:teuthology.orchestra.run.smithi186.stdout:48 files removed 2024-04-21T08:54:27.431 INFO:teuthology.orchestra.run.smithi066.stdout:48 files removed 2024-04-21T08:54:27.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:54:27.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:54:27.459 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:54:28.035 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 246 kB/s | 83 kB 00:00 2024-04-21T08:54:28.179 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 187 kB/s | 83 kB 00:00 2024-04-21T08:54:28.221 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 166 kB/s | 83 kB 00:00 2024-04-21T08:54:28.594 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-21T08:54:28.731 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-21T08:54:28.806 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-21T08:54:29.253 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-21T08:54:29.406 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-04-21T08:54:29.422 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-04-21T08:54:29.972 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-21T08:54:30.003 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-21T08:54:31.147 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-21T08:54:31.924 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-21T08:54:33.774 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-21T08:54:34.795 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 19 MB 00:03 2024-04-21T08:54:37.599 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-21T08:54:38.938 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-21T08:54:39.199 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-04-21T08:54:39.308 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-21T08:54:40.514 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-21T08:54:40.791 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-21T08:54:41.387 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-21T08:54:42.271 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-21T08:54:43.665 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 21 MB 00:03 2024-04-21T08:54:46.652 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-21T08:54:47.343 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-21T08:54:48.799 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-21T08:54:48.903 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:54:48.904 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:54:48.916 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:54:48.916 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:54:48.917 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:54:48.918 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:54:48.920 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:54:48.922 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:54:48.923 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:54:48.987 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:54:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:54:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:54:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-21T08:54:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:54:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:54:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:54:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:54:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:54:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:54:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:54:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:54:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-21T08:54:49.008 INFO:teuthology.orchestra.run.smithi066.stdout:Install 148 Packages 2024-04-21T08:54:49.009 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 2 Packages 2024-04-21T08:54:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:54:49.009 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 218 M 2024-04-21T08:54:49.009 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-21T08:54:49.519 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:54:49.519 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:54:49.530 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:54:49.531 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:54:49.532 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:54:49.533 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:54:49.534 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:54:49.536 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:54:49.537 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:54:49.598 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-21T08:54:49.619 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:54:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:54:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:54:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:54:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:54:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:54:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:54:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:54:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:54:49.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:54:49.629 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:54:49.630 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-21T08:54:49.631 INFO:teuthology.orchestra.run.smithi186.stdout:Install 148 Packages 2024-04-21T08:54:49.632 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 2 Packages 2024-04-21T08:54:49.632 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:54:49.632 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 218 M 2024-04-21T08:54:49.632 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-21T08:54:50.036 INFO:teuthology.orchestra.run.smithi066.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-21T08:54:50.461 INFO:teuthology.orchestra.run.smithi066.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 2.0 MB/s | 890 kB 00:00 2024-04-21T08:54:50.537 INFO:teuthology.orchestra.run.smithi066.stdout:(3/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 7.1 MB/s | 5.4 MB 00:00 2024-04-21T08:54:50.562 INFO:teuthology.orchestra.run.smithi066.stdout:(4/150): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-21T08:54:50.652 INFO:teuthology.orchestra.run.smithi186.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-21T08:54:50.804 INFO:teuthology.orchestra.run.smithi066.stdout:(5/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 8.8 MB/s | 2.4 MB 00:00 2024-04-21T08:54:50.941 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:54:50.942 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:54:50.952 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:54:50.953 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:54:50.954 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:54:50.955 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:54:50.956 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:54:50.958 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:54:50.959 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:54:51.018 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-21T08:54:51.034 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-21T08:54:51.034 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-04-21T08:54:51.034 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-21T08:54:51.034 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-04-21T08:54:51.034 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:54:51.035 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:54:51.036 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:54:51.037 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:54:51.038 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:54:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:54:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:54:51.041 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:54:51.042 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:54:51.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:54:51.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:54:51.045 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:54:51.046 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:54:51.047 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-21T08:54:51.047 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-21T08:54:51.047 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-04-21T08:54:51.047 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 2 Packages 2024-04-21T08:54:51.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:54:51.047 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 218 M 2024-04-21T08:54:51.047 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-21T08:54:51.070 INFO:teuthology.orchestra.run.smithi066.stdout:(6/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-04-21T08:54:51.346 INFO:teuthology.orchestra.run.smithi066.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 8.7 MB/s | 4.7 MB 00:00 2024-04-21T08:54:51.586 INFO:teuthology.orchestra.run.smithi186.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 954 kB/s | 890 kB 00:00 2024-04-21T08:54:51.694 INFO:teuthology.orchestra.run.smithi186.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.3 MB/s | 145 kB 00:00 2024-04-21T08:54:51.746 INFO:teuthology.orchestra.run.smithi066.stdout:(8/150): ceph-common-19.0.0-3154.ga630765c.el9. 11 MB/s | 21 MB 00:01 2024-04-21T08:54:51.813 INFO:teuthology.orchestra.run.smithi066.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 378 kB/s | 25 kB 00:00 2024-04-21T08:54:52.278 INFO:teuthology.orchestra.run.smithi186.stdout:(4/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-04-21T08:54:52.500 INFO:teuthology.orchestra.run.smithi083.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-21T08:54:52.795 INFO:teuthology.orchestra.run.smithi186.stdout:(5/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-04-21T08:54:52.928 INFO:teuthology.orchestra.run.smithi186.stdout:(6/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-21T08:54:53.092 INFO:teuthology.orchestra.run.smithi083.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 1.5 MB/s | 890 kB 00:00 2024-04-21T08:54:53.222 INFO:teuthology.orchestra.run.smithi083.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.1 MB/s | 145 kB 00:00 2024-04-21T08:54:53.638 INFO:teuthology.orchestra.run.smithi066.stdout:(10/150): ceph-radosgw-19.0.0-3154.ga630765c.el 4.6 MB/s | 11 MB 00:02 2024-04-21T08:54:54.128 INFO:teuthology.orchestra.run.smithi083.stdout:(4/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-04-21T08:54:54.196 INFO:teuthology.orchestra.run.smithi066.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 403 kB/s | 225 kB 00:00 2024-04-21T08:54:54.272 INFO:teuthology.orchestra.run.smithi066.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 361 kB/s | 27 kB 00:00 2024-04-21T08:54:54.304 INFO:teuthology.orchestra.run.smithi186.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 3.1 MB/s | 4.7 MB 00:01 2024-04-21T08:54:54.322 INFO:teuthology.orchestra.run.smithi066.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 637 kB/s | 32 kB 00:00 2024-04-21T08:54:54.498 INFO:teuthology.orchestra.run.smithi066.stdout:(14/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 4.9 MB/s | 17 MB 00:03 2024-04-21T08:54:54.611 INFO:teuthology.orchestra.run.smithi083.stdout:(5/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-21T08:54:54.614 INFO:teuthology.orchestra.run.smithi066.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 1.4 MB/s | 163 kB 00:00 2024-04-21T08:54:54.740 INFO:teuthology.orchestra.run.smithi066.stdout:(16/150): libcephfs2-19.0.0-3154.ga630765c.el9. 1.7 MB/s | 746 kB 00:00 2024-04-21T08:54:54.765 INFO:teuthology.orchestra.run.smithi066.stdout:(17/150): librados-devel-19.0.0-3154.ga630765c. 844 kB/s | 127 kB 00:00 2024-04-21T08:54:54.990 INFO:teuthology.orchestra.run.smithi066.stdout:(18/150): libradosstriper1-19.0.0-3154.ga630765 2.0 MB/s | 501 kB 00:00 2024-04-21T08:54:55.053 INFO:teuthology.orchestra.run.smithi083.stdout:(6/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-21T08:54:55.090 INFO:teuthology.orchestra.run.smithi066.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 452 kB/s | 45 kB 00:00 2024-04-21T08:54:55.182 INFO:teuthology.orchestra.run.smithi066.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 1.4 MB/s | 135 kB 00:00 2024-04-21T08:54:55.274 INFO:teuthology.orchestra.run.smithi066.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 1.7 MB/s | 162 kB 00:00 2024-04-21T08:54:55.449 INFO:teuthology.orchestra.run.smithi066.stdout:(22/150): python3-rados-19.0.0-3154.ga630765c.e 1.8 MB/s | 321 kB 00:00 2024-04-21T08:54:55.625 INFO:teuthology.orchestra.run.smithi066.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 1.7 MB/s | 298 kB 00:00 2024-04-21T08:54:55.725 INFO:teuthology.orchestra.run.smithi066.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 998 kB/s | 100 kB 00:00 2024-04-21T08:54:55.834 INFO:teuthology.orchestra.run.smithi066.stdout:(25/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 786 kB/s | 85 kB 00:00 2024-04-21T08:54:56.287 INFO:teuthology.orchestra.run.smithi186.stdout:(8/150): ceph-osd-19.0.0-3154.ga630765c.el9.x86 5.0 MB/s | 17 MB 00:03 2024-04-21T08:54:56.387 INFO:teuthology.orchestra.run.smithi186.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 252 kB/s | 25 kB 00:00 2024-04-21T08:54:56.426 INFO:teuthology.orchestra.run.smithi066.stdout:(26/150): librgw2-19.0.0-3154.ga630765c.el9.x86 3.2 MB/s | 5.4 MB 00:01 2024-04-21T08:54:56.453 INFO:teuthology.orchestra.run.smithi083.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 2.6 MB/s | 4.7 MB 00:01 2024-04-21T08:54:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-21T08:54:56.626 INFO:teuthology.orchestra.run.smithi066.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 316 kB/s | 34 kB 00:00 2024-04-21T08:54:56.735 INFO:teuthology.orchestra.run.smithi066.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.3 MB/s | 142 kB 00:00 2024-04-21T08:54:56.762 INFO:teuthology.orchestra.run.smithi186.stdout:(10/150): ceph-radosgw-19.0.0-3154.ga630765c.el 4.3 MB/s | 11 MB 00:02 2024-04-21T08:54:56.863 INFO:teuthology.orchestra.run.smithi186.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 2.2 MB/s | 225 kB 00:00 2024-04-21T08:54:56.943 INFO:teuthology.orchestra.run.smithi066.stdout:(30/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 2.8 MB/s | 3.1 MB 00:01 2024-04-21T08:54:56.954 INFO:teuthology.orchestra.run.smithi186.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 295 kB/s | 27 kB 00:00 2024-04-21T08:54:57.038 INFO:teuthology.orchestra.run.smithi186.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 381 kB/s | 32 kB 00:00 2024-04-21T08:54:57.288 INFO:teuthology.orchestra.run.smithi186.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 2.9 MB/s | 746 kB 00:00 2024-04-21T08:54:57.372 INFO:teuthology.orchestra.run.smithi186.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 1.9 MB/s | 163 kB 00:00 2024-04-21T08:54:57.464 INFO:teuthology.orchestra.run.smithi186.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.4 MB/s | 127 kB 00:00 2024-04-21T08:54:57.622 INFO:teuthology.orchestra.run.smithi186.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 3.1 MB/s | 501 kB 00:00 2024-04-21T08:54:57.777 INFO:teuthology.orchestra.run.smithi066.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 3.4 MB/s | 3.5 MB 00:01 2024-04-21T08:54:57.894 INFO:teuthology.orchestra.run.smithi066.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 2.1 MB/s | 249 kB 00:00 2024-04-21T08:54:57.995 INFO:teuthology.orchestra.run.smithi066.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 492 kB/s | 49 kB 00:00 2024-04-21T08:54:58.095 INFO:teuthology.orchestra.run.smithi066.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 165 kB/s | 16 kB 00:00 2024-04-21T08:54:58.203 INFO:teuthology.orchestra.run.smithi066.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 2.5 MB/s | 273 kB 00:00 2024-04-21T08:54:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 2.7 MB/s | 762 kB 00:00 2024-04-21T08:54:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 219 kB/s | 18 kB 00:00 2024-04-21T08:54:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:(18/150): ceph-common-19.0.0-3154.ga630765c.el9 2.6 MB/s | 21 MB 00:08 2024-04-21T08:54:58.665 INFO:teuthology.orchestra.run.smithi186.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 493 kB/s | 45 kB 00:00 2024-04-21T08:54:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-04-21T08:54:58.788 INFO:teuthology.orchestra.run.smithi066.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-21T08:54:58.806 INFO:teuthology.orchestra.run.smithi186.stdout:(20/150): librgw2-19.0.0-3154.ga630765c.el9.x86 4.5 MB/s | 5.4 MB 00:01 2024-04-21T08:54:58.832 INFO:teuthology.orchestra.run.smithi186.stdout:(21/150): python3-ceph-common-19.0.0-3154.ga630 808 kB/s | 135 kB 00:00 2024-04-21T08:54:58.838 INFO:teuthology.orchestra.run.smithi066.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-21T08:54:58.874 INFO:teuthology.orchestra.run.smithi186.stdout:(22/150): python3-cephfs-19.0.0-3154.ga630765c. 2.4 MB/s | 162 kB 00:00 2024-04-21T08:54:58.888 INFO:teuthology.orchestra.run.smithi066.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi186.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 3.5 MB/s | 298 kB 00:00 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi066.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi186.stdout:(24/150): python3-rados-19.0.0-3154.ga630765c.e 2.0 MB/s | 321 kB 00:00 2024-04-21T08:54:59.016 INFO:teuthology.orchestra.run.smithi186.stdout:(25/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.7 MB/s | 100 kB 00:00 2024-04-21T08:54:59.048 INFO:teuthology.orchestra.run.smithi066.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-21T08:54:59.075 INFO:teuthology.orchestra.run.smithi186.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-21T08:54:59.098 INFO:teuthology.orchestra.run.smithi066.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-21T08:54:59.148 INFO:teuthology.orchestra.run.smithi066.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-21T08:54:59.175 INFO:teuthology.orchestra.run.smithi186.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-21T08:54:59.198 INFO:teuthology.orchestra.run.smithi066.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-21T08:54:59.259 INFO:teuthology.orchestra.run.smithi186.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 410 kB/s | 34 kB 00:00 2024-04-21T08:54:59.299 INFO:teuthology.orchestra.run.smithi066.stdout:(47/150): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-04-21T08:54:59.351 INFO:teuthology.orchestra.run.smithi186.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.5 MB/s | 142 kB 00:00 2024-04-21T08:54:59.524 INFO:teuthology.orchestra.run.smithi066.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi066.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi066.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi186.stdout:(30/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 5.2 MB/s | 3.1 MB 00:00 2024-04-21T08:54:59.649 INFO:teuthology.orchestra.run.smithi066.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-21T08:54:59.675 INFO:teuthology.orchestra.run.smithi066.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-21T08:54:59.717 INFO:teuthology.orchestra.run.smithi066.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-04-21T08:54:59.801 INFO:teuthology.orchestra.run.smithi066.stdout:(54/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-21T08:54:59.826 INFO:teuthology.orchestra.run.smithi066.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi066.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-21T08:54:59.893 INFO:teuthology.orchestra.run.smithi066.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-21T08:54:59.935 INFO:teuthology.orchestra.run.smithi066.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-04-21T08:54:59.969 INFO:teuthology.orchestra.run.smithi066.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-21T08:55:00.019 INFO:teuthology.orchestra.run.smithi066.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-21T08:55:00.053 INFO:teuthology.orchestra.run.smithi066.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-21T08:55:00.095 INFO:teuthology.orchestra.run.smithi066.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-21T08:55:00.120 INFO:teuthology.orchestra.run.smithi066.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-21T08:55:00.162 INFO:teuthology.orchestra.run.smithi066.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-21T08:55:00.179 INFO:teuthology.orchestra.run.smithi083.stdout:(8/150): ceph-radosgw-19.0.0-3154.ga630765c.el9 2.8 MB/s | 11 MB 00:03 2024-04-21T08:55:00.187 INFO:teuthology.orchestra.run.smithi066.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-21T08:55:00.253 INFO:teuthology.orchestra.run.smithi083.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 337 kB/s | 25 kB 00:00 2024-04-21T08:55:00.354 INFO:teuthology.orchestra.run.smithi066.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-21T08:55:00.368 INFO:teuthology.orchestra.run.smithi186.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 3.5 MB/s | 3.5 MB 00:01 2024-04-21T08:55:00.429 INFO:teuthology.orchestra.run.smithi066.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 17 MB/s | 4.1 MB 00:00 2024-04-21T08:55:00.468 INFO:teuthology.orchestra.run.smithi186.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 2.4 MB/s | 249 kB 00:00 2024-04-21T08:55:00.504 INFO:teuthology.orchestra.run.smithi066.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-21T08:55:00.538 INFO:teuthology.orchestra.run.smithi066.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-21T08:55:00.560 INFO:teuthology.orchestra.run.smithi186.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 538 kB/s | 49 kB 00:00 2024-04-21T08:55:00.572 INFO:teuthology.orchestra.run.smithi066.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-21T08:55:00.660 INFO:teuthology.orchestra.run.smithi186.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 164 kB/s | 16 kB 00:00 2024-04-21T08:55:00.663 INFO:teuthology.orchestra.run.smithi066.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-21T08:55:00.739 INFO:teuthology.orchestra.run.smithi066.stdout:(72/150): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-21T08:55:00.763 INFO:teuthology.orchestra.run.smithi186.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 2.6 MB/s | 273 kB 00:00 2024-04-21T08:55:00.772 INFO:teuthology.orchestra.run.smithi066.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 805 kB/s | 87 kB 00:00 2024-04-21T08:55:00.913 INFO:teuthology.orchestra.run.smithi186.stdout:(36/150): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-04-21T08:55:00.972 INFO:teuthology.orchestra.run.smithi186.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 313 kB/s | 18 kB 00:00 2024-04-21T08:55:01.005 INFO:teuthology.orchestra.run.smithi186.stdout:(38/150): cephadm-19.0.0-3154.ga630765c.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-21T08:55:01.122 INFO:teuthology.orchestra.run.smithi186.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-04-21T08:55:01.206 INFO:teuthology.orchestra.run.smithi186.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-21T08:55:01.207 INFO:teuthology.orchestra.run.smithi066.stdout:(74/150): ceph-test-19.0.0-3154.ga630765c.el9.x 5.2 MB/s | 49 MB 00:09 2024-04-21T08:55:01.256 INFO:teuthology.orchestra.run.smithi066.stdout:(75/150): python3-devel-3.9.18-3.el9.x86_64.rpm 505 kB/s | 244 kB 00:00 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi186.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-21T08:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout:(10/150): ceph-common-19.0.0-3154.ga630765c.el9 2.3 MB/s | 21 MB 00:09 2024-04-21T08:55:01.373 INFO:teuthology.orchestra.run.smithi066.stdout:(76/150): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-04-21T08:55:01.390 INFO:teuthology.orchestra.run.smithi066.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 244 kB/s | 26 kB 00:00 2024-04-21T08:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 2.2 MB/s | 225 kB 00:00 2024-04-21T08:55:01.432 INFO:teuthology.orchestra.run.smithi066.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 334 kB/s | 19 kB 00:00 2024-04-21T08:55:01.440 INFO:teuthology.orchestra.run.smithi186.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-21T08:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 538 kB/s | 27 kB 00:00 2024-04-21T08:55:01.482 INFO:teuthology.orchestra.run.smithi066.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-21T08:55:01.498 INFO:teuthology.orchestra.run.smithi186.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-21T08:55:01.499 INFO:teuthology.orchestra.run.smithi066.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-04-21T08:55:01.525 INFO:teuthology.orchestra.run.smithi066.stdout:(81/150): python3-jinja2-2.11.3-5.el9.noarch.rp 780 kB/s | 248 kB 00:00 2024-04-21T08:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 636 kB/s | 32 kB 00:00 2024-04-21T08:55:01.582 INFO:teuthology.orchestra.run.smithi186.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-21T08:55:01.633 INFO:teuthology.orchestra.run.smithi066.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-21T08:55:01.659 INFO:teuthology.orchestra.run.smithi066.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-04-21T08:55:01.674 INFO:teuthology.orchestra.run.smithi186.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-21T08:55:01.689 INFO:teuthology.orchestra.run.smithi083.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 4.6 MB/s | 746 kB 00:00 2024-04-21T08:55:01.692 INFO:teuthology.orchestra.run.smithi066.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-21T08:55:01.718 INFO:teuthology.orchestra.run.smithi066.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-04-21T08:55:01.747 INFO:teuthology.orchestra.run.smithi083.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 2.7 MB/s | 163 kB 00:00 2024-04-21T08:55:01.757 INFO:teuthology.orchestra.run.smithi186.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-21T08:55:01.791 INFO:teuthology.orchestra.run.smithi186.stdout:(47/150): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-04-21T08:55:01.793 INFO:teuthology.orchestra.run.smithi066.stdout:(86/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-21T08:55:01.797 INFO:teuthology.orchestra.run.smithi083.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 2.5 MB/s | 127 kB 00:00 2024-04-21T08:55:01.818 INFO:teuthology.orchestra.run.smithi066.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-21T08:55:01.844 INFO:teuthology.orchestra.run.smithi066.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-21T08:55:01.869 INFO:teuthology.orchestra.run.smithi066.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-21T08:55:01.894 INFO:teuthology.orchestra.run.smithi066.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-04-21T08:55:01.914 INFO:teuthology.orchestra.run.smithi083.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 4.2 MB/s | 501 kB 00:00 2024-04-21T08:55:01.919 INFO:teuthology.orchestra.run.smithi066.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-21T08:55:01.978 INFO:teuthology.orchestra.run.smithi066.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 500 kB/s | 42 kB 00:00 2024-04-21T08:55:02.011 INFO:teuthology.orchestra.run.smithi066.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-04-21T08:55:02.087 INFO:teuthology.orchestra.run.smithi066.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-21T08:55:02.104 INFO:teuthology.orchestra.run.smithi066.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-04-21T08:55:02.145 INFO:teuthology.orchestra.run.smithi066.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-04-21T08:55:02.212 INFO:teuthology.orchestra.run.smithi066.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-21T08:55:02.229 INFO:teuthology.orchestra.run.smithi066.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 691 kB/s | 58 kB 00:00 2024-04-21T08:55:02.397 INFO:teuthology.orchestra.run.smithi066.stdout:(99/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-21T08:55:02.422 INFO:teuthology.orchestra.run.smithi066.stdout:(100/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 449 kB/s | 94 kB 00:00 2024-04-21T08:55:02.447 INFO:teuthology.orchestra.run.smithi066.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 293 kB/s | 64 kB 00:00 2024-04-21T08:55:02.472 INFO:teuthology.orchestra.run.smithi066.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-04-21T08:55:02.514 INFO:teuthology.orchestra.run.smithi066.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-04-21T08:55:02.648 INFO:teuthology.orchestra.run.smithi066.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-04-21T08:55:02.731 INFO:teuthology.orchestra.run.smithi083.stdout:(18/150): librgw2-19.0.0-3154.ga630765c.el9.x86 6.6 MB/s | 5.4 MB 00:00 2024-04-21T08:55:02.782 INFO:teuthology.orchestra.run.smithi083.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 908 kB/s | 45 kB 00:00 2024-04-21T08:55:02.825 INFO:teuthology.orchestra.run.smithi186.stdout:(48/150): ceph-test-19.0.0-3154.ga630765c.el9.x 7.6 MB/s | 49 MB 00:06 2024-04-21T08:55:02.840 INFO:teuthology.orchestra.run.smithi083.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 2.3 MB/s | 135 kB 00:00 2024-04-21T08:55:02.858 INFO:teuthology.orchestra.run.smithi186.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 626 kB/s | 689 kB 00:01 2024-04-21T08:55:02.884 INFO:teuthology.orchestra.run.smithi186.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 863 kB/s | 943 kB 00:01 2024-04-21T08:55:02.941 INFO:teuthology.orchestra.run.smithi066.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 66 kB/s | 19 kB 00:00 2024-04-21T08:55:02.959 INFO:teuthology.orchestra.run.smithi186.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-04-21T08:55:02.993 INFO:teuthology.orchestra.run.smithi186.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-21T08:55:03.040 INFO:teuthology.orchestra.run.smithi066.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 588 kB/s | 309 kB 00:00 2024-04-21T08:55:03.051 INFO:teuthology.orchestra.run.smithi186.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-04-21T08:55:03.066 INFO:teuthology.orchestra.run.smithi083.stdout:(21/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 2.1 MB/s | 17 MB 00:08 2024-04-21T08:55:03.091 INFO:teuthology.orchestra.run.smithi083.stdout:(22/150): python3-cephfs-19.0.0-3154.ga630765c. 645 kB/s | 162 kB 00:00 2024-04-21T08:55:03.148 INFO:teuthology.orchestra.run.smithi066.stdout:(107/150): abseil-cpp-20211102.0-3.el9.x86_64.r 811 kB/s | 548 kB 00:00 2024-04-21T08:55:03.158 INFO:teuthology.orchestra.run.smithi083.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 4.4 MB/s | 298 kB 00:00 2024-04-21T08:55:03.159 INFO:teuthology.orchestra.run.smithi186.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-21T08:55:03.174 INFO:teuthology.orchestra.run.smithi066.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 194 kB/s | 26 kB 00:00 2024-04-21T08:55:03.217 INFO:teuthology.orchestra.run.smithi083.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.7 MB/s | 100 kB 00:00 2024-04-21T08:55:03.241 INFO:teuthology.orchestra.run.smithi066.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-04-21T08:55:03.242 INFO:teuthology.orchestra.run.smithi083.stdout:(25/150): python3-rados-19.0.0-3154.ga630765c.e 1.8 MB/s | 321 kB 00:00 2024-04-21T08:55:03.251 INFO:teuthology.orchestra.run.smithi186.stdout:(55/150): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-04-21T08:55:03.268 INFO:teuthology.orchestra.run.smithi083.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-21T08:55:03.277 INFO:teuthology.orchestra.run.smithi186.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-04-21T08:55:03.318 INFO:teuthology.orchestra.run.smithi083.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 3.3 MB/s | 171 kB 00:00 2024-04-21T08:55:03.333 INFO:teuthology.orchestra.run.smithi066.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 425 kB/s | 67 kB 00:00 2024-04-21T08:55:03.368 INFO:teuthology.orchestra.run.smithi083.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 685 kB/s | 34 kB 00:00 2024-04-21T08:55:03.369 INFO:teuthology.orchestra.run.smithi186.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-21T08:55:03.394 INFO:teuthology.orchestra.run.smithi186.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-04-21T08:55:03.419 INFO:teuthology.orchestra.run.smithi083.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 2.8 MB/s | 142 kB 00:00 2024-04-21T08:55:03.425 INFO:teuthology.orchestra.run.smithi066.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 823 kB/s | 151 kB 00:00 2024-04-21T08:55:03.469 INFO:teuthology.orchestra.run.smithi186.stdout:(59/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.3 MB/s | 3.0 MB 00:00 2024-04-21T08:55:03.569 INFO:teuthology.orchestra.run.smithi186.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-04-21T08:55:03.645 INFO:teuthology.orchestra.run.smithi186.stdout:(61/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-04-21T08:55:03.658 INFO:teuthology.orchestra.run.smithi066.stdout:(112/150): libarrow-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-04-21T08:55:03.670 INFO:teuthology.orchestra.run.smithi186.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-21T08:55:03.683 INFO:teuthology.orchestra.run.smithi066.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-04-21T08:55:03.709 INFO:teuthology.orchestra.run.smithi066.stdout:(114/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-04-21T08:55:03.753 INFO:teuthology.orchestra.run.smithi186.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-21T08:55:03.767 INFO:teuthology.orchestra.run.smithi066.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-04-21T08:55:03.787 INFO:teuthology.orchestra.run.smithi186.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-21T08:55:03.793 INFO:teuthology.orchestra.run.smithi066.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-21T08:55:03.821 INFO:teuthology.orchestra.run.smithi186.stdout:(65/150): libtool-2.4.6-45.el9.x86_64.rpm 1.3 MB/s | 582 kB 00:00 2024-04-21T08:55:03.843 INFO:teuthology.orchestra.run.smithi066.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-04-21T08:55:03.854 INFO:teuthology.orchestra.run.smithi186.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 349 kB/s | 35 kB 00:00 2024-04-21T08:55:03.886 INFO:teuthology.orchestra.run.smithi083.stdout:(30/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 7.6 MB/s | 3.5 MB 00:00 2024-04-21T08:55:03.901 INFO:teuthology.orchestra.run.smithi066.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-21T08:55:03.927 INFO:teuthology.orchestra.run.smithi066.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-21T08:55:03.994 INFO:teuthology.orchestra.run.smithi066.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-04-21T08:55:04.019 INFO:teuthology.orchestra.run.smithi066.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-04-21T08:55:04.020 INFO:teuthology.orchestra.run.smithi083.stdout:(31/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 4.0 MB/s | 3.1 MB 00:00 2024-04-21T08:55:04.061 INFO:teuthology.orchestra.run.smithi066.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-04-21T08:55:04.128 INFO:teuthology.orchestra.run.smithi083.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 2.2 MB/s | 249 kB 00:00 2024-04-21T08:55:04.138 INFO:teuthology.orchestra.run.smithi186.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-04-21T08:55:04.188 INFO:teuthology.orchestra.run.smithi066.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 861 kB/s | 144 kB 00:00 2024-04-21T08:55:04.195 INFO:teuthology.orchestra.run.smithi083.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 741 kB/s | 49 kB 00:00 2024-04-21T08:55:04.211 INFO:teuthology.orchestra.run.smithi066.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2024-04-21T08:55:04.262 INFO:teuthology.orchestra.run.smithi066.stdout:(125/150): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-04-21T08:55:04.262 INFO:teuthology.orchestra.run.smithi083.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 247 kB/s | 16 kB 00:00 2024-04-21T08:55:04.264 INFO:teuthology.orchestra.run.smithi186.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-04-21T08:55:04.287 INFO:teuthology.orchestra.run.smithi066.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 178 kB/s | 18 kB 00:00 2024-04-21T08:55:04.312 INFO:teuthology.orchestra.run.smithi066.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-21T08:55:04.362 INFO:teuthology.orchestra.run.smithi066.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-21T08:55:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 2.5 MB/s | 273 kB 00:00 2024-04-21T08:55:04.372 INFO:teuthology.orchestra.run.smithi186.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-21T08:55:04.388 INFO:teuthology.orchestra.run.smithi066.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-21T08:55:04.480 INFO:teuthology.orchestra.run.smithi066.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-21T08:55:04.505 INFO:teuthology.orchestra.run.smithi066.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 676 kB/s | 79 kB 00:00 2024-04-21T08:55:04.572 INFO:teuthology.orchestra.run.smithi066.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-04-21T08:55:04.615 INFO:teuthology.orchestra.run.smithi083.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-21T08:55:04.622 INFO:teuthology.orchestra.run.smithi066.stdout:(133/150): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-04-21T08:55:04.648 INFO:teuthology.orchestra.run.smithi066.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-21T08:55:04.679 INFO:teuthology.orchestra.run.smithi083.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 283 kB/s | 18 kB 00:00 2024-04-21T08:55:04.681 INFO:teuthology.orchestra.run.smithi066.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-21T08:55:04.715 INFO:teuthology.orchestra.run.smithi066.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-04-21T08:55:04.739 INFO:teuthology.orchestra.run.smithi066.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-21T08:55:04.755 INFO:teuthology.orchestra.run.smithi186.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.4 MB/s | 4.1 MB 00:00 2024-04-21T08:55:04.788 INFO:teuthology.orchestra.run.smithi083.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-04-21T08:55:04.790 INFO:teuthology.orchestra.run.smithi066.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-04-21T08:55:04.840 INFO:teuthology.orchestra.run.smithi066.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-21T08:55:04.865 INFO:teuthology.orchestra.run.smithi066.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2024-04-21T08:55:04.891 INFO:teuthology.orchestra.run.smithi066.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-21T08:55:04.930 INFO:teuthology.orchestra.run.smithi083.stdout:(39/150): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-04-21T08:55:04.933 INFO:teuthology.orchestra.run.smithi066.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-04-21T08:55:04.980 INFO:teuthology.orchestra.run.smithi083.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-21T08:55:04.983 INFO:teuthology.orchestra.run.smithi066.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-21T08:55:05.009 INFO:teuthology.orchestra.run.smithi066.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 764 kB/s | 90 kB 00:00 2024-04-21T08:55:05.022 INFO:teuthology.orchestra.run.smithi083.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-21T08:55:05.067 INFO:teuthology.orchestra.run.smithi066.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-21T08:55:05.089 INFO:teuthology.orchestra.run.smithi083.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-21T08:55:05.092 INFO:teuthology.orchestra.run.smithi066.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-04-21T08:55:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:(43/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-21T08:55:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-21T08:55:05.159 INFO:teuthology.orchestra.run.smithi066.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-21T08:55:05.165 INFO:teuthology.orchestra.run.smithi083.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-21T08:55:05.191 INFO:teuthology.orchestra.run.smithi083.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-21T08:55:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-21T08:55:05.401 INFO:teuthology.orchestra.run.smithi066.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-04-21T08:55:05.618 INFO:teuthology.orchestra.run.smithi066.stdout:(149/150): librbd1-19.0.0-3154.ga630765c.el9.x8 6.9 MB/s | 3.1 MB 00:00 2024-04-21T08:55:05.668 INFO:teuthology.orchestra.run.smithi066.stdout:(150/150): librados2-19.0.0-3154.ga630765c.el9. 5.9 MB/s | 3.4 MB 00:00 2024-04-21T08:55:05.673 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:55:05.673 INFO:teuthology.orchestra.run.smithi066.stdout:Total 13 MB/s | 218 MB 00:16 2024-04-21T08:55:06.039 INFO:teuthology.orchestra.run.smithi186.stdout:(71/150): postgresql-server-13.14-1.el9.x86_64. 3.5 MB/s | 5.8 MB 00:01 2024-04-21T08:55:06.139 INFO:teuthology.orchestra.run.smithi186.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 879 kB/s | 87 kB 00:00 2024-04-21T08:55:06.256 INFO:teuthology.orchestra.run.smithi186.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 689 kB/s | 1.0 MB 00:01 2024-04-21T08:55:06.408 INFO:teuthology.orchestra.run.smithi083.stdout:(48/150): ceph-test-19.0.0-3154.ga630765c.el9.x 8.0 MB/s | 49 MB 00:06 2024-04-21T08:55:06.441 INFO:teuthology.orchestra.run.smithi083.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 551 kB/s | 689 kB 00:01 2024-04-21T08:55:06.465 INFO:teuthology.orchestra.run.smithi186.stdout:(74/150): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-21T08:55:06.467 INFO:teuthology.orchestra.run.smithi083.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 754 kB/s | 943 kB 00:01 2024-04-21T08:55:06.542 INFO:teuthology.orchestra.run.smithi083.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-04-21T08:55:06.573 INFO:teuthology.orchestra.run.smithi186.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-21T08:55:06.640 INFO:teuthology.orchestra.run.smithi186.stdout:(76/150): python3-devel-3.9.18-3.el9.x86_64.rpm 636 kB/s | 244 kB 00:00 2024-04-21T08:55:06.659 INFO:teuthology.orchestra.run.smithi083.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-04-21T08:55:06.674 INFO:teuthology.orchestra.run.smithi186.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-04-21T08:55:06.684 INFO:teuthology.orchestra.run.smithi083.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-04-21T08:55:06.741 INFO:teuthology.orchestra.run.smithi186.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-21T08:55:06.780 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-21T08:55:06.783 INFO:teuthology.orchestra.run.smithi186.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-21T08:55:06.793 INFO:teuthology.orchestra.run.smithi083.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-21T08:55:06.827 INFO:teuthology.orchestra.run.smithi083.stdout:(55/150): boost-program-options-1.75.0-8.el9.x8 255 kB/s | 107 kB 00:00 2024-04-21T08:55:06.850 INFO:teuthology.orchestra.run.smithi186.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-21T08:55:06.919 INFO:teuthology.orchestra.run.smithi083.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-21T08:55:06.924 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-21T08:55:06.924 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-21T08:55:07.011 INFO:teuthology.orchestra.run.smithi083.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 872 kB/s | 160 kB 00:00 2024-04-21T08:55:07.025 INFO:teuthology.orchestra.run.smithi186.stdout:(81/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-21T08:55:07.036 INFO:teuthology.orchestra.run.smithi083.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-04-21T08:55:07.134 INFO:teuthology.orchestra.run.smithi186.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-21T08:55:07.153 INFO:teuthology.orchestra.run.smithi083.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-21T08:55:07.234 INFO:teuthology.orchestra.run.smithi186.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 772 kB/s | 77 kB 00:00 2024-04-21T08:55:07.253 INFO:teuthology.orchestra.run.smithi083.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-21T08:55:07.259 INFO:teuthology.orchestra.run.smithi186.stdout:(84/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 612 kB/s | 250 kB 00:00 2024-04-21T08:55:07.278 INFO:teuthology.orchestra.run.smithi083.stdout:(61/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-04-21T08:55:07.329 INFO:teuthology.orchestra.run.smithi083.stdout:(62/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.5 MB/s | 3.0 MB 00:00 2024-04-21T08:55:07.334 INFO:teuthology.orchestra.run.smithi186.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-04-21T08:55:07.354 INFO:teuthology.orchestra.run.smithi083.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-04-21T08:55:07.396 INFO:teuthology.orchestra.run.smithi083.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-21T08:55:07.438 INFO:teuthology.orchestra.run.smithi083.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-21T08:55:07.443 INFO:teuthology.orchestra.run.smithi186.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-21T08:55:07.471 INFO:teuthology.orchestra.run.smithi083.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 301 kB/s | 35 kB 00:00 2024-04-21T08:55:07.568 INFO:teuthology.orchestra.run.smithi186.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-21T08:55:07.677 INFO:teuthology.orchestra.run.smithi186.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 472 kB/s | 51 kB 00:00 2024-04-21T08:55:07.680 INFO:teuthology.orchestra.run.smithi083.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-21T08:55:07.794 INFO:teuthology.orchestra.run.smithi186.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 460 kB/s | 54 kB 00:00 2024-04-21T08:55:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-04-21T08:55:07.819 INFO:teuthology.orchestra.run.smithi186.stdout:(90/150): python3-protobuf-3.14.0-13.el9.noarch 481 kB/s | 269 kB 00:00 2024-04-21T08:55:07.936 INFO:teuthology.orchestra.run.smithi186.stdout:(91/150): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-04-21T08:55:07.947 INFO:teuthology.orchestra.run.smithi083.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 941 kB/s | 133 kB 00:00 2024-04-21T08:55:07.953 INFO:teuthology.orchestra.run.smithi186.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 312 kB/s | 42 kB 00:00 2024-04-21T08:55:08.028 INFO:teuthology.orchestra.run.smithi186.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-04-21T08:55:08.078 INFO:teuthology.orchestra.run.smithi186.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-21T08:55:08.120 INFO:teuthology.orchestra.run.smithi186.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 390 kB/s | 36 kB 00:00 2024-04-21T08:55:08.179 INFO:teuthology.orchestra.run.smithi186.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-04-21T08:55:08.279 INFO:teuthology.orchestra.run.smithi186.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-04-21T08:55:08.480 INFO:teuthology.orchestra.run.smithi186.stdout:(98/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-21T08:55:08.505 INFO:teuthology.orchestra.run.smithi186.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 416 kB/s | 94 kB 00:00 2024-04-21T08:55:08.588 INFO:teuthology.orchestra.run.smithi186.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-04-21T08:55:08.614 INFO:teuthology.orchestra.run.smithi186.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-04-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi186.stdout:(102/150): socat-1.7.4.1-5.el9.x86_64.rpm 588 kB/s | 305 kB 00:00 2024-04-21T08:55:08.697 INFO:teuthology.orchestra.run.smithi186.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-21T08:55:08.766 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-21T08:55:08.767 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-21T08:55:08.798 INFO:teuthology.orchestra.run.smithi186.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-21T08:55:08.948 INFO:teuthology.orchestra.run.smithi186.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-04-21T08:55:08.973 INFO:teuthology.orchestra.run.smithi186.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-21T08:55:08.998 INFO:teuthology.orchestra.run.smithi186.stdout:(107/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-21T08:55:09.040 INFO:teuthology.orchestra.run.smithi186.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-04-21T08:55:09.091 INFO:teuthology.orchestra.run.smithi186.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-04-21T08:55:09.116 INFO:teuthology.orchestra.run.smithi186.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-04-21T08:55:09.158 INFO:teuthology.orchestra.run.smithi186.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-21T08:55:09.242 INFO:teuthology.orchestra.run.smithi186.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-21T08:55:09.267 INFO:teuthology.orchestra.run.smithi186.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-21T08:55:09.309 INFO:teuthology.orchestra.run.smithi186.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-04-21T08:55:09.334 INFO:teuthology.orchestra.run.smithi186.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-04-21T08:55:09.376 INFO:teuthology.orchestra.run.smithi186.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-21T08:55:09.434 INFO:teuthology.orchestra.run.smithi186.stdout:(117/150): libarrow-9.0.0-10.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-21T08:55:09.460 INFO:teuthology.orchestra.run.smithi186.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-21T08:55:09.485 INFO:teuthology.orchestra.run.smithi186.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-21T08:55:09.510 INFO:teuthology.orchestra.run.smithi186.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-21T08:55:09.535 INFO:teuthology.orchestra.run.smithi186.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-21T08:55:09.561 INFO:teuthology.orchestra.run.smithi186.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-21T08:55:09.619 INFO:teuthology.orchestra.run.smithi186.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-21T08:55:09.645 INFO:teuthology.orchestra.run.smithi186.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-21T08:55:09.670 INFO:teuthology.orchestra.run.smithi186.stdout:(125/150): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-21T08:55:09.695 INFO:teuthology.orchestra.run.smithi186.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-21T08:55:09.729 INFO:teuthology.orchestra.run.smithi186.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-21T08:55:09.754 INFO:teuthology.orchestra.run.smithi186.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 231 kB/s | 19 kB 00:00 2024-04-21T08:55:09.779 INFO:teuthology.orchestra.run.smithi186.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-04-21T08:55:09.821 INFO:teuthology.orchestra.run.smithi186.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-21T08:55:09.846 INFO:teuthology.orchestra.run.smithi186.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-04-21T08:55:09.888 INFO:teuthology.orchestra.run.smithi186.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-21T08:55:09.914 INFO:teuthology.orchestra.run.smithi186.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-04-21T08:55:09.939 INFO:teuthology.orchestra.run.smithi186.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-21T08:55:09.964 INFO:teuthology.orchestra.run.smithi186.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-21T08:55:09.989 INFO:teuthology.orchestra.run.smithi186.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-21T08:55:10.015 INFO:teuthology.orchestra.run.smithi186.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-21T08:55:10.032 INFO:teuthology.orchestra.run.smithi186.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-04-21T08:55:10.057 INFO:teuthology.orchestra.run.smithi186.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-21T08:55:10.074 INFO:teuthology.orchestra.run.smithi186.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-21T08:55:10.091 INFO:teuthology.orchestra.run.smithi186.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-04-21T08:55:10.116 INFO:teuthology.orchestra.run.smithi186.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 870 kB/s | 51 kB 00:00 2024-04-21T08:55:10.141 INFO:teuthology.orchestra.run.smithi186.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-21T08:55:10.167 INFO:teuthology.orchestra.run.smithi186.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-21T08:55:10.192 INFO:teuthology.orchestra.run.smithi186.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-21T08:55:10.217 INFO:teuthology.orchestra.run.smithi186.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-21T08:55:10.242 INFO:teuthology.orchestra.run.smithi186.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-21T08:55:10.284 INFO:teuthology.orchestra.run.smithi186.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-21T08:55:10.760 INFO:teuthology.orchestra.run.smithi186.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 6.3 MB/s | 3.4 MB 00:00 2024-04-21T08:55:10.943 INFO:teuthology.orchestra.run.smithi186.stdout:(150/150): librbd1-19.0.0-3154.ga630765c.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-21T08:55:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:55:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:Total 10 MB/s | 218 MB 00:21 2024-04-21T08:55:10.956 INFO:teuthology.orchestra.run.smithi083.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 1.2 MB/s | 4.1 MB 00:03 2024-04-21T08:55:11.115 INFO:teuthology.orchestra.run.smithi083.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 1.3 MB/s | 4.7 MB 00:03 2024-04-21T08:55:11.298 INFO:teuthology.orchestra.run.smithi083.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 478 kB/s | 87 kB 00:00 2024-04-21T08:55:12.022 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-21T08:55:12.147 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-21T08:55:12.147 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-21T08:55:12.406 INFO:teuthology.orchestra.run.smithi083.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 712 kB/s | 1.0 MB 00:01 2024-04-21T08:55:12.757 INFO:teuthology.orchestra.run.smithi083.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 697 kB/s | 244 kB 00:00 2024-04-21T08:55:13.107 INFO:teuthology.orchestra.run.smithi083.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 709 kB/s | 248 kB 00:00 2024-04-21T08:55:13.207 INFO:teuthology.orchestra.run.smithi083.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-04-21T08:55:13.308 INFO:teuthology.orchestra.run.smithi083.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-21T08:55:13.424 INFO:teuthology.orchestra.run.smithi083.stdout:(78/150): postgresql-server-13.14-1.el9.x86_64. 1.1 MB/s | 5.8 MB 00:05 2024-04-21T08:55:13.525 INFO:teuthology.orchestra.run.smithi083.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-04-21T08:55:13.642 INFO:teuthology.orchestra.run.smithi083.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 515 kB/s | 172 kB 00:00 2024-04-21T08:55:13.853 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-21T08:55:13.853 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-21T08:55:13.984 INFO:teuthology.orchestra.run.smithi083.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 733 kB/s | 250 kB 00:00 2024-04-21T08:55:14.343 INFO:teuthology.orchestra.run.smithi083.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 620 kB/s | 222 kB 00:00 2024-04-21T08:55:14.543 INFO:teuthology.orchestra.run.smithi083.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 386 kB/s | 77 kB 00:00 2024-04-21T08:55:14.660 INFO:teuthology.orchestra.run.smithi083.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-04-21T08:55:14.740 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-21T08:55:14.856 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:55:15.013 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:55:15.059 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:55:15.085 INFO:teuthology.orchestra.run.smithi083.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 633 kB/s | 269 kB 00:00 2024-04-21T08:55:15.352 INFO:teuthology.orchestra.run.smithi083.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-04-21T08:55:15.403 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:55:15.505 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:55:15.629 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:55:15.656 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:55:15.749 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:55:15.777 INFO:teuthology.orchestra.run.smithi083.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 656 kB/s | 279 kB 00:00 2024-04-21T08:55:15.791 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:55:15.819 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:55:15.845 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:55:15.886 INFO:teuthology.orchestra.run.smithi083.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-04-21T08:55:15.986 INFO:teuthology.orchestra.run.smithi083.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-04-21T08:55:16.014 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:55:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:55:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:55:16.200 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:55:16.229 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:55:16.275 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:55:16.305 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:55:16.329 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:55:16.523 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:55:16.555 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:55:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:55:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:55:16.805 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:55:16.842 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:55:16.918 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:55:16.991 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:55:17.151 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:55:17.183 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:55:17.211 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:55:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:55:17.348 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:55:17.380 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:55:17.424 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:55:17.505 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:55:17.543 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:55:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:55:17.615 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:55:17.693 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:55:17.738 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:55:17.827 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:55:17.828 INFO:teuthology.orchestra.run.smithi083.stdout:(90/150): python3-babel-2.9.1-2.el9.noarch.rpm 935 kB/s | 6.0 MB 00:06 2024-04-21T08:55:17.937 INFO:teuthology.orchestra.run.smithi083.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-04-21T08:55:17.977 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:55:18.035 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:55:18.037 INFO:teuthology.orchestra.run.smithi083.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-04-21T08:55:18.146 INFO:teuthology.orchestra.run.smithi083.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-21T08:55:18.237 INFO:teuthology.orchestra.run.smithi083.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-04-21T08:55:18.338 INFO:teuthology.orchestra.run.smithi083.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-21T08:55:18.447 INFO:teuthology.orchestra.run.smithi083.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-21T08:55:18.538 INFO:teuthology.orchestra.run.smithi083.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 631 kB/s | 58 kB 00:00 2024-04-21T08:55:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:55:18.639 INFO:teuthology.orchestra.run.smithi083.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 936 kB/s | 94 kB 00:00 2024-04-21T08:55:18.708 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:55:18.739 INFO:teuthology.orchestra.run.smithi083.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-04-21T08:55:18.740 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:55:18.783 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:55:18.822 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:55:18.839 INFO:teuthology.orchestra.run.smithi083.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-04-21T08:55:18.854 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:55:18.912 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:55:18.923 INFO:teuthology.orchestra.run.smithi083.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 270 kB/s | 22 kB 00:00 2024-04-21T08:55:18.987 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:55:19.030 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:55:19.048 INFO:teuthology.orchestra.run.smithi083.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 6.8 MB/s | 863 kB 00:00 2024-04-21T08:55:19.066 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:55:19.116 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:55:19.149 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:55:19.193 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:55:19.236 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:55:19.348 INFO:teuthology.orchestra.run.smithi083.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-21T08:55:19.384 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:55:19.415 INFO:teuthology.orchestra.run.smithi083.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-04-21T08:55:19.424 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:55:19.454 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:55:19.490 INFO:teuthology.orchestra.run.smithi083.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-21T08:55:19.496 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:55:19.597 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:55:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:55:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:55:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:55:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:55:19.756 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-21T08:55:19.799 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:55:19.870 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:55:19.895 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:55:19.937 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:55:19.939 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:55:19.967 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:55:19.987 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:55:20.008 INFO:teuthology.orchestra.run.smithi083.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-04-21T08:55:20.075 INFO:teuthology.orchestra.run.smithi083.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-04-21T08:55:20.088 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:55:20.142 INFO:teuthology.orchestra.run.smithi083.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 731 kB/s | 49 kB 00:00 2024-04-21T08:55:20.167 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:55:20.204 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:55:20.209 INFO:teuthology.orchestra.run.smithi083.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-21T08:55:20.244 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:55:20.273 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:55:20.276 INFO:teuthology.orchestra.run.smithi083.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-21T08:55:20.307 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:55:20.328 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:55:20.341 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:55:20.383 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:55:20.384 INFO:teuthology.orchestra.run.smithi083.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-04-21T08:55:20.397 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:55:20.425 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:55:20.451 INFO:teuthology.orchestra.run.smithi083.stdout:(112/150): python3-numpy-1.20.1-5.el9.x86_64.rp 754 kB/s | 5.1 MB 00:06 2024-04-21T08:55:20.477 INFO:teuthology.orchestra.run.smithi083.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-21T08:55:20.510 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:55:20.518 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:55:20.544 INFO:teuthology.orchestra.run.smithi083.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 483 kB/s | 32 kB 00:00 2024-04-21T08:55:20.547 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:55:20.565 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:55:20.597 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:55:20.610 INFO:teuthology.orchestra.run.smithi083.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-04-21T08:55:20.636 INFO:teuthology.orchestra.run.smithi083.stdout:(116/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 236 kB/s | 43 kB 00:00 2024-04-21T08:55:20.639 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:55:20.643 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:55:20.678 INFO:teuthology.orchestra.run.smithi083.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-21T08:55:20.681 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:55:20.718 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:55:20.744 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:55:20.745 INFO:teuthology.orchestra.run.smithi083.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-21T08:55:20.757 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:55:20.778 INFO:teuthology.orchestra.run.smithi083.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-21T08:55:20.807 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:55:20.812 INFO:teuthology.orchestra.run.smithi083.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-04-21T08:55:20.846 INFO:teuthology.orchestra.run.smithi083.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-04-21T08:55:20.899 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:55:20.912 INFO:teuthology.orchestra.run.smithi083.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-21T08:55:20.957 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:55:20.979 INFO:teuthology.orchestra.run.smithi083.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-21T08:55:20.986 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:55:21.030 INFO:teuthology.orchestra.run.smithi083.stdout:(124/150): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-21T08:55:21.055 INFO:teuthology.orchestra.run.smithi083.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-21T08:55:21.081 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:55:21.089 INFO:teuthology.orchestra.run.smithi083.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-21T08:55:21.103 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:55:21.114 INFO:teuthology.orchestra.run.smithi083.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-21T08:55:21.147 INFO:teuthology.orchestra.run.smithi083.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-04-21T08:55:21.149 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:55:21.171 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:55:21.189 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:55:21.206 INFO:teuthology.orchestra.run.smithi083.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 792 kB/s | 46 kB 00:00 2024-04-21T08:55:21.241 INFO:teuthology.orchestra.run.smithi083.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-21T08:55:21.273 INFO:teuthology.orchestra.run.smithi083.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-21T08:55:21.299 INFO:teuthology.orchestra.run.smithi083.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-04-21T08:55:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-21T08:55:21.362 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:55:21.396 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:55:21.424 INFO:teuthology.orchestra.run.smithi083.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-21T08:55:21.450 INFO:teuthology.orchestra.run.smithi083.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-04-21T08:55:21.492 INFO:teuthology.orchestra.run.smithi083.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-04-21T08:55:21.509 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:55:21.517 INFO:teuthology.orchestra.run.smithi083.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-04-21T08:55:21.559 INFO:teuthology.orchestra.run.smithi083.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-21T08:55:21.584 INFO:teuthology.orchestra.run.smithi083.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 884 kB/s | 59 kB 00:00 2024-04-21T08:55:21.612 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:55:21.618 INFO:teuthology.orchestra.run.smithi083.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-04-21T08:55:21.643 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:55:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 868 kB/s | 51 kB 00:00 2024-04-21T08:55:21.645 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:55:21.685 INFO:teuthology.orchestra.run.smithi083.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-21T08:55:21.689 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:55:21.702 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:55:21.711 INFO:teuthology.orchestra.run.smithi083.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-21T08:55:21.736 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:55:21.753 INFO:teuthology.orchestra.run.smithi083.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-21T08:55:21.765 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:55:21.769 INFO:teuthology.orchestra.run.smithi083.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-21T08:55:21.805 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:55:21.820 INFO:teuthology.orchestra.run.smithi083.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-21T08:55:21.846 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:55:21.848 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:55:21.880 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:55:21.904 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:55:21.908 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:55:21.937 INFO:teuthology.orchestra.run.smithi083.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-21T08:55:21.980 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:55:21.996 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:55:22.032 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:55:22.037 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:55:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:55:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:55:22.191 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:55:22.195 INFO:teuthology.orchestra.run.smithi083.stdout:(148/150): librados2-19.0.0-3154.ga630765c.el9. 9.0 MB/s | 3.4 MB 00:00 2024-04-21T08:55:22.224 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:55:22.260 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:55:22.267 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:55:22.295 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:55:22.301 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:55:22.368 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:55:22.381 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:55:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:55:22.487 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:55:22.633 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:55:22.698 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:55:22.747 INFO:teuthology.orchestra.run.smithi083.stdout:(149/150): librbd1-19.0.0-3154.ga630765c.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-04-21T08:55:23.098 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:55:23.224 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:55:23.245 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:55:23.335 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:55:23.370 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:55:23.430 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:55:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:55:23.501 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:55:23.557 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:55:23.630 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:55:23.660 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:55:23.697 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:55:23.744 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:55:23.788 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:55:23.831 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:55:23.873 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:55:24.012 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:55:24.062 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:55:24.092 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:55:24.133 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:55:24.229 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:55:24.266 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:55:24.299 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:55:24.324 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:55:24.349 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:55:24.425 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:55:24.553 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:55:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:55:24.589 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:55:24.617 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:55:24.651 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:55:24.663 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:55:24.692 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:55:24.695 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:55:24.725 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:55:24.733 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:55:24.764 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:55:24.794 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:55:24.829 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:55:24.892 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:55:25.035 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:55:25.053 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:55:25.087 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:55:25.138 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:55:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:55:25.192 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:55:25.227 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:55:25.228 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:55:25.277 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:55:25.308 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:55:25.356 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:55:25.558 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:55:25.619 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:55:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:55:25.765 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:55:25.837 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:55:25.866 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:55:26.371 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:55:26.424 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:55:26.482 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:55:26.586 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:55:26.648 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:55:26.738 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:55:26.776 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:55:26.954 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:55:26.981 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:55:27.060 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:55:27.730 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:55:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:55:28.276 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:28.313 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:28.925 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:55:29.017 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:55:29.018 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:55:29.018 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:55:29.018 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:29.050 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:55:29.128 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:55:29.210 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:55:29.238 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:55:29.271 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:55:29.582 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:55:29.625 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:55:29.716 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:55:29.747 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:55:29.783 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:55:30.260 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:55:30.295 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:55:30.366 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:55:30.388 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:55:32.658 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:32.692 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:33.232 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:33.270 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:55:33.359 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:55:33.360 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:55:33.360 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:55:33.360 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:33.388 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:55:34.212 INFO:teuthology.orchestra.run.smithi083.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 913 kB/s | 16 MB 00:18 2024-04-21T08:55:34.215 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:55:34.215 INFO:teuthology.orchestra.run.smithi083.stdout:Total 5.0 MB/s | 218 MB 00:43 2024-04-21T08:55:35.227 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-21T08:55:35.351 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-21T08:55:35.351 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-21T08:55:37.054 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-21T08:55:37.055 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-21T08:55:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:55:40.714 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-04-21T08:55:40.714 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-04-21T08:55:40.715 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-04-21T08:55:40.715 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-04-21T08:55:40.715 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-04-21T08:55:40.715 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-04-21T08:55:40.715 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-04-21T08:55:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:40.911 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:55:40.927 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:55:40.928 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:40.928 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:55:40.928 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:55:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:55:42.490 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:42.490 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:55:42.491 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:55:42.491 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:55:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:42.723 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:55:42.747 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:55:42.748 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:42.748 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:55:42.748 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:55:42.748 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:55:42.748 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-21T08:55:43.055 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:55:43.089 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:55:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:55:43.216 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:55:43.243 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:55:43.243 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:43.243 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:55:43.243 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:55:43.244 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:55:43.244 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:43.356 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:55:43.414 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:55:43.460 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:55:43.478 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:55:43.573 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:55:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:55:43.686 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:55:43.716 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:55:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:55:43.790 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:55:43.808 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:55:43.833 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:55:43.862 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:55:43.888 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:55:43.907 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:55:44.037 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:55:44.051 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:55:44.092 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:55:44.093 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:55:44.121 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:55:44.124 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:55:44.151 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:55:44.225 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:55:44.248 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:55:44.310 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:55:44.332 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:55:44.351 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:55:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:55:44.574 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:55:44.597 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:55:44.625 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:55:44.658 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:55:44.689 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:55:44.722 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:55:44.756 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:55:44.786 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:55:44.823 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:55:44.858 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:55:44.974 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:55:45.018 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-04-21T08:55:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-04-21T08:55:45.034 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:45.050 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:55:45.078 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:55:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:55:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:55:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:55:45.232 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:55:45.248 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:55:45.249 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:45.249 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:55:45.249 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:55:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:55:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:55:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:55:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:55:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:55:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:55:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:55:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:55:45.857 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:55:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:55:45.891 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:55:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:55:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:55:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:55:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:55:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:55:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:55:46.710 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:55:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:55:46.736 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:55:46.736 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:46.736 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:55:46.736 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:55:46.736 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:55:46.736 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:55:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:55:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:55:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:55:46.909 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:55:46.941 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:55:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:55:46.961 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:55:46.983 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:55:46.984 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:46.984 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:55:46.984 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:55:46.984 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:55:46.984 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:55:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:55:47.053 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:55:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:55:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:55:47.220 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:55:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:55:47.252 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:55:47.277 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:55:47.277 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:47.277 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:55:47.277 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:55:47.277 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:55:47.277 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:55:47.328 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:55:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:55:47.382 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:55:47.420 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:55:47.430 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:55:47.441 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:55:47.443 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:55:47.443 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:47.443 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:55:47.443 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:55:47.443 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:55:47.443 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:47.469 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:55:47.494 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:55:47.495 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:55:47.520 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:55:47.550 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:55:47.594 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:55:47.607 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:55:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:55:47.722 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:55:47.761 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:55:47.789 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:55:47.823 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:55:47.860 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:55:47.867 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:55:47.904 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:55:47.905 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:55:47.936 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:55:47.966 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:55:47.983 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:55:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:55:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:55:48.072 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:55:48.076 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:55:48.110 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:55:48.198 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:55:48.214 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:55:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:55:48.254 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:55:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:55:48.286 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:55:48.314 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:55:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:55:48.427 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:55:48.469 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:55:48.473 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:55:48.505 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:55:48.529 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:55:48.530 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:48.530 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:55:48.530 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:55:48.530 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:55:48.530 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:48.572 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:55:48.596 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:55:48.597 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:48.597 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:55:48.597 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:48.648 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:55:48.672 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:55:48.672 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:48.672 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:55:48.672 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:55:48.672 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:55:48.673 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:48.766 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:55:48.795 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:55:48.827 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:55:48.889 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:55:48.918 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:55:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:55:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:55:49.015 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:49.015 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:55:49.015 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:55:49.015 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:55:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:55:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:55:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:55:49.288 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:55:49.388 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:55:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:55:49.525 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:55:49.557 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:55:49.750 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:55:49.796 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:55:49.874 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:55:49.957 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:55:49.995 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:55:50.524 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:55:50.637 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:55:50.964 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:55:50.987 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:55:51.092 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:55:51.259 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:55:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:55:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:55:51.321 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:51.321 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:55:51.321 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:55:51.321 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:55:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:51.365 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:55:51.376 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:55:51.474 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:55:51.522 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:55:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:55:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:55:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:55:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:55:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:55:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:55:52.442 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:55:52.476 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:55:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:55:52.500 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:55:52.500 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:52.500 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:55:52.500 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:55:52.500 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:55:52.500 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:55:52.543 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:55:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:55:52.565 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:55:52.566 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:52.566 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:55:52.566 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:52.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:55:52.642 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:55:52.642 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:52.642 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:55:52.642 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:55:52.642 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:55:52.643 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:52.953 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:55:52.976 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:55:52.976 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:55:52.976 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:55:52.976 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:55:52.976 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:55:52.976 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:55:53.030 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:55:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:55:53.174 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:55:53.196 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:55:53.853 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:55:53.894 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:55:53.936 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:55:53.967 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:55:53.986 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:55:54.180 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:55:54.223 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:55:54.253 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:55:54.887 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:55:54.925 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:55:54.947 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:55:55.446 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:55.476 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:55.823 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:55:55.904 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:55:55.939 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:55:56.024 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:55:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:55:56.063 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:55:56.075 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:55:56.075 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:55:56.152 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:55:56.152 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:55:56.152 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:55:56.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:55:56.159 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:55:56.159 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:55:56.181 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:55:57.565 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:55:57.606 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:55:57.647 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:55:57.679 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:55:57.706 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:55:57.894 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:55:57.944 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:55:57.982 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:55:58.599 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:55:58.638 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:55:58.658 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:55:59.452 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:55:59.533 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:55:59.574 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:55:59.657 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:55:59.687 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:55:59.687 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:55:59.787 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:55:59.788 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:56:01.762 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:56:01.763 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:56:01.764 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:56:01.775 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:56:01.775 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:56:01.775 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:56:01.775 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:56:01.775 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:56:01.775 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:56:01.775 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:56:01.776 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:56:01.777 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:56:01.778 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:56:01.779 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:56:01.780 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:56:01.781 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:56:01.782 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:56:01.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:56:01.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:56:03.853 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:56:03.855 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.856 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:56:03.857 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:56:03.858 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:56:03.859 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:56:03.860 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:56:03.861 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.862 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:56:03.863 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:56:03.864 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:56:03.864 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.864 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:56:03.864 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:56:03.864 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:56:03.864 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.865 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.866 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:03.866 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:56:03.866 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:56:03.866 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:56:03.866 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:56:03.866 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:56:03.866 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:56:03.867 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:56:03.867 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:56:03.867 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-21T08:56:04.206 DEBUG:teuthology.parallel:result is None 2024-04-21T08:56:05.616 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:05.616 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:56:05.616 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:56:05.616 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:56:05.617 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:56:05.618 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:56:05.619 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:56:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:56:05.621 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:56:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:56:05.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:56:05.624 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:56:05.625 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:56:05.626 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:56:05.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:56:05.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:56:05.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:56:05.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:56:05.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:56:05.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:56:07.477 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:56:07.477 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-04-21T08:56:07.477 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-04-21T08:56:07.478 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-04-21T08:56:07.478 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-04-21T08:56:07.478 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-04-21T08:56:07.478 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-04-21T08:56:07.478 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-04-21T08:56:07.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:07.670 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:56:07.685 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:56:07.685 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:07.685 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:56:07.685 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:07.710 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.711 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.712 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:56:07.713 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.714 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:56:07.715 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:56:07.716 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:56:07.717 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:56:07.718 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.719 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:56:07.720 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:56:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:56:07.722 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:56:07.722 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:56:07.722 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-21T08:56:08.059 DEBUG:teuthology.parallel:result is None 2024-04-21T08:56:09.173 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:56:09.198 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:56:09.199 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:09.199 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:56:09.199 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:56:09.199 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:56:09.199 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:09.424 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:56:09.446 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:56:09.446 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:09.446 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:56:09.446 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:56:09.446 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:56:09.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:56:09.913 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:56:09.913 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:09.913 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:56:09.913 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:56:09.913 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:56:09.913 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:10.013 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:56:10.069 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:56:10.133 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:56:10.318 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:56:10.367 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:56:10.445 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:56:10.540 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:56:10.672 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:56:10.709 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:56:10.741 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:56:10.768 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:56:11.153 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:56:11.184 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:56:11.216 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:56:11.277 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:56:11.307 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:56:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:56:12.376 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:56:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:56:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:56:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:56:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:56:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:56:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:56:13.693 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:13.693 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:56:13.693 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:56:13.693 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:56:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:56:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:56:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:56:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:56:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:56:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:56:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:56:14.888 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:14.888 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:56:14.888 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:56:14.888 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:56:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:56:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:56:14.954 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:14.954 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:56:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:15.000 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:56:15.022 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:56:15.026 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:15.026 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:56:15.026 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:56:15.026 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:56:15.026 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:15.326 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:56:15.348 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:56:15.348 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:56:15.348 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:56:15.348 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:56:15.348 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:56:15.348 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:19.929 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:56:19.970 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:56:20.011 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:56:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:56:20.071 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:56:20.250 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:56:20.292 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:56:20.322 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:56:20.891 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:56:20.928 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:56:20.948 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:56:21.723 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:56:21.808 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:56:21.848 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:56:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:56:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:56:21.971 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:56:22.055 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:56:22.055 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:28.184 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:56:28.185 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:56:28.186 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:56:28.187 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:56:28.187 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:56:28.187 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:56:28.187 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:56:28.187 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:56:28.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:56:28.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:56:28.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:56:28.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:56:28.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:56:28.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:56:28.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:56:28.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:56:28.201 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:56:28.202 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:56:28.203 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:56:28.204 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:56:28.205 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:56:28.206 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:56:28.207 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:56:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:56:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:30.249 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:56:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:56:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:56:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:56:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:56:30.258 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:56:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:56:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:56:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:56:30.261 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-21T08:56:30.261 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-21T08:56:30.620 DEBUG:teuthology.parallel:result is None 2024-04-21T08:56:30.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:56:30.837 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:56:30.864 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:56:30.865 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:56:30.865 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:56:30.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:56:31.088 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:56:31.109 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:56:31.109 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:56:31.110 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:56:31.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:56:31.327 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:56:31.353 INFO:teuthology.orchestra.run.smithi186.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:56:31.353 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:56:31.353 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:56:31.355 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-21T08:56:31.355 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:56:31.355 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:56:31.387 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:56:31.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:56:31.414 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:56:31.414 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:56:31.444 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-21T08:56:31.444 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:56:31.444 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:56:31.474 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:56:31.542 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:56:31.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:56:31.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:56:31.630 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:56:31.631 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:56:31.659 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:56:31.727 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-21T08:56:31.727 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:56:31.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:56:31.755 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:56:31.821 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:56:31.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:56:31.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:56:31.907 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:56:31.907 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:56:31.931 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:56:31.995 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-21T08:56:31.995 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:56:31.995 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:56:32.020 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:56:32.084 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:56:32.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:56:32.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:56:32.170 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:56:32.170 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:56:32.194 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:56:32.258 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-21T08:56:32.355 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-21T08:56:32.356 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:56:32.356 INFO:tasks.cephadm:Cluster fsid is 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:56:32.356 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-21T08:56:32.356 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.66', 'mon.b': '172.21.15.83', 'mon.c': '172.21.15.186'} 2024-04-21T08:56:32.356 INFO:tasks.cephadm:First mon is mon.a on smithi066 2024-04-21T08:56:32.357 INFO:tasks.cephadm:First mgr is x 2024-04-21T08:56:32.357 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-21T08:56:32.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2024-04-21T08:56:32.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-04-21T08:56:32.406 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-04-21T08:56:32.430 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-21T08:56:32.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:56:32.568 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78740/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3154-ga630765c', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3154.ga630765c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-19 11:13:53.331234', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-21T08:56:32.782 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 a630765c1f93c6e61d9b40c4b76437611d5ef29e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:56:32.783 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm 2024-04-21T08:56:32.784 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:33.419 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:56 /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:33.419 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:34.073 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:56 /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:34.073 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:34.679 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:56 /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:34.680 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:34.699 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:34.716 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:56:34.750 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e on all hosts... 2024-04-21T08:56:34.750 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:56:34.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:56:34.759 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:56:36.290 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:56:36.305 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:56:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:57:11.961 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-21T08:57:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:57:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:57:11.961 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-21T08:57:22.734 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-21T08:57:22.734 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:57:22.734 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:57:22.734 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-04-21T08:57:22.735 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:57:22.735 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-04-21T08:57:22.735 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-21T08:57:25.941 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T08:57:25.941 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:57:25.941 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:57:25.941 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2024-04-21T08:57:25.941 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:57:25.941 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T08:57:25.941 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T08:57:25.966 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2024-04-21T08:57:26.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-04-21T08:57:26.026 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-04-21T08:57:26.057 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2024-04-21T08:57:26.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-04-21T08:57:26.104 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-04-21T08:57:26.131 INFO:tasks.cephadm:Writing seed config... 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-21T08:57:26.132 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-21T08:57:26.133 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-21T08:57:26.134 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-21T08:57:26.135 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-21T08:57:26.135 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-21T08:57:26.135 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-21T08:57:26.135 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:57:26.135 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-21T08:57:26.151 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0cca84f6-ffbd-11ee-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-21T08:57:26.151 DEBUG:teuthology.orchestra.run.smithi066:mon.a> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a.service 2024-04-21T08:57:26.193 DEBUG:teuthology.orchestra.run.smithi066:mgr.x> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.x.service 2024-04-21T08:57:26.236 INFO:tasks.cephadm:Bootstrapping... 2024-04-21T08:57:26.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e -v bootstrap --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.66 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:57:26.432 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:57:26.432 INFO:teuthology.orchestra.run.smithi066.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e', '-v', 'bootstrap', '--fsid', '0cca84f6-ffbd-11ee-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.66', '--skip-admin-label'] 2024-04-21T08:57:26.432 INFO:teuthology.orchestra.run.smithi066.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-21T08:57:26.433 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying podman|docker is present... 2024-04-21T08:57:26.474 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:57:26.474 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying lvm2 is present... 2024-04-21T08:57:26.475 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying time synchronization is in place... 2024-04-21T08:57:26.482 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:57:26.482 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:57:26.488 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:57:26.488 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-04-21T08:57:26.495 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2024-04-21T08:57:26.501 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2024-04-21T08:57:26.501 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:57:26.501 INFO:teuthology.orchestra.run.smithi066.stdout:Repeating the final host check... 2024-04-21T08:57:26.527 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:57:26.527 INFO:teuthology.orchestra.run.smithi066.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-21T08:57:26.527 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl is present 2024-04-21T08:57:26.527 INFO:teuthology.orchestra.run.smithi066.stdout:lvcreate is present 2024-04-21T08:57:26.534 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:57:26.534 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:57:26.540 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:57:26.540 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-04-21T08:57:26.546 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2024-04-21T08:57:26.553 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2024-04-21T08:57:26.553 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:57:26.553 INFO:teuthology.orchestra.run.smithi066.stdout:Host looks OK 2024-04-21T08:57:26.553 INFO:teuthology.orchestra.run.smithi066.stdout:Cluster fsid: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:26.553 INFO:teuthology.orchestra.run.smithi066.stdout:Acquiring lock 140182174092448 on /run/cephadm/0cca84f6-ffbd-11ee-bc93-c7b262605968.lock 2024-04-21T08:57:26.553 INFO:teuthology.orchestra.run.smithi066.stdout:Lock 140182174092448 acquired on /run/cephadm/0cca84f6-ffbd-11ee-bc93-c7b262605968.lock 2024-04-21T08:57:26.553 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 3300 ... 2024-04-21T08:57:26.554 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 6789 ... 2024-04-21T08:57:26.554 INFO:teuthology.orchestra.run.smithi066.stdout:Base mon IP(s) is [172.21.15.66:3300, 172.21.15.66:6789], mon addrv is [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2024-04-21T08:57:26.557 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.66 metric 100 2024-04-21T08:57:26.557 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.66 metric 100 2024-04-21T08:57:26.560 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-21T08:57:26.560 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-21T08:57:26.560 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-21T08:57:26.562 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-21T08:57:26.562 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-21T08:57:26.562 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:57:26.562 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-21T08:57:26.563 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 fe80::37c:f2c2:71f0:1287/64 scope link noprefixroute 2024-04-21T08:57:26.563 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:57:26.563 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2024-04-21T08:57:26.563 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2024-04-21T08:57:26.563 INFO:teuthology.orchestra.run.smithi066.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-21T08:57:26.564 INFO:teuthology.orchestra.run.smithi066.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-21T08:57:26.564 INFO:teuthology.orchestra.run.smithi066.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:57:27.077 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:57:27.077 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:57:27.077 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Getting image source signatures 2024-04-21T08:57:27.077 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:7eab8a2da10d06ac88aea1fd8998a2559b4de9d07c585e3b1e6ee6c4d35fc119 2024-04-21T08:57:27.077 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-21T08:57:27.078 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying config sha256:b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:57:27.078 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-21T08:57:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:ceph: stdout ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:57:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph version: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:57:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:Extracting ceph user uid/gid from container image... 2024-04-21T08:57:28.846 INFO:teuthology.orchestra.run.smithi066.stdout:stat: stdout 167 167 2024-04-21T08:57:28.846 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial keys... 2024-04-21T08:57:29.775 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQD51CRmVe8+ChAApzPP7ha3pSr3aXSufIUOMg== 2024-04-21T08:57:30.768 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQD61CRmBHkrBhAAtwAiwjZ7JzwyHxf583jGfw== 2024-04-21T08:57:31.624 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQD71CRmv1nFAxAAj2OSu5HNkHZbZYblvxvJ2g== 2024-04-21T08:57:31.625 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial monmap... 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool for a [v2:172.21.15.66:3300,v1:172.21.15.66:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = quincy 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: set fsid to 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:57:32.515 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mon... 2024-04-21T08:57:33.396 INFO:teuthology.orchestra.run.smithi066.stdout:create mon.a on 2024-04-21T08:57:33.614 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-21T08:57:33.781 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-21T08:57:33.955 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968.target. 2024-04-21T08:57:33.955 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968.target. 2024-04-21T08:57:34.235 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a 2024-04-21T08:57:34.235 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a.service: Unit ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a.service not loaded. 2024-04-21T08:57:34.417 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968.target.wants/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@.service. 2024-04-21T08:57:34.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 systemd[1]: Starting Ceph mon.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T08:57:34.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 podman[27546]: 2024-04-21 08:57:34.662541118 +0000 UTC m=+0.093793572 container create e904528e3793a4e1c855fc6a1415814e8b48e591704b034530608d1feab6a576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-21T08:57:34.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 podman[27546]: 2024-04-21 08:57:34.587529192 +0000 UTC m=+0.018781681 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:57:34.937 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:57:34.937 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-04-21T08:57:34.946 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:57:34.946 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-04-21T08:57:34.946 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-04-21T08:57:34.946 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:57:34.946 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon to start... 2024-04-21T08:57:34.946 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon... 2024-04-21T08:57:35.075 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 podman[27546]: 2024-04-21 08:57:34.857830467 +0000 UTC m=+0.289082924 container init e904528e3793a4e1c855fc6a1415814e8b48e591704b034530608d1feab6a576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T08:57:35.076 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 podman[27546]: 2024-04-21 08:57:34.861323775 +0000 UTC m=+0.292576236 container start e904528e3793a4e1c855fc6a1415814e8b48e591704b034530608d1feab6a576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-21T08:57:35.076 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 ceph-mon[27573]: mkfs 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:35.076 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 ceph-mon[27573]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:57:35.076 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 bash[27546]: e904528e3793a4e1c855fc6a1415814e8b48e591704b034530608d1feab6a576 2024-04-21T08:57:35.076 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:34 smithi066 systemd[1]: Started Ceph mon.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T08:57:35.964 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout cluster: 2024-04-21T08:57:35.964 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout id: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:35.964 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-21T08:57:35.964 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:35.964 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout services: 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.557245s) 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout data: 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pgs: 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.386+0000 7fd70dc49700 1 Processor -- start 2024-04-21T08:57:35.965 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.386+0000 7fd70dc49700 1 -- start start 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.387+0000 7fd70dc49700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd708108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.387+0000 7fd70dc49700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd708108d30 con 0x7fd708108340 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.387+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd708108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.387+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd708108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42612/0 (socket says 172.21.15.66:42612) 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.387+0000 7fd7077fe700 1 -- 172.21.15.66:0/1920621577 learned_addr learned my addr 172.21.15.66:0/1920621577 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.387+0000 7fd7077fe700 1 -- 172.21.15.66:0/1920621577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd708109540 con 0x7fd708108340 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.388+0000 7fd7077fe700 1 --2- 172.21.15.66:0/1920621577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd708108760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80098a0 tx=0x7fd6f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32533c3b241bc24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.388+0000 7fd7067fc700 1 -- 172.21.15.66:0/1920621577 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6f8004160 con 0x7fd708108340 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.388+0000 7fd7067fc700 1 -- 172.21.15.66:0/1920621577 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f8004300 con 0x7fd708108340 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.388+0000 7fd7067fc700 1 -- 172.21.15.66:0/1920621577 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6f80036a0 con 0x7fd708108340 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.388+0000 7fd70dc49700 1 -- 172.21.15.66:0/1920621577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 msgr2=0x7fd708108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:35.966 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.388+0000 7fd70dc49700 1 --2- 172.21.15.66:0/1920621577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd708108760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80098a0 tx=0x7fd6f8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.389+0000 7fd70dc49700 1 -- 172.21.15.66:0/1920621577 shutdown_connections 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.389+0000 7fd70dc49700 1 --2- 172.21.15.66:0/1920621577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd708108760 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.389+0000 7fd70dc49700 1 -- 172.21.15.66:0/1920621577 >> 172.21.15.66:0/1920621577 conn(0x7fd7081038c0 msgr2=0x7fd708105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.389+0000 7fd70dc49700 1 -- 172.21.15.66:0/1920621577 shutdown_connections 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.389+0000 7fd70dc49700 1 -- 172.21.15.66:0/1920621577 wait complete. 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.389+0000 7fd70dc49700 1 Processor -- start 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.389+0000 7fd70dc49700 1 -- start start 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd70dc49700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd70819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd70dc49700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd708109a70 con 0x7fd708108340 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd70819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd70819b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42616/0 (socket says 172.21.15.66:42616) 2024-04-21T08:57:35.967 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd7077fe700 1 -- 172.21.15.66:0/4134643271 learned_addr learned my addr 172.21.15.66:0/4134643271 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd7077fe700 1 -- 172.21.15.66:0/4134643271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70819bf20 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd7077fe700 1 --2- 172.21.15.66:0/4134643271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd70819b9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd704ff9700 1 -- 172.21.15.66:0/4134643271 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6f8003db0 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd704ff9700 1 -- 172.21.15.66:0/4134643271 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f8011630 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.390+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70819c1b0 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.391+0000 7fd704ff9700 1 -- 172.21.15.66:0/4134643271 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6f8011950 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.391+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70819eea0 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.391+0000 7fd704ff9700 1 -- 172.21.15.66:0/4134643271 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6f8011af0 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.391+0000 7fd704ff9700 1 -- 172.21.15.66:0/4134643271 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6f801c450 con 0x7fd708108340 2024-04-21T08:57:35.968 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.392+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7080640a0 con 0x7fd708108340 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.394+0000 7fd704ff9700 1 -- 172.21.15.66:0/4134643271 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd6f8032020 con 0x7fd708108340 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.468+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd70802e100 con 0x7fd708108340 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.468+0000 7fd704ff9700 1 -- 172.21.15.66:0/4134643271 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd6f8004260 con 0x7fd708108340 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.469+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 msgr2=0x7fd70819b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.469+0000 7fd70dc49700 1 --2- 172.21.15.66:0/4134643271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd70819b9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8004750 comp rx=0 tx=0).stop 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.469+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 shutdown_connections 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.469+0000 7fd70dc49700 1 --2- 172.21.15.66:0/4134643271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd708108340 0x7fd70819b9e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.469+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 >> 172.21.15.66:0/4134643271 conn(0x7fd7081038c0 msgr2=0x7fd7081926a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:35.969 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.469+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 shutdown_connections 2024-04-21T08:57:35.970 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:35.469+0000 7fd70dc49700 1 -- 172.21.15.66:0/4134643271 wait complete. 2024-04-21T08:57:35.970 INFO:teuthology.orchestra.run.smithi066.stdout:mon is available 2024-04-21T08:57:35.970 INFO:teuthology.orchestra.run.smithi066.stdout:Assimilating anything we can from ceph.conf... 2024-04-21T08:57:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:57:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: monmap epoch 1 2024-04-21T08:57:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: last_changed 2024-04-21T08:57:31.934215+0000 2024-04-21T08:57:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:57:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: min_mon_release 19 (squid) 2024-04-21T08:57:36.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: election_strategy: 1 2024-04-21T08:57:36.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:57:36.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: fsmap 2024-04-21T08:57:36.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: osdmap e1: 0 total, 0 up, 0 in 2024-04-21T08:57:36.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: mgrmap e1: no daemons active 2024-04-21T08:57:36.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:35 smithi066 ceph-mon[27573]: from='client.? 172.21.15.66:0/4134643271' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-21T08:57:37.231 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:37 smithi066 ceph-mon[27573]: from='client.? 172.21.15.66:0/1022813459' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T08:57:37.232 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:37 smithi066 ceph-mon[27573]: from='client.? 172.21.15.66:0/1022813459' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-21T08:57:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:57:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.059+0000 7fd0141ca700 1 Processor -- start 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.059+0000 7fd0141ca700 1 -- start start 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.059+0000 7fd0141ca700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c108740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.059+0000 7fd0141ca700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c108d10 con 0x7fd00c108320 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd011f66700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c108740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd011f66700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c108740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42624/0 (socket says 172.21.15.66:42624) 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd011f66700 1 -- 172.21.15.66:0/2824133476 learned_addr learned my addr 172.21.15.66:0/2824133476 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd011f66700 1 -- 172.21.15.66:0/2824133476 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c109520 con 0x7fd00c108320 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd011f66700 1 --2- 172.21.15.66:0/2824133476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c108740 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd0080098a0 tx=0x7fd008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82922fe48cc61707 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd010f64700 1 -- 172.21.15.66:0/2824133476 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd008004160 con 0x7fd00c108320 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd010f64700 1 -- 172.21.15.66:0/2824133476 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008004300 con 0x7fd00c108320 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.060+0000 7fd010f64700 1 -- 172.21.15.66:0/2824133476 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0080036a0 con 0x7fd00c108320 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 -- 172.21.15.66:0/2824133476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 msgr2=0x7fd00c108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 --2- 172.21.15.66:0/2824133476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c108740 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd0080098a0 tx=0x7fd008009bb0 comp rx=0 tx=0).stop 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 -- 172.21.15.66:0/2824133476 shutdown_connections 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 --2- 172.21.15.66:0/2824133476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c108740 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 -- 172.21.15.66:0/2824133476 >> 172.21.15.66:0/2824133476 conn(0x7fd00c1038e0 msgr2=0x7fd00c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 -- 172.21.15.66:0/2824133476 shutdown_connections 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 -- 172.21.15.66:0/2824133476 wait complete. 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.061+0000 7fd0141ca700 1 Processor -- start 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd0141ca700 1 -- start start 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd0141ca700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd0141ca700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c109a50 con 0x7fd00c108320 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd011f66700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd011f66700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42638/0 (socket says 172.21.15.66:42638) 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd011f66700 1 -- 172.21.15.66:0/1022813459 learned_addr learned my addr 172.21.15.66:0/1022813459 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd011f66700 1 -- 172.21.15.66:0/1022813459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c19bed0 con 0x7fd00c108320 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd011f66700 1 --2- 172.21.15.66:0/1022813459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c19b990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd008000c00 tx=0x7fd008004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd008003db0 con 0x7fd00c108320 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008011630 con 0x7fd00c108320 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd008011950 con 0x7fd00c108320 2024-04-21T08:57:43.174 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c19c160 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.062+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c19ee50 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.063+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd008011af0 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.063+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd00801c450 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.063+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c0640a0 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.066+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd008032020 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.137+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd00c19c640 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.140+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fd00c19c640 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.140+0000 7fd002ffd700 1 -- 172.21.15.66:0/1022813459 <== mon.0 v2:172.21.15.66:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd008004260 con 0x7fd00c108320 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.141+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 msgr2=0x7fd00c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.141+0000 7fd0141ca700 1 --2- 172.21.15.66:0/1022813459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c19b990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd008000c00 tx=0x7fd008004750 comp rx=0 tx=0).stop 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.141+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 shutdown_connections 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.141+0000 7fd0141ca700 1 --2- 172.21.15.66:0/1022813459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd00c108320 0x7fd00c19b990 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:43.175 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.141+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 >> 172.21.15.66:0/1022813459 conn(0x7fd00c1038e0 msgr2=0x7fd00c192630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:43.176 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.141+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 shutdown_connections 2024-04-21T08:57:43.176 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:37.141+0000 7fd0141ca700 1 -- 172.21.15.66:0/1022813459 wait complete. 2024-04-21T08:57:43.176 INFO:teuthology.orchestra.run.smithi066.stdout:Generating new minimal ceph.conf... 2024-04-21T08:57:46.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:46 smithi066 ceph-mon[27573]: from='client.? 172.21.15.66:0/3129798859' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.012+0000 7f1cfe04e700 1 Processor -- start 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.012+0000 7f1cfe04e700 1 -- start start 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.012+0000 7f1cfe04e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf81087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.012+0000 7f1cfe04e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8108da0 con 0x7f1cf81083b0 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.012+0000 7f1cf77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf81087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.012+0000 7f1cf77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf81087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41632/0 (socket says 172.21.15.66:41632) 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.012+0000 7f1cf77fe700 1 -- 172.21.15.66:0/336960744 learned_addr learned my addr 172.21.15.66:0/336960744 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:47.905 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.013+0000 7f1cf77fe700 1 -- 172.21.15.66:0/336960744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf8109560 con 0x7f1cf81083b0 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.013+0000 7f1cf77fe700 1 --2- 172.21.15.66:0/336960744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf81087d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00098a0 tx=0x7f1ce0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e36ae43d73a2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.013+0000 7f1cf67fc700 1 -- 172.21.15.66:0/336960744 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ce0004160 con 0x7f1cf81083b0 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.013+0000 7f1cf67fc700 1 -- 172.21.15.66:0/336960744 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ce0004300 con 0x7f1cf81083b0 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.013+0000 7f1cf67fc700 1 -- 172.21.15.66:0/336960744 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ce0011af0 con 0x7f1cf81083b0 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.013+0000 7f1cfe04e700 1 -- 172.21.15.66:0/336960744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 msgr2=0x7f1cf81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.013+0000 7f1cfe04e700 1 --2- 172.21.15.66:0/336960744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf81087d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00098a0 tx=0x7f1ce0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.014+0000 7f1cfe04e700 1 -- 172.21.15.66:0/336960744 shutdown_connections 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.014+0000 7f1cfe04e700 1 --2- 172.21.15.66:0/336960744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf81087d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.014+0000 7f1cfe04e700 1 -- 172.21.15.66:0/336960744 >> 172.21.15.66:0/336960744 conn(0x7f1cf81039b0 msgr2=0x7f1cf8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.014+0000 7f1cfe04e700 1 -- 172.21.15.66:0/336960744 shutdown_connections 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.014+0000 7f1cfe04e700 1 -- 172.21.15.66:0/336960744 wait complete. 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.014+0000 7f1cfe04e700 1 Processor -- start 2024-04-21T08:57:47.906 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.014+0000 7f1cfe04e700 1 -- start start 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cfe04e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cfe04e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810c910 con 0x7f1cf81083b0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41638/0 (socket says 172.21.15.66:41638) 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf77fe700 1 -- 172.21.15.66:0/3129798859 learned_addr learned my addr 172.21.15.66:0/3129798859 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf77fe700 1 -- 172.21.15.66:0/3129798859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf819c790 con 0x7f1cf81083b0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf77fe700 1 --2- 172.21.15.66:0/3129798859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf819c250 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0003f40 tx=0x7f1ce0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf4ff9700 1 -- 172.21.15.66:0/3129798859 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ce00036a0 con 0x7f1cf81083b0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf4ff9700 1 -- 172.21.15.66:0/3129798859 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ce0024430 con 0x7f1cf81083b0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf819ca20 con 0x7f1cf81083b0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cf4ff9700 1 -- 172.21.15.66:0/3129798859 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ce001a5c0 con 0x7f1cf81083b0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.015+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf819f710 con 0x7f1cf81083b0 2024-04-21T08:57:47.907 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.016+0000 7f1cf4ff9700 1 -- 172.21.15.66:0/3129798859 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ce001a870 con 0x7f1cf81083b0 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.016+0000 7f1cf4ff9700 1 -- 172.21.15.66:0/3129798859 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ce002daf0 con 0x7f1cf81083b0 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.017+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf80640a0 con 0x7f1cf81083b0 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.019+0000 7f1cf4ff9700 1 -- 172.21.15.66:0/3129798859 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1ce0043e50 con 0x7f1cf81083b0 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.090+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1cf819f8e0 con 0x7f1cf81083b0 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.090+0000 7f1cf4ff9700 1 -- 172.21.15.66:0/3129798859 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f1ce0032030 con 0x7f1cf81083b0 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.091+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 msgr2=0x7f1cf819c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.091+0000 7f1cfe04e700 1 --2- 172.21.15.66:0/3129798859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf819c250 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0003f40 tx=0x7f1ce0004790 comp rx=0 tx=0).stop 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.091+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 shutdown_connections 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.091+0000 7f1cfe04e700 1 --2- 172.21.15.66:0/3129798859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cf81083b0 0x7f1cf819c250 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.091+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 >> 172.21.15.66:0/3129798859 conn(0x7f1cf81039b0 msgr2=0x7f1cf81044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.091+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 shutdown_connections 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:46.091+0000 7f1cfe04e700 1 -- 172.21.15.66:0/3129798859 wait complete. 2024-04-21T08:57:47.908 INFO:teuthology.orchestra.run.smithi066.stdout:Restarting the monitor... 2024-04-21T08:57:48.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:47 smithi066 systemd[1]: Stopping Ceph mon.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T08:57:48.593 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:48 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a[27569]: 2024-04-21T08:57:48.192+0000 7fe8e57f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:57:48.593 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:48 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a[27569]: 2024-04-21T08:57:48.192+0000 7fe8e57f5700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-21T08:57:48.593 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:48 smithi066 podman[27823]: 2024-04-21 08:57:48.34647015 +0000 UTC m=+0.257329346 container died e904528e3793a4e1c855fc6a1415814e8b48e591704b034530608d1feab6a576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-21T08:57:48.868 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:48 smithi066 podman[27823]: 2024-04-21 08:57:48.676545466 +0000 UTC m=+0.587404645 container cleanup e904528e3793a4e1c855fc6a1415814e8b48e591704b034530608d1feab6a576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:57:48.868 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:48 smithi066 bash[27823]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a 2024-04-21T08:57:49.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:48 smithi066 podman[27837]: 2024-04-21 08:57:48.869042252 +0000 UTC m=+0.521233943 container remove e904528e3793a4e1c855fc6a1415814e8b48e591704b034530608d1feab6a576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T08:57:49.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 systemd[1]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-21T08:57:49.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 systemd[1]: Stopped Ceph mon.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T08:57:49.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 systemd[1]: Starting Ceph mon.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T08:57:49.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 podman[27933]: 2024-04-21 08:57:49.23839996 +0000 UTC m=+0.018150379 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:57:49.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 podman[27933]: 2024-04-21 08:57:49.336705082 +0000 UTC m=+0.116455481 container create 8ad485c04e099fdaa3858d19741662bb72a551f19a99292213fda28bcd976c03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:57:49.575 INFO:teuthology.orchestra.run.smithi066.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-21T08:57:49.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 podman[27933]: 2024-04-21 08:57:49.514033635 +0000 UTC m=+0.293784035 container init 8ad485c04e099fdaa3858d19741662bb72a551f19a99292213fda28bcd976c03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-21T08:57:49.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 podman[27933]: 2024-04-21 08:57:49.517093456 +0000 UTC m=+0.296843857 container start 8ad485c04e099fdaa3858d19741662bb72a551f19a99292213fda28bcd976c03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-21T08:57:49.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T08:57:49.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T08:57:49.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: pidfile_write: ignore empty --pid-file 2024-04-21T08:57:49.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: load: jerasure load: lrc 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: RocksDB version: 7.9.2 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Git sha 0 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: DB SUMMARY 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: DB Session ID: ED7DVQOJCVJK68RWJBVU 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: CURRENT file: CURRENT 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104534 ; 2024-04-21T08:57:49.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.error_if_exists: 0 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.create_if_missing: 0 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.paranoid_checks: 1 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.env: 0x5581ea3afcc0 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.info_log: 0x5581eb3c5be0 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T08:57:49.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.statistics: (nil) 2024-04-21T08:57:49.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.use_fsync: 0 2024-04-21T08:57:49.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_log_file_size: 0 2024-04-21T08:57:49.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T08:57:49.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T08:57:49.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T08:57:49.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T08:57:49.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.allow_fallocate: 1 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.use_direct_reads: 0 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.db_log_dir: 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.wal_dir: 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.write_buffer_manager: 0x5581eb3590e0 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.unordered_write: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.row_cache: None 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.wal_filter: None 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.two_write_queues: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.wal_compression: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.atomic_flush: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.log_readahead_size: 0 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_background_jobs: 2 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_background_compactions: -1 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_subcompactions: 1 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T08:57:49.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_open_files: -1 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_background_flushes: -1 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Compression algorithms supported: 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T08:57:49.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kZSTD supported: 0 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kXpressCompression supported: 0 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kZlibCompression supported: 1 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kSnappyCompression supported: 1 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kLZ4Compression supported: 1 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: kBZip2Compression supported: 0 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.merge_operator: 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_filter: None 2024-04-21T08:57:49.712 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5581eb3c5dc0) 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks: 1 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_top_level_index_and_filter: 1 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_type: 0 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_index_type: 0 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_shortening: 1 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: checksum: 4 2024-04-21T08:57:49.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: no_block_cache: 0 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache: 0x5581eb38b1f0 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_name: BinnedLRUCache 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_options: 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: capacity : 536870912 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: num_shard_bits : 4 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: strict_capacity_limit : 0 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: high_pri_pool_ratio: 0.000 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_compressed: (nil) 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: persistent_cache: (nil) 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size: 4096 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size_deviation: 10 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_restart_interval: 16 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_block_restart_interval: 1 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: metadata_block_size: 4096 2024-04-21T08:57:49.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: partition_filters: 0 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: use_delta_encoding: 1 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: filter_policy: bloomfilter 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: whole_key_filtering: 1 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: verify_compression: 0 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: read_amp_bytes_per_bit: 0 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: format_version: 5 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_index_compression: 1 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_align: 0 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: max_auto_readahead_size: 262144 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: prepopulate_block_cache: 0 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: initial_auto_readahead_size: 8192 2024-04-21T08:57:49.715 INFO:journalctl@ceph.mon.a.smithi066.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression: NoCompression 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.num_levels: 7 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T08:57:49.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:57:49.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T08:57:49.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T08:57:49.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.inplace_update_support: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.bloom_locality: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.max_successive_merges: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.ttl: 2592000 2024-04-21T08:57:49.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.enable_blob_files: false 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.min_blob_size: 0 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T08:57:49.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1017fded-8083-4d8a-a7d8-e5670ac15f32 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713689869545975, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713689869547114, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99761, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97643, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14028, "raw_average_key_size": 49, "raw_value_size": 90001, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713689869, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1017fded-8083-4d8a-a7d8-e5670ac15f32", "db_session_id": "ED7DVQOJCVJK68RWJBVU", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713689869547214, "job": 1, "event": "recovery_finished"} 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5581eb4a0000 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: DB pointer 0x5581eb48a000 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T08:57:49.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** DB Stats ** 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T08:57:49.723 INFO:journalctl@ceph.mon.a.smithi066.stdout: L0 2/0 99.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: Sum 2/0 99.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T08:57:49.724 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative compaction: 0.00 GB write, 29.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval compaction: 0.00 GB write, 29.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: Block cache BinnedLRUCache@0x5581eb38b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: starting mon.a rank 0 at public addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] at bind addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???) e1 preinit fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???).mds e1 new map 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???).mds e1 print_map 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: e1 2024-04-21T08:57:49.725 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout: legacy client fscid: -1 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout: No filesystems configured 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 bash[27933]: 8ad485c04e099fdaa3858d19741662bb72a551f19a99292213fda28bcd976c03 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 systemd[1]: Started Ceph mon.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:57:49.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: monmap epoch 1 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: last_changed 2024-04-21T08:57:31.934215+0000 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: min_mon_release 19 (squid) 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: election_strategy: 1 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: fsmap 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: osdmap e1: 0 total, 0 up, 0 in 2024-04-21T08:57:49.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:49 smithi066 ceph-mon[27953]: mgrmap e1: no daemons active 2024-04-21T08:57:50.689 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.099+0000 7f3257c64700 1 Processor -- start 2024-04-21T08:57:50.689 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.099+0000 7f3257c64700 1 -- start start 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.099+0000 7f3257c64700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.099+0000 7f3257c64700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f325007c8e0 con 0x7f325007de60 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f3255a00700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f3255a00700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41642/0 (socket says 172.21.15.66:41642) 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f3255a00700 1 -- 172.21.15.66:0/2850604594 learned_addr learned my addr 172.21.15.66:0/2850604594 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f3255a00700 1 -- 172.21.15.66:0/2850604594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325007ca60 con 0x7f325007de60 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f3255a00700 1 --2- 172.21.15.66:0/2850604594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325007c310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f324c0098a0 tx=0x7f324c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d182c0447b14d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f32549fe700 1 -- 172.21.15.66:0/2850604594 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c004160 con 0x7f325007de60 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f32549fe700 1 -- 172.21.15.66:0/2850604594 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f324c004300 con 0x7f325007de60 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.100+0000 7f32549fe700 1 -- 172.21.15.66:0/2850604594 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c011af0 con 0x7f325007de60 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.101+0000 7f3257c64700 1 -- 172.21.15.66:0/2850604594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 msgr2=0x7f325007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:50.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.101+0000 7f3257c64700 1 --2- 172.21.15.66:0/2850604594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325007c310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f324c0098a0 tx=0x7f324c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.101+0000 7f3257c64700 1 -- 172.21.15.66:0/2850604594 shutdown_connections 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.101+0000 7f3257c64700 1 --2- 172.21.15.66:0/2850604594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325007c310 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.101+0000 7f3257c64700 1 -- 172.21.15.66:0/2850604594 >> 172.21.15.66:0/2850604594 conn(0x7f32501039b0 msgr2=0x7f3250105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.101+0000 7f3257c64700 1 -- 172.21.15.66:0/2850604594 shutdown_connections 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.101+0000 7f3257c64700 1 -- 172.21.15.66:0/2850604594 wait complete. 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3257c64700 1 Processor -- start 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3257c64700 1 -- start start 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3257c64700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3257c64700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f325010e8e0 con 0x7f325007de60 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3255a00700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3255a00700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325019e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41658/0 (socket says 172.21.15.66:41658) 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3255a00700 1 -- 172.21.15.66:0/2408483173 learned_addr learned my addr 172.21.15.66:0/2408483173 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:50.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3255a00700 1 -- 172.21.15.66:0/2408483173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325019e8f0 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.102+0000 7f3255a00700 1 --2- 172.21.15.66:0/2408483173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325019e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f324c003f40 tx=0x7f324c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.103+0000 7f3246ffd700 1 -- 172.21.15.66:0/2408483173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c0038b0 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.103+0000 7f3246ffd700 1 -- 172.21.15.66:0/2408483173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f324c003a50 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.103+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325019eb80 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.103+0000 7f3246ffd700 1 -- 172.21.15.66:0/2408483173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c01a5c0 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.103+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32501a1870 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.103+0000 7f3246ffd700 1 -- 172.21.15.66:0/2408483173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f324c01ac50 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.103+0000 7f3246ffd700 1 -- 172.21.15.66:0/2408483173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f324c019700 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.104+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32500640a0 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.106+0000 7f3246ffd700 1 -- 172.21.15.66:0/2408483173 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f324c019900 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.177+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f32501a1a40 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.179+0000 7f3246ffd700 1 -- 172.21.15.66:0/2408483173 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f324c019ae0 con 0x7f325007de60 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.179+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 msgr2=0x7f325019e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.179+0000 7f3257c64700 1 --2- 172.21.15.66:0/2408483173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325019e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f324c003f40 tx=0x7f324c011890 comp rx=0 tx=0).stop 2024-04-21T08:57:50.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.179+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 shutdown_connections 2024-04-21T08:57:50.693 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.180+0000 7f3257c64700 1 --2- 172.21.15.66:0/2408483173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f325007de60 0x7f325019e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:50.693 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.180+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 >> 172.21.15.66:0/2408483173 conn(0x7f32501039b0 msgr2=0x7f3250104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:50.693 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.180+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 shutdown_connections 2024-04-21T08:57:50.693 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:50.180+0000 7f3257c64700 1 -- 172.21.15.66:0/2408483173 wait complete. 2024-04-21T08:57:50.693 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-21T08:57:50.715 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:57:50.715 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mgr... 2024-04-21T08:57:50.715 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-21T08:57:50.716 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-21T08:57:50.966 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.x 2024-04-21T08:57:50.966 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.x.service: Unit ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-21T08:57:51.119 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968.target.wants/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@.service. 2024-04-21T08:57:51.359 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:51 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2408483173' entity='client.admin' 2024-04-21T08:57:51.620 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:51 smithi066 podman[28172]: 2024-04-21 08:57:51.556609243 +0000 UTC m=+0.311102919 container start d5f0a88d9d49850e725280cb02f1c415f8cf66ed958e37ceb0a1e3bc0b3dfaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T08:57:51.658 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:57:51.658 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-04-21T08:57:51.665 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:57:51.665 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-04-21T08:57:51.665 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-04-21T08:57:51.665 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:57:51.674 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:57:51.674 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-04-21T08:57:51.682 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:57:51.682 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-04-21T08:57:51.682 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-04-21T08:57:51.682 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-21T08:57:51.682 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr to start... 2024-04-21T08:57:51.682 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr... 2024-04-21T08:57:51.911 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:51 smithi066 bash[28172]: d5f0a88d9d49850e725280cb02f1c415f8cf66ed958e37ceb0a1e3bc0b3dfaaf 2024-04-21T08:57:51.911 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:51 smithi066 systemd[1]: Started Ceph mgr.x for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T08:57:51.911 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:51 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:51.715+0000 7f998b841200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:57:51.911 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:51 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:51.911+0000 7f998b841200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:57:52.206 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:52 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:52.023+0000 7f998b841200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:57:52.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:52 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3324276990' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:57:52.620 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:52 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:52.359+0000 7f998b841200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:57:52.620 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:52 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:52.451+0000 7f998b841200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:57:53.344 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:57:53.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:53.348 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:57:53.349 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:57:53.349 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:57:53.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:53.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:53.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:57:34.912970+0000", 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.126+0000 7f6eae48d700 1 Processor -- start 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.126+0000 7f6eae48d700 1 -- start start 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.126+0000 7f6eae48d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea8108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.126+0000 7f6eae48d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8108d30 con 0x7f6ea8108340 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.126+0000 7f6ea7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea8108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:53.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.127+0000 7f6ea7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea8108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41672/0 (socket says 172.21.15.66:41672) 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.127+0000 7f6ea7fff700 1 -- 172.21.15.66:0/922648721 learned_addr learned my addr 172.21.15.66:0/922648721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.127+0000 7f6ea7fff700 1 -- 172.21.15.66:0/922648721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea8109540 con 0x7f6ea8108340 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.127+0000 7f6ea7fff700 1 --2- 172.21.15.66:0/922648721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea8108760 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e900098a0 tx=0x7f6e90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275e5da52b332c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.127+0000 7f6ea6ffd700 1 -- 172.21.15.66:0/922648721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e90004160 con 0x7f6ea8108340 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.127+0000 7f6ea6ffd700 1 -- 172.21.15.66:0/922648721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e90004300 con 0x7f6ea8108340 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.127+0000 7f6ea6ffd700 1 -- 172.21.15.66:0/922648721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e90011af0 con 0x7f6ea8108340 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.128+0000 7f6eae48d700 1 -- 172.21.15.66:0/922648721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 msgr2=0x7f6ea8108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.128+0000 7f6eae48d700 1 --2- 172.21.15.66:0/922648721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea8108760 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e900098a0 tx=0x7f6e90009bb0 comp rx=0 tx=0).stop 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.128+0000 7f6eae48d700 1 -- 172.21.15.66:0/922648721 shutdown_connections 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.128+0000 7f6eae48d700 1 --2- 172.21.15.66:0/922648721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea8108760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.128+0000 7f6eae48d700 1 -- 172.21.15.66:0/922648721 >> 172.21.15.66:0/922648721 conn(0x7f6ea81038c0 msgr2=0x7f6ea8105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.128+0000 7f6eae48d700 1 -- 172.21.15.66:0/922648721 shutdown_connections 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.128+0000 7f6eae48d700 1 -- 172.21.15.66:0/922648721 wait complete. 2024-04-21T08:57:53.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6eae48d700 1 Processor -- start 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6eae48d700 1 -- start start 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6eae48d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6eae48d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810c8f0 con 0x7f6ea8108340 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6ea7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6ea7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41684/0 (socket says 172.21.15.66:41684) 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6ea7fff700 1 -- 172.21.15.66:0/3324276990 learned_addr learned my addr 172.21.15.66:0/3324276990 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.129+0000 7f6ea7fff700 1 -- 172.21.15.66:0/3324276990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea819c750 con 0x7f6ea8108340 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6ea7fff700 1 --2- 172.21.15.66:0/3324276990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea819c210 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e90003f40 tx=0x7f6e90011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6ea57fa700 1 -- 172.21.15.66:0/3324276990 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e900038b0 con 0x7f6ea8108340 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6ea57fa700 1 -- 172.21.15.66:0/3324276990 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e90003a50 con 0x7f6ea8108340 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea819c9e0 con 0x7f6ea8108340 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea819f6d0 con 0x7f6ea8108340 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6ea57fa700 1 -- 172.21.15.66:0/3324276990 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e9001a5c0 con 0x7f6ea8108340 2024-04-21T08:57:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6ea57fa700 1 -- 172.21.15.66:0/3324276990 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e9001ac00 con 0x7f6ea8108340 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.130+0000 7f6ea57fa700 1 -- 172.21.15.66:0/3324276990 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e900199d0 con 0x7f6ea8108340 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.131+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea80640a0 con 0x7f6ea8108340 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.133+0000 7f6ea57fa700 1 -- 172.21.15.66:0/3324276990 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6e90019bd0 con 0x7f6ea8108340 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.206+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6ea819f8a0 con 0x7f6ea8108340 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.207+0000 7f6ea57fa700 1 -- 172.21.15.66:0/3324276990 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6e90019db0 con 0x7f6ea8108340 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.208+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 msgr2=0x7f6ea819c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.208+0000 7f6eae48d700 1 --2- 172.21.15.66:0/3324276990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea819c210 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e90003f40 tx=0x7f6e90011890 comp rx=0 tx=0).stop 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.208+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 shutdown_connections 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.208+0000 7f6eae48d700 1 --2- 172.21.15.66:0/3324276990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ea8108340 0x7f6ea819c210 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.208+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 >> 172.21.15.66:0/3324276990 conn(0x7f6ea81038c0 msgr2=0x7f6ea8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.208+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 shutdown_connections 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:52.208+0000 7f6eae48d700 1 -- 172.21.15.66:0/3324276990 wait complete. 2024-04-21T08:57:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (1/15)... 2024-04-21T08:57:54.100 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:53 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:53.731+0000 7f998b841200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:57:54.100 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:53 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:53.911+0000 7f998b841200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:57:54.100 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:54 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:54.101+0000 7f998b841200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:57:54.719 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:54 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:54.402+0000 7f998b841200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:57:54.719 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:54 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:54.499+0000 7f998b841200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:57:54.719 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:54 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:54.719+0000 7f998b841200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:57:55.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:54 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:54.920+0000 7f998b841200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:57:55.671 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:55 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:55.569+0000 7f998b841200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:57:56.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:56 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3609469092' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:57:56.370 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:56 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:56.059+0000 7f998b841200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:57:56.370 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:56 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:56.161+0000 7f998b841200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:57:56.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:56.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:57:56.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T08:57:56.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:57:56.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:57:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:56.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:57:56.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:57:34.912970+0000", 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.906+0000 7f810be83700 1 Processor -- start 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f810be83700 1 -- start start 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f810be83700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f8104108830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f810be83700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8104108e00 con 0x7f8104108410 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f8109c1f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f8104108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f8109c1f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f8104108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47136/0 (socket says 172.21.15.66:47136) 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f8109c1f700 1 -- 172.21.15.66:0/789061745 learned_addr learned my addr 172.21.15.66:0/789061745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:56.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f8109c1f700 1 -- 172.21.15.66:0/789061745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8104109610 con 0x7f8104108410 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.907+0000 7f8109c1f700 1 --2- 172.21.15.66:0/789061745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f8104108830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81000098a0 tx=0x7f8100009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab331d46a4d888f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f8108c1d700 1 -- 172.21.15.66:0/789061745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8100004160 con 0x7f8104108410 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f8108c1d700 1 -- 172.21.15.66:0/789061745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8100004300 con 0x7f8104108410 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f8108c1d700 1 -- 172.21.15.66:0/789061745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8100011af0 con 0x7f8104108410 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f810be83700 1 -- 172.21.15.66:0/789061745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 msgr2=0x7f8104108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f810be83700 1 --2- 172.21.15.66:0/789061745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f8104108830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81000098a0 tx=0x7f8100009bb0 comp rx=0 tx=0).stop 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f810be83700 1 -- 172.21.15.66:0/789061745 shutdown_connections 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f810be83700 1 --2- 172.21.15.66:0/789061745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f8104108830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.908+0000 7f810be83700 1 -- 172.21.15.66:0/789061745 >> 172.21.15.66:0/789061745 conn(0x7f8104103990 msgr2=0x7f8104105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.909+0000 7f810be83700 1 -- 172.21.15.66:0/789061745 shutdown_connections 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.909+0000 7f810be83700 1 -- 172.21.15.66:0/789061745 wait complete. 2024-04-21T08:57:56.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.909+0000 7f810be83700 1 Processor -- start 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.909+0000 7f810be83700 1 -- start start 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.909+0000 7f810be83700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f810419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.909+0000 7f810be83700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f810410c9c0 con 0x7f8104108410 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f8109c1f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f810419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f8109c1f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f810419e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47140/0 (socket says 172.21.15.66:47140) 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f8109c1f700 1 -- 172.21.15.66:0/3609469092 learned_addr learned my addr 172.21.15.66:0/3609469092 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f8109c1f700 1 -- 172.21.15.66:0/3609469092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810419ea70 con 0x7f8104108410 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f8109c1f700 1 --2- 172.21.15.66:0/3609469092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f810419e530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f8100011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f80faffd700 1 -- 172.21.15.66:0/3609469092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8100003b20 con 0x7f8104108410 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810419ed00 con 0x7f8104108410 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f80faffd700 1 -- 172.21.15.66:0/3609469092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f810001a8c0 con 0x7f8104108410 2024-04-21T08:57:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f80faffd700 1 -- 172.21.15.66:0/3609469092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8100024940 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041a19f0 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.910+0000 7f80faffd700 1 -- 172.21.15.66:0/3609469092 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8100022020 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.911+0000 7f80faffd700 1 -- 172.21.15.66:0/3609469092 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f810002dde0 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.911+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81040640a0 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:55.914+0000 7f80faffd700 1 -- 172.21.15.66:0/3609469092 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f810001aa70 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.013+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f81041a1e90 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.014+0000 7f80faffd700 1 -- 172.21.15.66:0/3609469092 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f810001a430 con 0x7f8104108410 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.015+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 msgr2=0x7f810419e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.015+0000 7f810be83700 1 --2- 172.21.15.66:0/3609469092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f810419e530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f8100011b70 comp rx=0 tx=0).stop 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.015+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 shutdown_connections 2024-04-21T08:57:56.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.015+0000 7f810be83700 1 --2- 172.21.15.66:0/3609469092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8104108410 0x7f810419e530 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:56.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.015+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 >> 172.21.15.66:0/3609469092 conn(0x7f8104103990 msgr2=0x7f81041043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:56.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.015+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 shutdown_connections 2024-04-21T08:57:56.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:56.015+0000 7f810be83700 1 -- 172.21.15.66:0/3609469092 wait complete. 2024-04-21T08:57:56.519 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (2/15)... 2024-04-21T08:57:56.869 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:56 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:56.376+0000 7f998b841200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:57:57.274 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:56 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:56.925+0000 7f998b841200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:57:57.274 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:57 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:57.021+0000 7f998b841200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:57:57.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:57 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:57.274+0000 7f998b841200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:57:57.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:57 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:57.413+0000 7f998b841200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:57:57.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:57 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:57.512+0000 7f998b841200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:57:57.945 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:57 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:57.633+0000 7f998b841200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:57:58.369 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:57 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:57.946+0000 7f998b841200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:57:58.836 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:58.584+0000 7f998b841200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:57:58.836 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:57:58.813+0000 7f998b841200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: Activating manager daemon x 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: mgrmap e2: x(active, starting, since 0.00305481s) 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: Manager daemon x is now available 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:57:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' 2024-04-21T08:57:59.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:58 smithi066 ceph-mon[27953]: from='mgr.14100 172.21.15.66:0/4071534242' entity='mgr.x' 2024-04-21T08:57:59.733 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:57:59.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:57:59.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:57:59.743 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:57:59.743 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:57:59.743 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:57:59.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:57:34.912970+0000", 2024-04-21T08:57:59.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.937+0000 7fd533757700 1 Processor -- start 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.937+0000 7fd533757700 1 -- start start 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.937+0000 7fd533757700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c1087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.937+0000 7fd533757700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c108da0 con 0x7fd52c1083b0 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.937+0000 7fd5314f3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c1087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.937+0000 7fd5314f3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c1087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47218/0 (socket says 172.21.15.66:47218) 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.937+0000 7fd5314f3700 1 -- 172.21.15.66:0/595492468 learned_addr learned my addr 172.21.15.66:0/595492468 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.938+0000 7fd5314f3700 1 -- 172.21.15.66:0/595492468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c109560 con 0x7fd52c1083b0 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.938+0000 7fd5314f3700 1 --2- 172.21.15.66:0/595492468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c1087d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5280098a0 tx=0x7fd528009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952c3ec3c2ead537 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.938+0000 7fd523fff700 1 -- 172.21.15.66:0/595492468 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd528004160 con 0x7fd52c1083b0 2024-04-21T08:57:59.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.938+0000 7fd523fff700 1 -- 172.21.15.66:0/595492468 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd528004300 con 0x7fd52c1083b0 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.938+0000 7fd523fff700 1 -- 172.21.15.66:0/595492468 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd528011af0 con 0x7fd52c1083b0 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 -- 172.21.15.66:0/595492468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 msgr2=0x7fd52c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 --2- 172.21.15.66:0/595492468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c1087d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5280098a0 tx=0x7fd528009bb0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 -- 172.21.15.66:0/595492468 shutdown_connections 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 --2- 172.21.15.66:0/595492468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c1087d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 -- 172.21.15.66:0/595492468 >> 172.21.15.66:0/595492468 conn(0x7fd52c1039b0 msgr2=0x7fd52c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 -- 172.21.15.66:0/595492468 shutdown_connections 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 -- 172.21.15.66:0/595492468 wait complete. 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 Processor -- start 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.939+0000 7fd533757700 1 -- start start 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd533757700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd533757700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c10c910 con 0x7fd52c1083b0 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5314f3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5314f3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47232/0 (socket says 172.21.15.66:47232) 2024-04-21T08:57:59.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5314f3700 1 -- 172.21.15.66:0/3466518745 learned_addr learned my addr 172.21.15.66:0/3466518745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5314f3700 1 -- 172.21.15.66:0/3466518745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c19c790 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5314f3700 1 --2- 172.21.15.66:0/3466518745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c19c250 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd528000c00 tx=0x7fd528011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5227fc700 1 -- 172.21.15.66:0/3466518745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5280039e0 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5227fc700 1 -- 172.21.15.66:0/3466518745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd528003b80 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c19ca20 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd5227fc700 1 -- 172.21.15.66:0/3466518745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd528024730 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.940+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c19f710 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.941+0000 7fd5227fc700 1 -- 172.21.15.66:0/3466518745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fd528022020 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.942+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c0640a0 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.942+0000 7fd5227fc700 1 -- 172.21.15.66:0/3466518745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd52804cfa0 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:58.944+0000 7fd5227fc700 1 -- 172.21.15.66:0/3466518745 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd52801ab50 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.017+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd52c19f8e0 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd5227fc700 1 -- 172.21.15.66:0/3466518745 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd52c19f8e0 con 0x7fd52c1083b0 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 msgr2=0x7fd52c19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd533757700 1 --2- 172.21.15.66:0/3466518745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c19c250 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd528000c00 tx=0x7fd528011b70 comp rx=0 tx=0).stop 2024-04-21T08:57:59.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 shutdown_connections 2024-04-21T08:57:59.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd533757700 1 --2- 172.21.15.66:0/3466518745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd52c1083b0 0x7fd52c19c250 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 >> 172.21.15.66:0/3466518745 conn(0x7fd52c1039b0 msgr2=0x7fd52c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 shutdown_connections 2024-04-21T08:57:59.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:57:59.018+0000 7fd533757700 1 -- 172.21.15.66:0/3466518745 wait complete. 2024-04-21T08:57:59.749 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (3/15)... 2024-04-21T08:58:00.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:59 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3466518745' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:58:00.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:57:59 smithi066 ceph-mon[27953]: mgrmap e3: x(active, since 1.00754s) 2024-04-21T08:58:02.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:01 smithi066 ceph-mon[27953]: mgrmap e4: x(active, since 2s) 2024-04-21T08:58:03.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/18088723' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:58:03.064 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:58:03.064 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:58:03.064 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T08:58:03.064 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:58:03.064 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:58:03.064 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:58:03.064 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:58:03.065 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:58:03.066 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:58:03.067 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:58:03.068 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:57:34.912970+0000", 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.203+0000 7f105de72700 1 Processor -- start 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.203+0000 7f105de72700 1 -- start start 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.203+0000 7f105de72700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f10581087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.203+0000 7f105de72700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1058108da0 con 0x7f10581083b0 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10577fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f10581087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10577fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f10581087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47242/0 (socket says 172.21.15.66:47242) 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10577fe700 1 -- 172.21.15.66:0/1787278678 learned_addr learned my addr 172.21.15.66:0/1787278678 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:03.069 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10577fe700 1 -- 172.21.15.66:0/1787278678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058109560 con 0x7f10581083b0 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10577fe700 1 --2- 172.21.15.66:0/1787278678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f10581087d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10480098a0 tx=0x7f1048009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cb6470a131547f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10567fc700 1 -- 172.21.15.66:0/1787278678 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1048004160 con 0x7f10581083b0 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10567fc700 1 -- 172.21.15.66:0/1787278678 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1048004300 con 0x7f10581083b0 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.204+0000 7f10567fc700 1 -- 172.21.15.66:0/1787278678 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1048011af0 con 0x7f10581083b0 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.205+0000 7f105de72700 1 -- 172.21.15.66:0/1787278678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 msgr2=0x7f10581087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.205+0000 7f105de72700 1 --2- 172.21.15.66:0/1787278678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f10581087d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10480098a0 tx=0x7f1048009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.205+0000 7f105de72700 1 -- 172.21.15.66:0/1787278678 shutdown_connections 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.205+0000 7f105de72700 1 --2- 172.21.15.66:0/1787278678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f10581087d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.205+0000 7f105de72700 1 -- 172.21.15.66:0/1787278678 >> 172.21.15.66:0/1787278678 conn(0x7f10581039b0 msgr2=0x7f1058105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.205+0000 7f105de72700 1 -- 172.21.15.66:0/1787278678 shutdown_connections 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.205+0000 7f105de72700 1 -- 172.21.15.66:0/1787278678 wait complete. 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f105de72700 1 Processor -- start 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f105de72700 1 -- start start 2024-04-21T08:58:03.070 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f105de72700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f105819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f105de72700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f105810c910 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f10577fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f105819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f10577fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f105819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47248/0 (socket says 172.21.15.66:47248) 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f10577fe700 1 -- 172.21.15.66:0/18088723 learned_addr learned my addr 172.21.15.66:0/18088723 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.206+0000 7f10577fe700 1 -- 172.21.15.66:0/18088723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105819e940 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.207+0000 7f10577fe700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f105819e400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1048003f40 tx=0x7f1048011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.207+0000 7f1054ff9700 1 -- 172.21.15.66:0/18088723 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10480038b0 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.207+0000 7f1054ff9700 1 -- 172.21.15.66:0/18088723 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1048003a50 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.207+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105819ebd0 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.207+0000 7f1054ff9700 1 -- 172.21.15.66:0/18088723 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f104801a5c0 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.207+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10581a18c0 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.208+0000 7f1054ff9700 1 -- 172.21.15.66:0/18088723 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f104801a870 con 0x7f10581083b0 2024-04-21T08:58:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.208+0000 7f1054ff9700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f1040039620 0x7f104003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.208+0000 7f1054ff9700 1 -- 172.21.15.66:0/18088723 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f104804da30 con 0x7f10581083b0 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.208+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10580640a0 con 0x7f10581083b0 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.208+0000 7f1056ffd700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f1040039620 0x7f104003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.209+0000 7f1056ffd700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f1040039620 0x7f104003bae0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.214+0000 7f1054ff9700 1 -- 172.21.15.66:0/18088723 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1048019c70 con 0x7f10581083b0 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.510+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f10581a1e00 con 0x7f10581083b0 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.511+0000 7f1054ff9700 1 -- 172.21.15.66:0/18088723 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f104802c9c0 con 0x7f10581083b0 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f1040039620 msgr2=0x7f104003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f1040039620 0x7f104003bae0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 msgr2=0x7f105819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f105819e400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1048003f40 tx=0x7f1048011890 comp rx=0 tx=0).stop 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 shutdown_connections 2024-04-21T08:58:03.072 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f1040039620 0x7f104003bae0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.073 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 --2- 172.21.15.66:0/18088723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10581083b0 0x7f105819e400 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.073 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.513+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 >> 172.21.15.66:0/18088723 conn(0x7f10581039b0 msgr2=0x7f1058104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:03.073 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.514+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 shutdown_connections 2024-04-21T08:58:03.073 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:02.514+0000 7f105de72700 1 -- 172.21.15.66:0/18088723 wait complete. 2024-04-21T08:58:03.073 INFO:teuthology.orchestra.run.smithi066.stdout:mgr is available 2024-04-21T08:58:03.973 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:03 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3414045720' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:58:04.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.458+0000 7fc0100ed700 1 Processor -- start 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc0100ed700 1 -- start start 2024-04-21T08:58:04.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc0100ed700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc0081087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc0100ed700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc008108da0 con 0x7fc0081083b0 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc00de89700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc0081087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc00de89700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc0081087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47252/0 (socket says 172.21.15.66:47252) 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc00de89700 1 -- 172.21.15.66:0/3773160613 learned_addr learned my addr 172.21.15.66:0/3773160613 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc00de89700 1 -- 172.21.15.66:0/3773160613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc008109560 con 0x7fc0081083b0 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.459+0000 7fc00de89700 1 --2- 172.21.15.66:0/3773160613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc0081087d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc0040098a0 tx=0x7fc004009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f0f059cea54730 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc00ce87700 1 -- 172.21.15.66:0/3773160613 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc004004160 con 0x7fc0081083b0 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc00ce87700 1 -- 172.21.15.66:0/3773160613 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc004004300 con 0x7fc0081083b0 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc00ce87700 1 -- 172.21.15.66:0/3773160613 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc004011af0 con 0x7fc0081083b0 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc0100ed700 1 -- 172.21.15.66:0/3773160613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 msgr2=0x7fc0081087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc0100ed700 1 --2- 172.21.15.66:0/3773160613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc0081087d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc0040098a0 tx=0x7fc004009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc0100ed700 1 -- 172.21.15.66:0/3773160613 shutdown_connections 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc0100ed700 1 --2- 172.21.15.66:0/3773160613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc0081087d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc0100ed700 1 -- 172.21.15.66:0/3773160613 >> 172.21.15.66:0/3773160613 conn(0x7fc0081039b0 msgr2=0x7fc008105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc0100ed700 1 -- 172.21.15.66:0/3773160613 shutdown_connections 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.460+0000 7fc0100ed700 1 -- 172.21.15.66:0/3773160613 wait complete. 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.461+0000 7fc0100ed700 1 Processor -- start 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.461+0000 7fc0100ed700 1 -- start start 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.461+0000 7fc0100ed700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc00807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.461+0000 7fc0100ed700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc00810c910 con 0x7fc0081083b0 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.461+0000 7fc00de89700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc00807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.461+0000 7fc00de89700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc00807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47266/0 (socket says 172.21.15.66:47266) 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.461+0000 7fc00de89700 1 -- 172.21.15.66:0/3414045720 learned_addr learned my addr 172.21.15.66:0/3414045720 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.462+0000 7fc00de89700 1 -- 172.21.15.66:0/3414045720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00807e0a0 con 0x7fc0081083b0 2024-04-21T08:58:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.462+0000 7fc00de89700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc00807db60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc004000c00 tx=0x7fc004011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.462+0000 7fbffaffd700 1 -- 172.21.15.66:0/3414045720 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0040039e0 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.462+0000 7fbffaffd700 1 -- 172.21.15.66:0/3414045720 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc004003b80 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.462+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00807c290 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.462+0000 7fbffaffd700 1 -- 172.21.15.66:0/3414045720 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc004024730 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.462+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00807c7d0 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.463+0000 7fbffaffd700 1 -- 172.21.15.66:0/3414045720 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc004022020 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.463+0000 7fbffaffd700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fbff4039670 0x7fbff403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.463+0000 7fbffaffd700 1 -- 172.21.15.66:0/3414045720 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc00404d460 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.463+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0080640a0 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.463+0000 7fc00d688700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fbff4039670 0x7fbff403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.464+0000 7fc00d688700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fbff4039670 0x7fbff403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbffc006fd0 tx=0x7fbffc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.469+0000 7fbffaffd700 1 -- 172.21.15.66:0/3414045720 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc00404d1c0 con 0x7fc0081083b0 2024-04-21T08:58:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.676+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc00807ca50 con 0x7fc0081083b0 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.677+0000 7fbffaffd700 1 -- 172.21.15.66:0/3414045720 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fc00401a770 con 0x7fc0081083b0 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fbff4039670 msgr2=0x7fbff403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fbff4039670 0x7fbff403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbffc006fd0 tx=0x7fbffc006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 msgr2=0x7fc00807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc00807db60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc004000c00 tx=0x7fc004011b70 comp rx=0 tx=0).stop 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 shutdown_connections 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fbff4039670 0x7fbff403bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 --2- 172.21.15.66:0/3414045720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0081083b0 0x7fc00807db60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 >> 172.21.15.66:0/3414045720 conn(0x7fc0081039b0 msgr2=0x7fc0081044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.679+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 shutdown_connections 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:03.680+0000 7fc0100ed700 1 -- 172.21.15.66:0/3414045720 wait complete. 2024-04-21T08:58:04.205 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling cephadm module... 2024-04-21T08:58:05.238 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:04 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/791819787' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-21T08:58:05.238 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:05.238 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:05.238 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:05.113+0000 7f534e156200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:58:05.504 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:05.307+0000 7f534e156200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:58:05.505 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:05.414+0000 7f534e156200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.613+0000 7fa9db55d700 1 Processor -- start 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.613+0000 7fa9db55d700 1 -- start start 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9db55d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d41087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9db55d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4108da0 con 0x7fa9d41083b0 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9d92f9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d41087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9d92f9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d41087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47282/0 (socket says 172.21.15.66:47282) 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9d92f9700 1 -- 172.21.15.66:0/4168200741 learned_addr learned my addr 172.21.15.66:0/4168200741 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:05.512 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9d92f9700 1 -- 172.21.15.66:0/4168200741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d4109560 con 0x7fa9d41083b0 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9d92f9700 1 --2- 172.21.15.66:0/4168200741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d41087d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80098a0 tx=0x7fa9c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ad9b7528bd282c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9c7fff700 1 -- 172.21.15.66:0/4168200741 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9c8004160 con 0x7fa9d41083b0 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.614+0000 7fa9c7fff700 1 -- 172.21.15.66:0/4168200741 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa9c8004300 con 0x7fa9d41083b0 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9c7fff700 1 -- 172.21.15.66:0/4168200741 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9c8011af0 con 0x7fa9d41083b0 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9db55d700 1 -- 172.21.15.66:0/4168200741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 msgr2=0x7fa9d41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9db55d700 1 --2- 172.21.15.66:0/4168200741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d41087d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80098a0 tx=0x7fa9c8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9db55d700 1 -- 172.21.15.66:0/4168200741 shutdown_connections 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9db55d700 1 --2- 172.21.15.66:0/4168200741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d41087d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9db55d700 1 -- 172.21.15.66:0/4168200741 >> 172.21.15.66:0/4168200741 conn(0x7fa9d41039b0 msgr2=0x7fa9d4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9db55d700 1 -- 172.21.15.66:0/4168200741 shutdown_connections 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.615+0000 7fa9db55d700 1 -- 172.21.15.66:0/4168200741 wait complete. 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9db55d700 1 Processor -- start 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9db55d700 1 -- start start 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9db55d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:05.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9db55d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d410c910 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9d92f9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9d92f9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d419c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47296/0 (socket says 172.21.15.66:47296) 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9d92f9700 1 -- 172.21.15.66:0/791819787 learned_addr learned my addr 172.21.15.66:0/791819787 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9d92f9700 1 -- 172.21.15.66:0/791819787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d419c730 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.616+0000 7fa9d92f9700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d419c1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.617+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9c8003b20 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.617+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa9c801a8c0 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.617+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d419c9c0 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.617+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9c8024940 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.617+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d419f6b0 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.617+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa9c8022020 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.618+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d40640a0 con 0x7fa9d41083b0 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.618+0000 7fa9c67fc700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fa9c0039670 0x7fa9c003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:05.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.618+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa9c804d570 con 0x7fa9d41083b0 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.618+0000 7fa9d8af8700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fa9c0039670 0x7fa9c003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.618+0000 7fa9d8af8700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fa9c0039670 0x7fa9c003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006fd0 tx=0x7fa9d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.623+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9c804d2d0 con 0x7fa9d41083b0 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.891+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fa9d419f880 con 0x7fa9d41083b0 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.943+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fa9d419f880 con 0x7fa9d41083b0 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.944+0000 7fa9c67fc700 1 -- 172.21.15.66:0/791819787 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fa9c801aae0 con 0x7fa9d41083b0 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fa9c0039670 msgr2=0x7fa9c003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fa9c0039670 0x7fa9c003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006fd0 tx=0x7fa9d0006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 msgr2=0x7fa9d419c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d419c1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c8011b70 comp rx=0 tx=0).stop 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 shutdown_connections 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7fa9c0039670 0x7fa9c003bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 --2- 172.21.15.66:0/791819787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9d41083b0 0x7fa9d419c1f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.945+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 >> 172.21.15.66:0/791819787 conn(0x7fa9d41039b0 msgr2=0x7fa9d4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:05.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.946+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 shutdown_connections 2024-04-21T08:58:05.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:04.946+0000 7fa9db55d700 1 -- 172.21.15.66:0/791819787 wait complete. 2024-04-21T08:58:05.762 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:05.738+0000 7f534e156200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:58:06.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:05 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/791819787' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-21T08:58:06.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:05 smithi066 ceph-mon[27953]: mgrmap e5: x(active, since 6s) 2024-04-21T08:58:06.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:05.829+0000 7f534e156200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.976+0000 7f41e7621700 1 Processor -- start 2024-04-21T08:58:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.976+0000 7f41e7621700 1 -- start start 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.976+0000 7f41e7621700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e010b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.976+0000 7f41e7621700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41e007bea0 con 0x7f41e0108dd0 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.976+0000 7f41e53bd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e010b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.977+0000 7f41e53bd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e010b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46102/0 (socket says 172.21.15.66:46102) 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.977+0000 7f41e53bd700 1 -- 172.21.15.66:0/3426648379 learned_addr learned my addr 172.21.15.66:0/3426648379 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.977+0000 7f41e53bd700 1 -- 172.21.15.66:0/3426648379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e010b700 con 0x7f41e0108dd0 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.977+0000 7f41e53bd700 1 --2- 172.21.15.66:0/3426648379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e010b1c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f41d00098a0 tx=0x7f41d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7705f1ae72fb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.977+0000 7f41d7fff700 1 -- 172.21.15.66:0/3426648379 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41d0004160 con 0x7f41e0108dd0 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.977+0000 7f41d7fff700 1 -- 172.21.15.66:0/3426648379 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f41d0004300 con 0x7f41e0108dd0 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.977+0000 7f41d7fff700 1 -- 172.21.15.66:0/3426648379 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41d0011af0 con 0x7f41e0108dd0 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.978+0000 7f41e7621700 1 -- 172.21.15.66:0/3426648379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 msgr2=0x7f41e010b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.978+0000 7f41e7621700 1 --2- 172.21.15.66:0/3426648379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e010b1c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f41d00098a0 tx=0x7f41d0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.978+0000 7f41e7621700 1 -- 172.21.15.66:0/3426648379 shutdown_connections 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.978+0000 7f41e7621700 1 --2- 172.21.15.66:0/3426648379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e010b1c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.978+0000 7f41e7621700 1 -- 172.21.15.66:0/3426648379 >> 172.21.15.66:0/3426648379 conn(0x7f41e0102820 msgr2=0x7f41e0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.978+0000 7f41e7621700 1 -- 172.21.15.66:0/3426648379 shutdown_connections 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.978+0000 7f41e7621700 1 -- 172.21.15.66:0/3426648379 wait complete. 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e7621700 1 Processor -- start 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e7621700 1 -- start start 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e7621700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e0108650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e7621700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41e010c6d0 con 0x7f41e0108dd0 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e53bd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e0108650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e53bd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e0108650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46108/0 (socket says 172.21.15.66:46108) 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e53bd700 1 -- 172.21.15.66:0/4192258934 learned_addr learned my addr 172.21.15.66:0/4192258934 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.979+0000 7f41e53bd700 1 -- 172.21.15.66:0/4192258934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e0108b90 con 0x7f41e0108dd0 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.980+0000 7f41e53bd700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e0108650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41d0011ac0 tx=0x7f41d0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:06.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.980+0000 7f41d67fc700 1 -- 172.21.15.66:0/4192258934 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41d0003b20 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.980+0000 7f41d67fc700 1 -- 172.21.15.66:0/4192258934 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f41d001a8c0 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.980+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e0106d80 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.980+0000 7f41d67fc700 1 -- 172.21.15.66:0/4192258934 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41d00248d0 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.980+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e01072c0 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.981+0000 7f41d67fc700 1 -- 172.21.15.66:0/4192258934 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f41d0022020 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.981+0000 7f41d67fc700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 0x7f41cc03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.981+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e00640a0 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.981+0000 7f41e4bbc700 1 -- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 msgr2=0x7f41cc03bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3624224927 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.981+0000 7f41d67fc700 1 -- 172.21.15.66:0/4192258934 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f41d004d540 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.981+0000 7f41e4bbc700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 0x7f41cc03bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:05.986+0000 7f41d67fc700 1 -- 172.21.15.66:0/4192258934 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41d004d2a0 con 0x7f41e0108dd0 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.182+0000 7f41e4bbc700 1 -- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 msgr2=0x7f41cc03bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3624224927 2024-04-21T08:58:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.182+0000 7f41e4bbc700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 0x7f41cc03bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.301+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f41e0107bf0 con 0x7f41e0108dd0 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.301+0000 7f41d67fc700 1 -- 172.21.15.66:0/4192258934 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f41d000b5a0 con 0x7f41e0108dd0 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 msgr2=0x7f41cc03bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 0x7f41cc03bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 msgr2=0x7f41e0108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e0108650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41d0011ac0 tx=0x7f41d0011ba0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 shutdown_connections 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f41cc0396c0 0x7f41cc03bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 --2- 172.21.15.66:0/4192258934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0108dd0 0x7f41e0108650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 >> 172.21.15.66:0/4192258934 conn(0x7f41e0102820 msgr2=0x7f41e0103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 shutdown_connections 2024-04-21T08:58:06.882 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:06.303+0000 7f41e7621700 1 -- 172.21.15.66:0/4192258934 wait complete. 2024-04-21T08:58:06.883 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2024-04-21T08:58:06.883 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 5... 2024-04-21T08:58:07.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:06 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4192258934' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:58:07.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:07.093+0000 7f534e156200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:58:07.532 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:07.275+0000 7f534e156200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:58:07.532 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:07.453+0000 7f534e156200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:58:07.848 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:07.753+0000 7f534e156200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:58:07.848 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:07.849+0000 7f534e156200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:58:08.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:08.059+0000 7f534e156200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:58:08.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:08.259+0000 7f534e156200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:58:09.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:08.881+0000 7f534e156200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:58:09.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:09.318+0000 7f534e156200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:58:09.620 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:09.417+0000 7f534e156200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:58:10.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:09.625+0000 7f534e156200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:58:10.517 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:10.169+0000 7f534e156200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:58:10.517 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:10.265+0000 7f534e156200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:58:10.869 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:10.517+0000 7f534e156200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:58:10.869 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:10.655+0000 7f534e156200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:58:10.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:10.752+0000 7f534e156200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:58:11.186 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:10.872+0000 7f534e156200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:58:12.088 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:11.187+0000 7f534e156200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:58:12.088 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:11.812+0000 7f534e156200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:58:12.089 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:12.036+0000 7f534e156200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:58:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: Active manager daemon x restarted 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: Activating manager daemon x 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: osdmap e2: 0 total, 0 up, 0 in 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: mgrmap e6: x(active, starting, since 0.00324193s) 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: Manager daemon x is now available 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:12.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:12.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:12 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:58:13.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:13 smithi066 ceph-mon[27953]: Found migration_current of "None". Setting to last migration. 2024-04-21T08:58:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:13 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:58:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:13 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:13 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:13 smithi066 ceph-mon[27953]: mgrmap e7: x(active, since 1.0052s) 2024-04-21T08:58:13.659 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:58:13.659 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.327+0000 7f388e142700 1 Processor -- start 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.327+0000 7f388e142700 1 -- start start 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.327+0000 7f388e142700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f388810b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.327+0000 7f388e142700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f388807be60 con 0x7f3888108e30 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.327+0000 7f38877fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f388810b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.327+0000 7f38877fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f388810b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46116/0 (socket says 172.21.15.66:46116) 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.327+0000 7f38877fe700 1 -- 172.21.15.66:0/426527433 learned_addr learned my addr 172.21.15.66:0/426527433 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.328+0000 7f38877fe700 1 -- 172.21.15.66:0/426527433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388810b760 con 0x7f3888108e30 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.328+0000 7f38877fe700 1 --2- 172.21.15.66:0/426527433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f388810b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f3878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184c9091341c9923 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:13.660 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.328+0000 7f38867fc700 1 -- 172.21.15.66:0/426527433 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878004160 con 0x7f3888108e30 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.328+0000 7f38867fc700 1 -- 172.21.15.66:0/426527433 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3878004300 con 0x7f3888108e30 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.328+0000 7f38867fc700 1 -- 172.21.15.66:0/426527433 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878011af0 con 0x7f3888108e30 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 -- 172.21.15.66:0/426527433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 msgr2=0x7f388810b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 --2- 172.21.15.66:0/426527433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f388810b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f3878009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 -- 172.21.15.66:0/426527433 shutdown_connections 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 --2- 172.21.15.66:0/426527433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f388810b220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 -- 172.21.15.66:0/426527433 >> 172.21.15.66:0/426527433 conn(0x7f3888102820 msgr2=0x7f3888104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 -- 172.21.15.66:0/426527433 shutdown_connections 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 -- 172.21.15.66:0/426527433 wait complete. 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.329+0000 7f388e142700 1 Processor -- start 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.330+0000 7f388e142700 1 -- start start 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.330+0000 7f388e142700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f38881086b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.330+0000 7f388e142700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f388810c720 con 0x7f3888108e30 2024-04-21T08:58:13.661 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.330+0000 7f38877fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f38881086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.330+0000 7f38877fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f38881086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46118/0 (socket says 172.21.15.66:46118) 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.330+0000 7f38877fe700 1 -- 172.21.15.66:0/1505757086 learned_addr learned my addr 172.21.15.66:0/1505757086 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.330+0000 7f38877fe700 1 -- 172.21.15.66:0/1505757086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3888108bf0 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.331+0000 7f38877fe700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f38881086b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3878011ac0 tx=0x7f3878011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.331+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878003b20 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.331+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f387801a8c0 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.331+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3888106de0 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.331+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38780248d0 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.331+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3888107320 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.332+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f3878022020 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.332+0000 7f3884ff9700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.332+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 --> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f387003c240 con 0x7f3870039670 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.332+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f387804d550 con 0x7f3888108e30 2024-04-21T08:58:13.662 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.332+0000 7f3886ffd700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 msgr2=0x7f387003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3624224927 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.332+0000 7f3886ffd700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.533+0000 7f3886ffd700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 msgr2=0x7f387003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3624224927 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.533+0000 7f3886ffd700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.934+0000 7f3886ffd700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 msgr2=0x7f387003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3624224927 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:07.934+0000 7f3886ffd700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:08.735+0000 7f3886ffd700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 msgr2=0x7f387003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3624224927 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:08.735+0000 7f3886ffd700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:10.336+0000 7f3886ffd700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 msgr2=0x7f387003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3624224927 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:10.336+0000 7f3886ffd700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:12.039+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f387802d3f0 con 0x7f3888108e30 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:12.039+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 msgr2=0x7f387003bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:12.039+0000 7f3884ff9700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.040+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f387804f0e0 con 0x7f3888108e30 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.041+0000 7f3884ff9700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f387003cf60 0x7f387003f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:13.663 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.041+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f387804cfa0 con 0x7f387003cf60 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.041+0000 7f3886ffd700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f387003cf60 0x7f387003f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.042+0000 7f3886ffd700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f387003cf60 0x7f387003f350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f387c003a10 tx=0x7f387c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.042+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f387804cfa0 con 0x7f387003cf60 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3888069b50 con 0x7f387003cf60 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f3884ff9700 1 -- 172.21.15.66:0/1505757086 <== mgr.14120 v2:172.21.15.66:6800/763565357 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3888069b50 con 0x7f387003cf60 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f387003cf60 msgr2=0x7f387003f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f387003cf60 0x7f387003f350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f387c003a10 tx=0x7f387c009290 comp rx=0 tx=0).stop 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 msgr2=0x7f38881086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f38881086b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3878011ac0 tx=0x7f3878011ba0 comp rx=0 tx=0).stop 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 shutdown_connections 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f387003cf60 0x7f387003f350 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3888108e30 0x7f38881086b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 --2- 172.21.15.66:0/1505757086 >> [v2:172.21.15.66:6800/3624224927,v1:172.21.15.66:6801/3624224927] conn(0x7f3870039670 0x7f387003bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 >> 172.21.15.66:0/1505757086 conn(0x7f3888102820 msgr2=0x7f3888103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:13.665 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.049+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 shutdown_connections 2024-04-21T08:58:13.665 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:13.050+0000 7f388e142700 1 -- 172.21.15.66:0/1505757086 wait complete. 2024-04-21T08:58:13.665 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 5 is available 2024-04-21T08:58:13.665 INFO:teuthology.orchestra.run.smithi066.stdout:Setting orchestrator backend to cephadm... 2024-04-21T08:58:14.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:14 smithi066 ceph-mon[27953]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:58:14.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:14 smithi066 ceph-mon[27953]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:58:14.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:14 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:13] ENGINE Bus STARTING 2024-04-21T08:58:14.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:14 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:13] ENGINE Serving on https://172.21.15.66:7150 2024-04-21T08:58:14.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:14 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:14.898 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.090+0000 7f2479551700 1 Processor -- start 2024-04-21T08:58:14.898 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.090+0000 7f2479551700 1 -- start start 2024-04-21T08:58:14.898 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.090+0000 7f2479551700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f2474107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:14.898 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.090+0000 7f2479551700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2474108130 con 0x7f2474107740 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.091+0000 7f2472ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f2474107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.091+0000 7f2472ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f2474107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46190/0 (socket says 172.21.15.66:46190) 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.091+0000 7f2472ffd700 1 -- 172.21.15.66:0/932661958 learned_addr learned my addr 172.21.15.66:0/932661958 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.091+0000 7f2472ffd700 1 -- 172.21.15.66:0/932661958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2474108460 con 0x7f2474107740 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.091+0000 7f2472ffd700 1 --2- 172.21.15.66:0/932661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f2474107b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24640098a0 tx=0x7f2464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06545b0aff32ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2471ffb700 1 -- 172.21.15.66:0/932661958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2464004160 con 0x7f2474107740 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2471ffb700 1 -- 172.21.15.66:0/932661958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2464004300 con 0x7f2474107740 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2471ffb700 1 -- 172.21.15.66:0/932661958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2464011af0 con 0x7f2474107740 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2479551700 1 -- 172.21.15.66:0/932661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 msgr2=0x7f2474107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2479551700 1 --2- 172.21.15.66:0/932661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f2474107b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24640098a0 tx=0x7f2464009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2479551700 1 -- 172.21.15.66:0/932661958 shutdown_connections 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2479551700 1 --2- 172.21.15.66:0/932661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f2474107b60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2479551700 1 -- 172.21.15.66:0/932661958 >> 172.21.15.66:0/932661958 conn(0x7f2474101fa0 msgr2=0x7f24741043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:14.899 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2479551700 1 -- 172.21.15.66:0/932661958 shutdown_connections 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.093+0000 7f2479551700 1 -- 172.21.15.66:0/932661958 wait complete. 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.094+0000 7f2479551700 1 Processor -- start 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.094+0000 7f2479551700 1 -- start start 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.094+0000 7f2479551700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f247419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.094+0000 7f2479551700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f247410b810 con 0x7f2474107740 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.095+0000 7f2472ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f247419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.095+0000 7f2472ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f247419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46204/0 (socket says 172.21.15.66:46204) 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.095+0000 7f2472ffd700 1 -- 172.21.15.66:0/1637305763 learned_addr learned my addr 172.21.15.66:0/1637305763 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.095+0000 7f2472ffd700 1 -- 172.21.15.66:0/1637305763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247419b610 con 0x7f2474107740 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.095+0000 7f2472ffd700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f247419b0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2464011ac0 tx=0x7f2464011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.096+0000 7f245bfff700 1 -- 172.21.15.66:0/1637305763 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2464003b20 con 0x7f2474107740 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.096+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247419b8a0 con 0x7f2474107740 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.096+0000 7f245bfff700 1 -- 172.21.15.66:0/1637305763 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f246401a8c0 con 0x7f2474107740 2024-04-21T08:58:14.900 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.096+0000 7f245bfff700 1 -- 172.21.15.66:0/1637305763 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24640248d0 con 0x7f2474107740 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.096+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247419e590 con 0x7f2474107740 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.097+0000 7f245bfff700 1 -- 172.21.15.66:0/1637305763 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2464022020 con 0x7f2474107740 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.097+0000 7f245bfff700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f245c039670 0x7f245c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.097+0000 7f245bfff700 1 -- 172.21.15.66:0/1637305763 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f246404e810 con 0x7f2474107740 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.097+0000 7f24727fc700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f245c039670 0x7f245c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.098+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24740640a0 con 0x7f2474107740 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.098+0000 7f24727fc700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f245c039670 0x7f245c03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2468006fd0 tx=0x7f2468006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.103+0000 7f245bfff700 1 -- 172.21.15.66:0/1637305763 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f246400b5a0 con 0x7f2474107740 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.329+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f24741031d0 con 0x7f245c039670 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.334+0000 7f245bfff700 1 -- 172.21.15.66:0/1637305763 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24741031d0 con 0x7f245c039670 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f245c039670 msgr2=0x7f245c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f245c039670 0x7f245c03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2468006fd0 tx=0x7f2468006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 msgr2=0x7f247419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:14.901 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f247419b0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2464011ac0 tx=0x7f2464011ba0 comp rx=0 tx=0).stop 2024-04-21T08:58:14.902 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 shutdown_connections 2024-04-21T08:58:14.902 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f245c039670 0x7f245c03bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:14.902 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 --2- 172.21.15.66:0/1637305763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474107740 0x7f247419b0d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:14.902 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 >> 172.21.15.66:0/1637305763 conn(0x7f2474101fa0 msgr2=0x7f2474102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:14.902 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 shutdown_connections 2024-04-21T08:58:14.902 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:14.337+0000 7f2479551700 1 -- 172.21.15.66:0/1637305763 wait complete. 2024-04-21T08:58:15.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:15 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:14] ENGINE Serving on http://172.21.15.66:8765 2024-04-21T08:58:15.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:15 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:14] ENGINE Bus STARTED 2024-04-21T08:58:15.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:15 smithi066 ceph-mon[27953]: mgrmap e8: x(active, since 2s) 2024-04-21T08:58:15.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:15 smithi066 ceph-mon[27953]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:15.293 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:15 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:15.293 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:15 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:16.129 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-21T08:58:16.129 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.327+0000 7ff04cc35700 1 Processor -- start 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.327+0000 7ff04cc35700 1 -- start start 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.327+0000 7ff04cc35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff048106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04cc35700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff048106ff0 con 0x7ff048106600 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff048106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff048106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46140/0 (socket says 172.21.15.66:46140) 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04659c700 1 -- 172.21.15.66:0/1281312999 learned_addr learned my addr 172.21.15.66:0/1281312999 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04659c700 1 -- 172.21.15.66:0/1281312999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff048107170 con 0x7ff048106600 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04659c700 1 --2- 172.21.15.66:0/1281312999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff048106a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff040009e80 tx=0x7ff04000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29305eefa8c5c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04559a700 1 -- 172.21.15.66:0/1281312999 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff040017070 con 0x7ff048106600 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04559a700 1 -- 172.21.15.66:0/1281312999 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff04000b8c0 con 0x7ff048106600 2024-04-21T08:58:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.328+0000 7ff04559a700 1 -- 172.21.15.66:0/1281312999 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff040004490 con 0x7ff048106600 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.329+0000 7ff04cc35700 1 -- 172.21.15.66:0/1281312999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 msgr2=0x7ff048106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.329+0000 7ff04cc35700 1 --2- 172.21.15.66:0/1281312999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff048106a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff040009e80 tx=0x7ff04000b330 comp rx=0 tx=0).stop 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.329+0000 7ff04cc35700 1 -- 172.21.15.66:0/1281312999 shutdown_connections 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.329+0000 7ff04cc35700 1 --2- 172.21.15.66:0/1281312999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff048106a20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.329+0000 7ff04cc35700 1 -- 172.21.15.66:0/1281312999 >> 172.21.15.66:0/1281312999 conn(0x7ff048101fa0 msgr2=0x7ff0481043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.329+0000 7ff04cc35700 1 -- 172.21.15.66:0/1281312999 shutdown_connections 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.329+0000 7ff04cc35700 1 -- 172.21.15.66:0/1281312999 wait complete. 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04cc35700 1 Processor -- start 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04cc35700 1 -- start start 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04cc35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff04819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:16.131 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04cc35700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff04810a4e0 con 0x7ff048106600 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff04819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff04819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46142/0 (socket says 172.21.15.66:46142) 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04659c700 1 -- 172.21.15.66:0/3621675273 learned_addr learned my addr 172.21.15.66:0/3621675273 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.330+0000 7ff04659c700 1 -- 172.21.15.66:0/3621675273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04819e960 con 0x7ff048106600 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.331+0000 7ff04659c700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff04819e420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff040000f80 tx=0x7ff04000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.331+0000 7ff03f7fe700 1 -- 172.21.15.66:0/3621675273 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff040020020 con 0x7ff048106600 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.331+0000 7ff03f7fe700 1 -- 172.21.15.66:0/3621675273 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff040012430 con 0x7ff048106600 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.331+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04819ebf0 con 0x7ff048106600 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.331+0000 7ff03f7fe700 1 -- 172.21.15.66:0/3621675273 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff040017040 con 0x7ff048106600 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.331+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0481a18e0 con 0x7ff048106600 2024-04-21T08:58:16.132 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.332+0000 7ff03f7fe700 1 -- 172.21.15.66:0/3621675273 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff040023020 con 0x7ff048106600 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.332+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff048069b50 con 0x7ff048106600 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.332+0000 7ff03f7fe700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7ff034039670 0x7ff03403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.332+0000 7ff03f7fe700 1 -- 172.21.15.66:0/3621675273 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff04000f050 con 0x7ff048106600 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.332+0000 7ff045d9b700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7ff034039670 0x7ff03403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.333+0000 7ff045d9b700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7ff034039670 0x7ff03403bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff030006fd0 tx=0x7ff030006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.338+0000 7ff03f7fe700 1 -- 172.21.15.66:0/3621675273 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff04002a020 con 0x7ff048106600 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.548+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff048103150 con 0x7ff034039670 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.549+0000 7ff03f7fe700 1 -- 172.21.15.66:0/3621675273 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff048103150 con 0x7ff034039670 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7ff034039670 msgr2=0x7ff03403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7ff034039670 0x7ff03403bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff030006fd0 tx=0x7ff030006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:16.133 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 msgr2=0x7ff04819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff04819e420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff040000f80 tx=0x7ff04000bdc0 comp rx=0 tx=0).stop 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 shutdown_connections 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7ff034039670 0x7ff03403bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 --2- 172.21.15.66:0/3621675273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff048106600 0x7ff04819e420 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 >> 172.21.15.66:0/3621675273 conn(0x7ff048101fa0 msgr2=0x7ff048102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 shutdown_connections 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:15.551+0000 7ff04cc35700 1 -- 172.21.15.66:0/3621675273 wait complete. 2024-04-21T08:58:16.134 INFO:teuthology.orchestra.run.smithi066.stdout:Generating ssh key... 2024-04-21T08:58:16.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-mon[27953]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: Generating public/private rsa key pair. 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: Your identification has been saved in /tmp/tmpgec7sqap/key. 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: Your public key has been saved in /tmp/tmpgec7sqap/key.pub. 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: The key fingerprint is: 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: SHA256:6S7pTZuzqJyTqpd7zVREXQ2OZSfDnNnAzs5Gj7EKKxQ ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: The key's randomart image is: 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: +---[RSA 3072]----+ 2024-04-21T08:58:17.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | ... =OB. | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | . .=*+o | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | . .o. | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | E .. = | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | oS + = | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | o.. * . | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | . * .oo o | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: | o.+.*=oo. | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: |.oo+++oo*o | 2024-04-21T08:58:17.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: +----[SHA256]-----+ 2024-04-21T08:58:17.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.534+0000 7f85cdfad700 1 Processor -- start 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.534+0000 7f85cdfad700 1 -- start start 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.534+0000 7f85cdfad700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c810b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.534+0000 7f85cdfad700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85c807be80 con 0x7f85c8108de0 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.534+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c810b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.534+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c810b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46146/0 (socket says 172.21.15.66:46146) 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.534+0000 7f85c77fe700 1 -- 172.21.15.66:0/2176372685 learned_addr learned my addr 172.21.15.66:0/2176372685 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.535+0000 7f85c77fe700 1 -- 172.21.15.66:0/2176372685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c810b710 con 0x7f85c8108de0 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.535+0000 7f85c77fe700 1 --2- 172.21.15.66:0/2176372685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c810b1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85b80098a0 tx=0x7f85b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ba943c63cb2a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.535+0000 7f85c67fc700 1 -- 172.21.15.66:0/2176372685 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b8004160 con 0x7f85c8108de0 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.535+0000 7f85c67fc700 1 -- 172.21.15.66:0/2176372685 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f85b8004300 con 0x7f85c8108de0 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.535+0000 7f85c67fc700 1 -- 172.21.15.66:0/2176372685 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b8011af0 con 0x7f85c8108de0 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 -- 172.21.15.66:0/2176372685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 msgr2=0x7f85c810b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 --2- 172.21.15.66:0/2176372685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c810b1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85b80098a0 tx=0x7f85b8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:17.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 -- 172.21.15.66:0/2176372685 shutdown_connections 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 --2- 172.21.15.66:0/2176372685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c810b1d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 -- 172.21.15.66:0/2176372685 >> 172.21.15.66:0/2176372685 conn(0x7f85c81027f0 msgr2=0x7f85c8104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 -- 172.21.15.66:0/2176372685 shutdown_connections 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 -- 172.21.15.66:0/2176372685 wait complete. 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 Processor -- start 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.536+0000 7f85cdfad700 1 -- start start 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85cdfad700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85cdfad700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85c810c660 con 0x7f85c8108de0 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c819e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46148/0 (socket says 172.21.15.66:46148) 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c77fe700 1 -- 172.21.15.66:0/1090951450 learned_addr learned my addr 172.21.15.66:0/1090951450 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c77fe700 1 -- 172.21.15.66:0/1090951450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c819e890 con 0x7f85c8108de0 2024-04-21T08:58:17.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c77fe700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c819e350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f85b8003f40 tx=0x7f85b8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c4ff9700 1 -- 172.21.15.66:0/1090951450 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b80038b0 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c4ff9700 1 -- 172.21.15.66:0/1090951450 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f85b8003a50 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c819eb80 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85c4ff9700 1 -- 172.21.15.66:0/1090951450 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b801a5c0 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.537+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c81a1810 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.538+0000 7f85c4ff9700 1 -- 172.21.15.66:0/1090951450 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f85b801a870 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.538+0000 7f85c4ff9700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f85b0039670 0x7f85b003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.538+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c80640a0 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.538+0000 7f85c4ff9700 1 -- 172.21.15.66:0/1090951450 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85b804dc60 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.539+0000 7f85c6ffd700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f85b0039670 0x7f85b003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.539+0000 7f85c6ffd700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f85b0039670 0x7f85b003bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85bc006fd0 tx=0x7f85bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.544+0000 7f85c4ff9700 1 -- 172.21.15.66:0/1090951450 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85b8019c70 con 0x7f85c8108de0 2024-04-21T08:58:17.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.752+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f85c81a19e0 con 0x7f85b0039670 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.850+0000 7f85c4ff9700 1 -- 172.21.15.66:0/1090951450 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85c81a19e0 con 0x7f85b0039670 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f85b0039670 msgr2=0x7f85b003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f85b0039670 0x7f85b003bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85bc006fd0 tx=0x7f85bc006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 msgr2=0x7f85c819e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c819e350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f85b8003f40 tx=0x7f85b8011890 comp rx=0 tx=0).stop 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 shutdown_connections 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f85b0039670 0x7f85b003bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 --2- 172.21.15.66:0/1090951450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85c8108de0 0x7f85c819e350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 >> 172.21.15.66:0/1090951450 conn(0x7f85c81027f0 msgr2=0x7f85c81031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 shutdown_connections 2024-04-21T08:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:16.852+0000 7f85cdfad700 1 -- 172.21.15.66:0/1090951450 wait complete. 2024-04-21T08:58:18.074 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:17 smithi066 ceph-mon[27953]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:18.074 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:17 smithi066 ceph-mon[27953]: Generating ssh key... 2024-04-21T08:58:18.074 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:17 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:18.074 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:17 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHxzkvE3r8M/y9o8yG6CgqqAjHFuQ7Ot1M4wL+tSufdAL1VRwX/aMHcEt8sd7G5WZjjuyl1XuqME8tTrEnLsbKMgx3gncXfqAW9cBYdSiwhnU2FVpPahCo+Ify/YjjEntyiMkOBW7wajCmk4zVj1BDiiEtuoCuzJNtHiqAAGtKrXmXkfCYjQBaCpsoZUultKw4qgXlHfFrDOYPWpqQkiOyng/574Lsxcx6/72BjW8ZKZVm9/SdfYCzUR/841YRY2e9ILR4I0HvQ9p+cpsWb6bVJkuGfcLRoMyvUHR49MH1Kn3jMcFcGO9pjrPBbYXhsiV6g4hEvUAVhf6Y0xmBpB1STcDgbCORvVXCXkfQIwJFfoH754omPRRmNbN0Y+wOQ0urL5irMh+HRBGXwM8WEzdKDHS/JP1xyBey0xiprpjEASjElQ2pJdnjS9Vf5eM/TmTzahyxNi2e4iVLcnEDCUJEJqWGqwnDDsaOHFAvdOtogz5Bz3OF5zCblPHpxmrp6c= ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:58:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.854+0000 7f8988ce7700 1 Processor -- start 2024-04-21T08:58:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.854+0000 7f8988ce7700 1 -- start start 2024-04-21T08:58:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f8988ce7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898410b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f8988ce7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f898407be60 con 0x7f8984108e00 2024-04-21T08:58:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f898259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898410b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f898259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898410b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46160/0 (socket says 172.21.15.66:46160) 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f898259c700 1 -- 172.21.15.66:0/1568106997 learned_addr learned my addr 172.21.15.66:0/1568106997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f898259c700 1 -- 172.21.15.66:0/1568106997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898410b730 con 0x7f8984108e00 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f898259c700 1 --2- 172.21.15.66:0/1568106997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898410b1f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098a0 tx=0x7f896c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c559c01f79450287 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.855+0000 7f898159a700 1 -- 172.21.15.66:0/1568106997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c004160 con 0x7f8984108e00 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f898159a700 1 -- 172.21.15.66:0/1568106997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f896c004300 con 0x7f8984108e00 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f898159a700 1 -- 172.21.15.66:0/1568106997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c011af0 con 0x7f8984108e00 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f8988ce7700 1 -- 172.21.15.66:0/1568106997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 msgr2=0x7f898410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f8988ce7700 1 --2- 172.21.15.66:0/1568106997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898410b1f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098a0 tx=0x7f896c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f8988ce7700 1 -- 172.21.15.66:0/1568106997 shutdown_connections 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f8988ce7700 1 --2- 172.21.15.66:0/1568106997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898410b1f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f8988ce7700 1 -- 172.21.15.66:0/1568106997 >> 172.21.15.66:0/1568106997 conn(0x7f89841027f0 msgr2=0x7f8984104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f8988ce7700 1 -- 172.21.15.66:0/1568106997 shutdown_connections 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.856+0000 7f8988ce7700 1 -- 172.21.15.66:0/1568106997 wait complete. 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.857+0000 7f8988ce7700 1 Processor -- start 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.857+0000 7f8988ce7700 1 -- start start 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.857+0000 7f8988ce7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898419c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.857+0000 7f8988ce7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f898410c6f0 con 0x7f8984108e00 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.857+0000 7f898259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898419c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.857+0000 7f898259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898419c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46162/0 (socket says 172.21.15.66:46162) 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.857+0000 7f898259c700 1 -- 172.21.15.66:0/2037673533 learned_addr learned my addr 172.21.15.66:0/2037673533 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.858+0000 7f898259c700 1 -- 172.21.15.66:0/2037673533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898419c6e0 con 0x7f8984108e00 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.858+0000 7f898259c700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898419c1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f896c003f40 tx=0x7f896c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.858+0000 7f897b7fe700 1 -- 172.21.15.66:0/2037673533 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c0038b0 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.858+0000 7f897b7fe700 1 -- 172.21.15.66:0/2037673533 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f896c003a50 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.858+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898419c970 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.858+0000 7f897b7fe700 1 -- 172.21.15.66:0/2037673533 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c01a5c0 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.858+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898419f660 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.859+0000 7f897b7fe700 1 -- 172.21.15.66:0/2037673533 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f896c01a870 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.859+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89840640a0 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.859+0000 7f897b7fe700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f8970039670 0x7f897003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.859+0000 7f897b7fe700 1 -- 172.21.15.66:0/2037673533 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f896c04dc20 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.859+0000 7f8981d9b700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f8970039670 0x7f897003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.860+0000 7f8981d9b700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f8970039670 0x7f897003bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8974006fd0 tx=0x7f8974006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:17.864+0000 7f897b7fe700 1 -- 172.21.15.66:0/2037673533 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f896c019c70 con 0x7f8984108e00 2024-04-21T08:58:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.074+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8984103980 con 0x7f8970039670 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.075+0000 7f897b7fe700 1 -- 172.21.15.66:0/2037673533 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8984103980 con 0x7f8970039670 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f8970039670 msgr2=0x7f897003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f8970039670 0x7f897003bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8974006fd0 tx=0x7f8974006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 msgr2=0x7f898419c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898419c1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f896c003f40 tx=0x7f896c011890 comp rx=0 tx=0).stop 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 shutdown_connections 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f8970039670 0x7f897003bb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 --2- 172.21.15.66:0/2037673533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8984108e00 0x7f898419c1a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 >> 172.21.15.66:0/2037673533 conn(0x7f89841027f0 msgr2=0x7f8984103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 shutdown_connections 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:18.077+0000 7f8988ce7700 1 -- 172.21.15.66:0/2037673533 wait complete. 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:Adding key to root@localhost authorized_keys... 2024-04-21T08:58:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:Adding host smithi066... 2024-04-21T08:58:19.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:19 smithi066 ceph-mon[27953]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:20.189 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:20 smithi066 ceph-mon[27953]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "addr": "172.21.15.66", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:20.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:20 smithi066 ceph-mon[27953]: Deploying cephadm binary to smithi066 2024-04-21T08:58:24.090 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Added host 'smithi066' with addr '172.21.15.66' 2024-04-21T08:58:24.090 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.068+0000 7fd283694700 1 Processor -- start 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.068+0000 7fd283694700 1 -- start start 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.068+0000 7fd283694700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.068+0000 7fd283694700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c107030 con 0x7fd27c106640 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.068+0000 7fd281430700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.068+0000 7fd281430700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46172/0 (socket says 172.21.15.66:46172) 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.068+0000 7fd281430700 1 -- 172.21.15.66:0/3087056213 learned_addr learned my addr 172.21.15.66:0/3087056213 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.069+0000 7fd281430700 1 -- 172.21.15.66:0/3087056213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c1095d0 con 0x7fd27c106640 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.069+0000 7fd281430700 1 --2- 172.21.15.66:0/3087056213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c106a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2780098a0 tx=0x7fd278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6dd879418a96bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.069+0000 7fd273fff700 1 -- 172.21.15.66:0/3087056213 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd278004160 con 0x7fd27c106640 2024-04-21T08:58:24.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.069+0000 7fd273fff700 1 -- 172.21.15.66:0/3087056213 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd278004300 con 0x7fd27c106640 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.069+0000 7fd273fff700 1 -- 172.21.15.66:0/3087056213 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd278011af0 con 0x7fd27c106640 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 -- 172.21.15.66:0/3087056213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 msgr2=0x7fd27c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 --2- 172.21.15.66:0/3087056213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c106a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2780098a0 tx=0x7fd278009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 -- 172.21.15.66:0/3087056213 shutdown_connections 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 --2- 172.21.15.66:0/3087056213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c106a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 -- 172.21.15.66:0/3087056213 >> 172.21.15.66:0/3087056213 conn(0x7fd27c101fb0 msgr2=0x7fd27c1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 -- 172.21.15.66:0/3087056213 shutdown_connections 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 -- 172.21.15.66:0/3087056213 wait complete. 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 Processor -- start 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.070+0000 7fd283694700 1 -- start start 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd283694700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd283694700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c10c980 con 0x7fd27c106640 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd281430700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd281430700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c19c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46192/0 (socket says 172.21.15.66:46192) 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd281430700 1 -- 172.21.15.66:0/2938528514 learned_addr learned my addr 172.21.15.66:0/2938528514 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd281430700 1 -- 172.21.15.66:0/2938528514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c19c710 con 0x7fd27c106640 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd281430700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c19c1d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd278003f40 tx=0x7fd278011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd2727fc700 1 -- 172.21.15.66:0/2938528514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2780038b0 con 0x7fd27c106640 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd2727fc700 1 -- 172.21.15.66:0/2938528514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd278003a50 con 0x7fd27c106640 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c19c9a0 con 0x7fd27c106640 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd2727fc700 1 -- 172.21.15.66:0/2938528514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27801a5c0 con 0x7fd27c106640 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.071+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c19f690 con 0x7fd27c106640 2024-04-21T08:58:24.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.072+0000 7fd2727fc700 1 -- 172.21.15.66:0/2938528514 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd27801a8c0 con 0x7fd27c106640 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.072+0000 7fd2727fc700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fd268039620 0x7fd26803bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.072+0000 7fd2727fc700 1 -- 172.21.15.66:0/2938528514 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd27804db10 con 0x7fd27c106640 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.072+0000 7fd280c2f700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fd268039620 0x7fd26803bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.072+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c0640a0 con 0x7fd27c106640 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.073+0000 7fd280c2f700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fd268039620 0x7fd26803bae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.078+0000 7fd2727fc700 1 -- 172.21.15.66:0/2938528514 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd278019c70 con 0x7fd27c106640 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:19.291+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi066", "addr": "172.21.15.66", "target": ["mon-mgr", ""]}) -- 0x7fd27c1030a0 con 0x7fd268039620 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.512+0000 7fd2727fc700 1 -- 172.21.15.66:0/2938528514 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd27c1030a0 con 0x7fd268039620 2024-04-21T08:58:24.094 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fd268039620 msgr2=0x7fd26803bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fd268039620 0x7fd26803bae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 msgr2=0x7fd27c19c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c19c1d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd278003f40 tx=0x7fd278011890 comp rx=0 tx=0).stop 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 shutdown_connections 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fd268039620 0x7fd26803bae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 --2- 172.21.15.66:0/2938528514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd27c106640 0x7fd27c19c1d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 >> 172.21.15.66:0/2938528514 conn(0x7fd27c101fb0 msgr2=0x7fd27c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 shutdown_connections 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:23.515+0000 7fd283694700 1 -- 172.21.15.66:0/2938528514 wait complete. 2024-04-21T08:58:24.095 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mon service... 2024-04-21T08:58:24.743 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:24 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:24.743 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:24 smithi066 ceph-mon[27953]: Added host smithi066 2024-04-21T08:58:24.743 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:24 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.743+0000 7f0b8c450700 1 Processor -- start 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.743+0000 7f0b8c450700 1 -- start start 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.744+0000 7f0b8c450700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b84107060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.744+0000 7f0b8c450700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8407be40 con 0x7f0b84106c40 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.744+0000 7f0b8a1ec700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b84107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.744+0000 7f0b8a1ec700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b84107060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44224/0 (socket says 172.21.15.66:44224) 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.744+0000 7f0b8a1ec700 1 -- 172.21.15.66:0/3362721686 learned_addr learned my addr 172.21.15.66:0/3362721686 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.744+0000 7f0b8a1ec700 1 -- 172.21.15.66:0/3362721686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b841075a0 con 0x7f0b84106c40 2024-04-21T08:58:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b8a1ec700 1 --2- 172.21.15.66:0/3362721686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b84107060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b800098a0 tx=0x7f0b80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec43fa3314b1eea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b891ea700 1 -- 172.21.15.66:0/3362721686 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b80004160 con 0x7f0b84106c40 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b891ea700 1 -- 172.21.15.66:0/3362721686 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0b80004300 con 0x7f0b84106c40 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b891ea700 1 -- 172.21.15.66:0/3362721686 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b80011af0 con 0x7f0b84106c40 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b8c450700 1 -- 172.21.15.66:0/3362721686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 msgr2=0x7f0b84107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b8c450700 1 --2- 172.21.15.66:0/3362721686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b84107060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b800098a0 tx=0x7f0b80009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b8c450700 1 -- 172.21.15.66:0/3362721686 shutdown_connections 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b8c450700 1 --2- 172.21.15.66:0/3362721686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b84107060 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.745+0000 7f0b8c450700 1 -- 172.21.15.66:0/3362721686 >> 172.21.15.66:0/3362721686 conn(0x7f0b84102860 msgr2=0x7f0b84104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.746+0000 7f0b8c450700 1 -- 172.21.15.66:0/3362721686 shutdown_connections 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.746+0000 7f0b8c450700 1 -- 172.21.15.66:0/3362721686 wait complete. 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.746+0000 7f0b8c450700 1 Processor -- start 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.746+0000 7f0b8c450700 1 -- start start 2024-04-21T08:58:25.569 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.746+0000 7f0b8c450700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b8419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.746+0000 7f0b8c450700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841085d0 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b8a1ec700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b8419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b8a1ec700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b8419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44228/0 (socket says 172.21.15.66:44228) 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b8a1ec700 1 -- 172.21.15.66:0/1959729591 learned_addr learned my addr 172.21.15.66:0/1959729591 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b8a1ec700 1 -- 172.21.15.66:0/1959729591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8419e920 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b8a1ec700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b8419e3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b80011ac0 tx=0x7f0b80011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b7b7fe700 1 -- 172.21.15.66:0/1959729591 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b80003b20 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b7b7fe700 1 -- 172.21.15.66:0/1959729591 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0b8001a8c0 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8419ebb0 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b7b7fe700 1 -- 172.21.15.66:0/1959729591 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b800248d0 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.747+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b841a18a0 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.748+0000 7f0b7b7fe700 1 -- 172.21.15.66:0/1959729591 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0b80022020 con 0x7f0b84106c40 2024-04-21T08:58:25.570 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.748+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b840640a0 con 0x7f0b84106c40 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.748+0000 7f0b7b7fe700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0b700396c0 0x7f0b7003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.748+0000 7f0b7b7fe700 1 -- 172.21.15.66:0/1959729591 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b8004d9c0 con 0x7f0b84106c40 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.749+0000 7f0b899eb700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0b700396c0 0x7f0b7003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.749+0000 7f0b899eb700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0b700396c0 0x7f0b7003bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b74006fd0 tx=0x7f0b74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.754+0000 7f0b7b7fe700 1 -- 172.21.15.66:0/1959729591 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b8001ade0 con 0x7f0b84106c40 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.993+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0b84076600 con 0x7f0b700396c0 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:24.998+0000 7f0b7b7fe700 1 -- 172.21.15.66:0/1959729591 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0b84076600 con 0x7f0b700396c0 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0b700396c0 msgr2=0x7f0b7003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0b700396c0 0x7f0b7003bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b74006fd0 tx=0x7f0b74006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 msgr2=0x7f0b8419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b8419e3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b80011ac0 tx=0x7f0b80011ba0 comp rx=0 tx=0).stop 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 shutdown_connections 2024-04-21T08:58:25.571 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0b700396c0 0x7f0b7003bb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:25.572 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 --2- 172.21.15.66:0/1959729591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b84106c40 0x7f0b8419e3e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:25.572 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 >> 172.21.15.66:0/1959729591 conn(0x7f0b84102860 msgr2=0x7f0b84194ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:25.572 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 shutdown_connections 2024-04-21T08:58:25.572 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.000+0000 7f0b8c450700 1 -- 172.21.15.66:0/1959729591 wait complete. 2024-04-21T08:58:25.572 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mgr service... 2024-04-21T08:58:26.096 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:25 smithi066 ceph-mon[27953]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:26.097 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:25 smithi066 ceph-mon[27953]: Saving service mon spec with placement count:5 2024-04-21T08:58:26.097 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:25 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.951+0000 7f972bc66700 1 Processor -- start 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.951+0000 7f972bc66700 1 -- start start 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.951+0000 7f972bc66700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972410b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.951+0000 7f972bc66700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f972407be40 con 0x7f9724108e00 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9729a02700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972410b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9729a02700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972410b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44242/0 (socket says 172.21.15.66:44242) 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9729a02700 1 -- 172.21.15.66:0/1596405027 learned_addr learned my addr 172.21.15.66:0/1596405027 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9729a02700 1 -- 172.21.15.66:0/1596405027 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972410b730 con 0x7f9724108e00 2024-04-21T08:58:26.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9729a02700 1 --2- 172.21.15.66:0/1596405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972410b1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97200098a0 tx=0x7f9720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea6ad3d86d6ea25 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9728a00700 1 -- 172.21.15.66:0/1596405027 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9720004160 con 0x7f9724108e00 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9728a00700 1 -- 172.21.15.66:0/1596405027 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9720004300 con 0x7f9724108e00 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.952+0000 7f9728a00700 1 -- 172.21.15.66:0/1596405027 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9720011af0 con 0x7f9724108e00 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.953+0000 7f972bc66700 1 -- 172.21.15.66:0/1596405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 msgr2=0x7f972410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.953+0000 7f972bc66700 1 --2- 172.21.15.66:0/1596405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972410b1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97200098a0 tx=0x7f9720009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.953+0000 7f972bc66700 1 -- 172.21.15.66:0/1596405027 shutdown_connections 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.953+0000 7f972bc66700 1 --2- 172.21.15.66:0/1596405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972410b1f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.953+0000 7f972bc66700 1 -- 172.21.15.66:0/1596405027 >> 172.21.15.66:0/1596405027 conn(0x7f97241027d0 msgr2=0x7f9724104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.953+0000 7f972bc66700 1 -- 172.21.15.66:0/1596405027 shutdown_connections 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.953+0000 7f972bc66700 1 -- 172.21.15.66:0/1596405027 wait complete. 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f972bc66700 1 Processor -- start 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f972bc66700 1 -- start start 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f972bc66700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:26.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f972bc66700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f972410c6a0 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f9729a02700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f9729a02700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44244/0 (socket says 172.21.15.66:44244) 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f9729a02700 1 -- 172.21.15.66:0/529298438 learned_addr learned my addr 172.21.15.66:0/529298438 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f9729a02700 1 -- 172.21.15.66:0/529298438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972419c720 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.954+0000 7f9729a02700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972419c1e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9720003f40 tx=0x7f9720011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.955+0000 7f971affd700 1 -- 172.21.15.66:0/529298438 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97200038b0 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.955+0000 7f971affd700 1 -- 172.21.15.66:0/529298438 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9720003a50 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.955+0000 7f971affd700 1 -- 172.21.15.66:0/529298438 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f972001a5c0 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.955+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972419c9b0 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.955+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972419f6a0 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.956+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97240640a0 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.956+0000 7f971affd700 1 -- 172.21.15.66:0/529298438 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9720024430 con 0x7f9724108e00 2024-04-21T08:58:26.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.956+0000 7f971affd700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9710039670 0x7f971003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.956+0000 7f971affd700 1 -- 172.21.15.66:0/529298438 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f972004daf0 con 0x7f9724108e00 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.956+0000 7f9729201700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9710039670 0x7f971003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.957+0000 7f9729201700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9710039670 0x7f971003bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9714006fd0 tx=0x7f9714006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:25.965+0000 7f971affd700 1 -- 172.21.15.66:0/529298438 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f972002cc00 con 0x7f9724108e00 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.192+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f97241039f0 con 0x7f9710039670 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.200+0000 7f971affd700 1 -- 172.21.15.66:0/529298438 <== mgr.14120 v2:172.21.15.66:6800/763565357 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f97241039f0 con 0x7f9710039670 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9710039670 msgr2=0x7f971003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9710039670 0x7f971003bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9714006fd0 tx=0x7f9714006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 msgr2=0x7f972419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972419c1e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9720003f40 tx=0x7f9720011890 comp rx=0 tx=0).stop 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 shutdown_connections 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9710039670 0x7f971003bb30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 --2- 172.21.15.66:0/529298438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9724108e00 0x7f972419c1e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:26.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 >> 172.21.15.66:0/529298438 conn(0x7f97241027d0 msgr2=0x7f97241032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:26.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 shutdown_connections 2024-04-21T08:58:26.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:26.203+0000 7f972bc66700 1 -- 172.21.15.66:0/529298438 wait complete. 2024-04-21T08:58:27.287 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:27 smithi066 ceph-mon[27953]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:27.287 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:27 smithi066 ceph-mon[27953]: Saving service mgr spec with placement count:2 2024-04-21T08:58:27.287 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:27 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:28.322 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e18461700 1 Processor -- start 2024-04-21T08:58:28.322 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e18461700 1 -- start start 2024-04-21T08:58:28.322 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e18461700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1010b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:28.323 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e18461700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1007be40 con 0x7f2e10108e50 2024-04-21T08:58:28.323 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e161fd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1010b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:28.323 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e161fd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1010b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44252/0 (socket says 172.21.15.66:44252) 2024-04-21T08:58:28.323 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e161fd700 1 -- 172.21.15.66:0/3936212894 learned_addr learned my addr 172.21.15.66:0/3936212894 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:28.323 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.307+0000 7f2e161fd700 1 -- 172.21.15.66:0/3936212894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1010b780 con 0x7f2e10108e50 2024-04-21T08:58:28.323 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.308+0000 7f2e161fd700 1 --2- 172.21.15.66:0/3936212894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1010b240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0098a0 tx=0x7f2e0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e843eebde849ad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:28.324 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.308+0000 7f2e151fb700 1 -- 172.21.15.66:0/3936212894 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e0c004160 con 0x7f2e10108e50 2024-04-21T08:58:28.324 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.308+0000 7f2e151fb700 1 -- 172.21.15.66:0/3936212894 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e0c004300 con 0x7f2e10108e50 2024-04-21T08:58:28.324 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.308+0000 7f2e151fb700 1 -- 172.21.15.66:0/3936212894 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e0c011af0 con 0x7f2e10108e50 2024-04-21T08:58:28.324 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.308+0000 7f2e18461700 1 -- 172.21.15.66:0/3936212894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 msgr2=0x7f2e1010b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:28.324 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.308+0000 7f2e18461700 1 --2- 172.21.15.66:0/3936212894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1010b240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0098a0 tx=0x7f2e0c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:28.324 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 -- 172.21.15.66:0/3936212894 shutdown_connections 2024-04-21T08:58:28.324 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 --2- 172.21.15.66:0/3936212894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1010b240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:28.325 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 -- 172.21.15.66:0/3936212894 >> 172.21.15.66:0/3936212894 conn(0x7f2e10102860 msgr2=0x7f2e10104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:28.325 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 -- 172.21.15.66:0/3936212894 shutdown_connections 2024-04-21T08:58:28.325 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 -- 172.21.15.66:0/3936212894 wait complete. 2024-04-21T08:58:28.325 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 Processor -- start 2024-04-21T08:58:28.325 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 -- start start 2024-04-21T08:58:28.325 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:28.326 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.309+0000 7f2e18461700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1010c6f0 con 0x7f2e10108e50 2024-04-21T08:58:28.326 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e161fd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:28.326 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e161fd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44258/0 (socket says 172.21.15.66:44258) 2024-04-21T08:58:28.326 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e161fd700 1 -- 172.21.15.66:0/4244915657 learned_addr learned my addr 172.21.15.66:0/4244915657 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:28.326 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e161fd700 1 -- 172.21.15.66:0/4244915657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1019c6f0 con 0x7f2e10108e50 2024-04-21T08:58:28.327 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e161fd700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1019c1b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011ac0 tx=0x7f2e0c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:28.327 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e077fe700 1 -- 172.21.15.66:0/4244915657 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e0c003b20 con 0x7f2e10108e50 2024-04-21T08:58:28.327 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e077fe700 1 -- 172.21.15.66:0/4244915657 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e0c01a8c0 con 0x7f2e10108e50 2024-04-21T08:58:28.327 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e077fe700 1 -- 172.21.15.66:0/4244915657 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e0c0248d0 con 0x7f2e10108e50 2024-04-21T08:58:28.327 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1019c980 con 0x7f2e10108e50 2024-04-21T08:58:28.327 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.310+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1019f670 con 0x7f2e10108e50 2024-04-21T08:58:28.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.311+0000 7f2e077fe700 1 -- 172.21.15.66:0/4244915657 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2e0c022020 con 0x7f2e10108e50 2024-04-21T08:58:28.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.312+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e100640a0 con 0x7f2e10108e50 2024-04-21T08:58:28.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.312+0000 7f2e077fe700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f2dfc0396c0 0x7f2dfc03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:28.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.312+0000 7f2e077fe700 1 -- 172.21.15.66:0/4244915657 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2e0c04d800 con 0x7f2e10108e50 2024-04-21T08:58:28.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.312+0000 7f2e159fc700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f2dfc0396c0 0x7f2dfc03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:28.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.312+0000 7f2e159fc700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f2dfc0396c0 0x7f2dfc03bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e00006fd0 tx=0x7f2e00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:28.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.317+0000 7f2e077fe700 1 -- 172.21.15.66:0/4244915657 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e0c00f2e0 con 0x7f2e10108e50 2024-04-21T08:58:28.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.530+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2e1019fcc0 con 0x7f2e10108e50 2024-04-21T08:58:28.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.535+0000 7f2e077fe700 1 -- 172.21.15.66:0/4244915657 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f2e0c01aa70 con 0x7f2e10108e50 2024-04-21T08:58:28.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f2dfc0396c0 msgr2=0x7f2dfc03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:28.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f2dfc0396c0 0x7f2dfc03bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e00006fd0 tx=0x7f2e00006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:28.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 msgr2=0x7f2e1019c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:28.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1019c1b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011ac0 tx=0x7f2e0c011ba0 comp rx=0 tx=0).stop 2024-04-21T08:58:28.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 shutdown_connections 2024-04-21T08:58:28.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f2dfc0396c0 0x7f2dfc03bb80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:28.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 --2- 172.21.15.66:0/4244915657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e10108e50 0x7f2e1019c1b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:28.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 >> 172.21.15.66:0/4244915657 conn(0x7f2e10102860 msgr2=0x7f2e10103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:28.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 shutdown_connections 2024-04-21T08:58:28.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:27.537+0000 7f2e18461700 1 -- 172.21.15.66:0/4244915657 wait complete. 2024-04-21T08:58:28.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:28 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4244915657' entity='client.admin' 2024-04-21T08:58:28.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:28 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:28.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:28 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:30.218 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:29 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3075220636' entity='client.admin' 2024-04-21T08:58:30.219 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:29 smithi066 ceph-mon[27953]: from='mgr.14120 172.21.15.66:0/3359354210' entity='mgr.x' 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.736+0000 7f041b323700 1 Processor -- start 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.736+0000 7f041b323700 1 -- start start 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.736+0000 7f041b323700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f0414107070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.736+0000 7f041b323700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f041407be00 con 0x7f0414106c50 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f04190bf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f0414107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f04190bf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f0414107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44272/0 (socket says 172.21.15.66:44272) 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f04190bf700 1 -- 172.21.15.66:0/485620918 learned_addr learned my addr 172.21.15.66:0/485620918 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:30.226 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f04190bf700 1 -- 172.21.15.66:0/485620918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04141075b0 con 0x7f0414106c50 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f04190bf700 1 --2- 172.21.15.66:0/485620918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f0414107070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04040098a0 tx=0x7f0404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1897ee80db219f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f040bfff700 1 -- 172.21.15.66:0/485620918 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0404004160 con 0x7f0414106c50 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f040bfff700 1 -- 172.21.15.66:0/485620918 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0404004300 con 0x7f0414106c50 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.737+0000 7f040bfff700 1 -- 172.21.15.66:0/485620918 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0404011a20 con 0x7f0414106c50 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.738+0000 7f041b323700 1 -- 172.21.15.66:0/485620918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 msgr2=0x7f0414107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.738+0000 7f041b323700 1 --2- 172.21.15.66:0/485620918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f0414107070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04040098a0 tx=0x7f0404009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.738+0000 7f041b323700 1 -- 172.21.15.66:0/485620918 shutdown_connections 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.738+0000 7f041b323700 1 --2- 172.21.15.66:0/485620918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f0414107070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.738+0000 7f041b323700 1 -- 172.21.15.66:0/485620918 >> 172.21.15.66:0/485620918 conn(0x7f0414102890 msgr2=0x7f0414104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.738+0000 7f041b323700 1 -- 172.21.15.66:0/485620918 shutdown_connections 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.738+0000 7f041b323700 1 -- 172.21.15.66:0/485620918 wait complete. 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.739+0000 7f041b323700 1 Processor -- start 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.739+0000 7f041b323700 1 -- start start 2024-04-21T08:58:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.739+0000 7f041b323700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f041419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.739+0000 7f041b323700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0414108620 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.739+0000 7f04190bf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f041419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.739+0000 7f04190bf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f041419e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44288/0 (socket says 172.21.15.66:44288) 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.739+0000 7f04190bf700 1 -- 172.21.15.66:0/3075220636 learned_addr learned my addr 172.21.15.66:0/3075220636 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.740+0000 7f04190bf700 1 -- 172.21.15.66:0/3075220636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041419e900 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.740+0000 7f04190bf700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f041419e3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f04040116e0 tx=0x7f0404011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.740+0000 7f040a7fc700 1 -- 172.21.15.66:0/3075220636 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0404003a60 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.740+0000 7f040a7fc700 1 -- 172.21.15.66:0/3075220636 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040401b8f0 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.740+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041419eb90 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.740+0000 7f040a7fc700 1 -- 172.21.15.66:0/3075220636 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0404019860 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.740+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04141a1880 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.741+0000 7f040a7fc700 1 -- 172.21.15.66:0/3075220636 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0404019b80 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.741+0000 7f040a7fc700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0400039620 0x7f040003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.741+0000 7f040a7fc700 1 -- 172.21.15.66:0/3075220636 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f040404e150 con 0x7f0414106c50 2024-04-21T08:58:30.228 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.741+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04140640a0 con 0x7f0414106c50 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.741+0000 7f04188be700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0400039620 0x7f040003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.742+0000 7f04188be700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0400039620 0x7f040003bae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0410006fd0 tx=0x7f0410006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.747+0000 7f040a7fc700 1 -- 172.21.15.66:0/3075220636 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f040401f030 con 0x7f0414106c50 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.971+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f041402e1b0 con 0x7f0414106c50 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.974+0000 7f040a7fc700 1 -- 172.21.15.66:0/3075220636 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f040402b350 con 0x7f0414106c50 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.977+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0400039620 msgr2=0x7f040003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.977+0000 7f041b323700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0400039620 0x7f040003bae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0410006fd0 tx=0x7f0410006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.977+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 msgr2=0x7f041419e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.977+0000 7f041b323700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f041419e3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f04040116e0 tx=0x7f0404011aa0 comp rx=0 tx=0).stop 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.978+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 shutdown_connections 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.978+0000 7f041b323700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f0400039620 0x7f040003bae0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.978+0000 7f041b323700 1 --2- 172.21.15.66:0/3075220636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0414106c50 0x7f041419e3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.978+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 >> 172.21.15.66:0/3075220636 conn(0x7f0414102890 msgr2=0x7f0414192660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.978+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 shutdown_connections 2024-04-21T08:58:30.230 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:28.978+0000 7f041b323700 1 -- 172.21.15.66:0/3075220636 wait complete. 2024-04-21T08:58:30.230 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling the dashboard module... 2024-04-21T08:58:31.364 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1848218320' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-21T08:58:31.365 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:31.365 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:31.365 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:31.219+0000 7f43ba47b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:58:31.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:31.411+0000 7f43ba47b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:58:31.620 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:31.518+0000 7f43ba47b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.696+0000 7f9941371700 1 Processor -- start 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.696+0000 7f9941371700 1 -- start start 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f9941371700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f9941371700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f993c108d30 con 0x7f993c108340 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f993affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f993affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44298/0 (socket says 172.21.15.66:44298) 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f993affd700 1 -- 172.21.15.66:0/675208802 learned_addr learned my addr 172.21.15.66:0/675208802 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f993affd700 1 -- 172.21.15.66:0/675208802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c109540 con 0x7f993c108340 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f993affd700 1 --2- 172.21.15.66:0/675208802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c108760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098a0 tx=0x7f992c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0486273df7ce05 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.697+0000 7f9939ffb700 1 -- 172.21.15.66:0/675208802 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f992c004160 con 0x7f993c108340 2024-04-21T08:58:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9939ffb700 1 -- 172.21.15.66:0/675208802 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c004300 con 0x7f993c108340 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9939ffb700 1 -- 172.21.15.66:0/675208802 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f992c011a20 con 0x7f993c108340 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9941371700 1 -- 172.21.15.66:0/675208802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 msgr2=0x7f993c108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9941371700 1 --2- 172.21.15.66:0/675208802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c108760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098a0 tx=0x7f992c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9941371700 1 -- 172.21.15.66:0/675208802 shutdown_connections 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9941371700 1 --2- 172.21.15.66:0/675208802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c108760 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9941371700 1 -- 172.21.15.66:0/675208802 >> 172.21.15.66:0/675208802 conn(0x7f993c1038c0 msgr2=0x7f993c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9941371700 1 -- 172.21.15.66:0/675208802 shutdown_connections 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.698+0000 7f9941371700 1 -- 172.21.15.66:0/675208802 wait complete. 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.699+0000 7f9941371700 1 Processor -- start 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.699+0000 7f9941371700 1 -- start start 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.699+0000 7f9941371700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.699+0000 7f9941371700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f993c10c8f0 con 0x7f993c108340 2024-04-21T08:58:31.669 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f993affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f993affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44314/0 (socket says 172.21.15.66:44314) 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f993affd700 1 -- 172.21.15.66:0/1848218320 learned_addr learned my addr 172.21.15.66:0/1848218320 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f993affd700 1 -- 172.21.15.66:0/1848218320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c07e0a0 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f993affd700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c07db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f992c009520 tx=0x7f992c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f992c0039e0 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c07c290 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.700+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c01b8f0 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.701+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f992c019770 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.701+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c07c7d0 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.701+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f992c019a90 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.701+0000 7f9923fff700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9924039670 0x7f992403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.702+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f992c01baa0 con 0x7f993c108340 2024-04-21T08:58:31.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.702+0000 7f993a7fc700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9924039670 0x7f992403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.702+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c0640a0 con 0x7f993c108340 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.702+0000 7f993a7fc700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9924039670 0x7f992403bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9930006fd0 tx=0x7f9930006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.712+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f992c01f030 con 0x7f993c108340 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:30.981+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f993c07ce90 con 0x7f993c108340 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.033+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f992c021e50 con 0x7f993c108340 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.034+0000 7f9923fff700 1 -- 172.21.15.66:0/1848218320 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f992c01ed20 con 0x7f993c108340 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9924039670 msgr2=0x7f992403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9924039670 0x7f992403bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9930006fd0 tx=0x7f9930006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 msgr2=0x7f993c07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c07db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f992c009520 tx=0x7f992c011aa0 comp rx=0 tx=0).stop 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 shutdown_connections 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f9924039670 0x7f992403bb30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 --2- 172.21.15.66:0/1848218320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f993c108340 0x7f993c07db60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:31.671 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 >> 172.21.15.66:0/1848218320 conn(0x7f993c1038c0 msgr2=0x7f993c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:31.672 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 shutdown_connections 2024-04-21T08:58:31.672 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:31.036+0000 7f9941371700 1 -- 172.21.15.66:0/1848218320 wait complete. 2024-04-21T08:58:31.936 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:31.845+0000 7f43ba47b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:58:31.937 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:31.937+0000 7f43ba47b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:58:32.295 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:32 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1848218320' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-21T08:58:32.295 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:32 smithi066 ceph-mon[27953]: mgrmap e9: x(active, since 18s) 2024-04-21T08:58:32.956 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:58:32.956 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-21T08:58:32.956 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.090+0000 7fb6f1d35700 1 Processor -- start 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.090+0000 7fb6f1d35700 1 -- start start 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.090+0000 7fb6f1d35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.090+0000 7fb6f1d35700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec107c20 con 0x7fb6ec107230 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec107650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44336/0 (socket says 172.21.15.66:44336) 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6eb7fe700 1 -- 172.21.15.66:0/4000010856 learned_addr learned my addr 172.21.15.66:0/4000010856 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6eb7fe700 1 -- 172.21.15.66:0/4000010856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec108430 con 0x7fb6ec107230 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6eb7fe700 1 --2- 172.21.15.66:0/4000010856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec107650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40098a0 tx=0x7fb6d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456bc2b235cab88 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6ea7fc700 1 -- 172.21.15.66:0/4000010856 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d4004160 con 0x7fb6ec107230 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6ea7fc700 1 -- 172.21.15.66:0/4000010856 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6d4004300 con 0x7fb6ec107230 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.091+0000 7fb6ea7fc700 1 -- 172.21.15.66:0/4000010856 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d4011a20 con 0x7fb6ec107230 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.092+0000 7fb6f1d35700 1 -- 172.21.15.66:0/4000010856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 msgr2=0x7fb6ec107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.092+0000 7fb6f1d35700 1 --2- 172.21.15.66:0/4000010856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec107650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40098a0 tx=0x7fb6d4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.092+0000 7fb6f1d35700 1 -- 172.21.15.66:0/4000010856 shutdown_connections 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.092+0000 7fb6f1d35700 1 --2- 172.21.15.66:0/4000010856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec107650 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.092+0000 7fb6f1d35700 1 -- 172.21.15.66:0/4000010856 >> 172.21.15.66:0/4000010856 conn(0x7fb6ec1027d0 msgr2=0x7fb6ec104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.092+0000 7fb6f1d35700 1 -- 172.21.15.66:0/4000010856 shutdown_connections 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.092+0000 7fb6f1d35700 1 -- 172.21.15.66:0/4000010856 wait complete. 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.093+0000 7fb6f1d35700 1 Processor -- start 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.093+0000 7fb6f1d35700 1 -- start start 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.093+0000 7fb6f1d35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec1923a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:32.958 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.093+0000 7fb6f1d35700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10b7e0 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.093+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec1923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.093+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec1923a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44338/0 (socket says 172.21.15.66:44338) 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.093+0000 7fb6eb7fe700 1 -- 172.21.15.66:0/1259860119 learned_addr learned my addr 172.21.15.66:0/1259860119 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.094+0000 7fb6eb7fe700 1 -- 172.21.15.66:0/1259860119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec1928e0 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.094+0000 7fb6eb7fe700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec1923a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40116e0 tx=0x7fb6d4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.094+0000 7fb6e8ff9700 1 -- 172.21.15.66:0/1259860119 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d40039e0 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.094+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec190ad0 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.094+0000 7fb6e8ff9700 1 -- 172.21.15.66:0/1259860119 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6d401b8f0 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.094+0000 7fb6e8ff9700 1 -- 172.21.15.66:0/1259860119 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d4023760 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.094+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec191010 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.095+0000 7fb6e8ff9700 1 -- 172.21.15.66:0/1259860119 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fb6d40239d0 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.095+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec0640a0 con 0x7fb6ec107230 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.095+0000 7fb6e8ff9700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 0x7fb6d803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:32.959 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.095+0000 7fb6e8ff9700 1 -- 172.21.15.66:0/1259860119 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb6d404df70 con 0x7fb6ec107230 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.095+0000 7fb6eaffd700 1 -- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 msgr2=0x7fb6d803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/763565357 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.095+0000 7fb6eaffd700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 0x7fb6d803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.100+0000 7fb6e8ff9700 1 -- 172.21.15.66:0/1259860119 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6d401f030 con 0x7fb6ec107230 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.296+0000 7fb6eaffd700 1 -- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 msgr2=0x7fb6d803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/763565357 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.296+0000 7fb6eaffd700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 0x7fb6d803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.414+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb6ec196130 con 0x7fb6ec107230 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.414+0000 7fb6e8ff9700 1 -- 172.21.15.66:0/1259860119 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb6d4023dc0 con 0x7fb6ec107230 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 msgr2=0x7fb6d803bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 0x7fb6d803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 msgr2=0x7fb6ec1923a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec1923a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40116e0 tx=0x7fb6d4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 shutdown_connections 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7fb6d8039670 0x7fb6d803bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 --2- 172.21.15.66:0/1259860119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6ec107230 0x7fb6ec1923a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:32.961 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 >> 172.21.15.66:0/1259860119 conn(0x7fb6ec1027d0 msgr2=0x7fb6ec1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:32.961 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 shutdown_connections 2024-04-21T08:58:32.961 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:32.417+0000 7fb6f1d35700 1 -- 172.21.15.66:0/1259860119 wait complete. 2024-04-21T08:58:32.961 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2024-04-21T08:58:32.961 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 9... 2024-04-21T08:58:33.300 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:33 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1259860119' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:58:33.301 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:33.192+0000 7f43ba47b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:58:33.551 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:33.373+0000 7f43ba47b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:58:33.552 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:33.552+0000 7f43ba47b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:58:34.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:33.854+0000 7f43ba47b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:58:34.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:33.950+0000 7f43ba47b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:58:34.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:34.160+0000 7f43ba47b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:58:34.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:34.361+0000 7f43ba47b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:58:35.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:34.988+0000 7f43ba47b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:58:35.738 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:35.429+0000 7f43ba47b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:58:35.738 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:35.529+0000 7f43ba47b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:58:36.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:35.742+0000 7f43ba47b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:58:36.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:36.290+0000 7f43ba47b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:58:36.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:36.391+0000 7f43ba47b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:58:37.008 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:36.647+0000 7f43ba47b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:58:37.008 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:36.786+0000 7f43ba47b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:58:37.008 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:36.886+0000 7f43ba47b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:58:37.008 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:37.007+0000 7f43ba47b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:58:37.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:37.349+0000 7f43ba47b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:58:38.263 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:37.981+0000 7f43ba47b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:58:38.263 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T08:58:38.208+0000 7f43ba47b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:58:38.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: Active manager daemon x restarted 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: Activating manager daemon x 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: osdmap e3: 0 total, 0 up, 0 in 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: mgrmap e10: x(active, starting, since 0.00371219s) 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: Manager daemon x is now available 2024-04-21T08:58:38.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:39.423 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:58:39.423 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:58:39.423 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:39 smithi066 ceph-mon[27953]: mgrmap e11: x(active, since 1.00609s) 2024-04-21T08:58:40.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.405+0000 7f7c71997700 1 Processor -- start 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.405+0000 7f7c71997700 1 -- start start 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.405+0000 7f7c71997700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c10b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.405+0000 7f7c71997700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c07be60 con 0x7f7c6c108de0 2024-04-21T08:58:40.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.405+0000 7f7c6affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c10b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.405+0000 7f7c6affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c10b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44344/0 (socket says 172.21.15.66:44344) 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.405+0000 7f7c6affd700 1 -- 172.21.15.66:0/2818620950 learned_addr learned my addr 172.21.15.66:0/2818620950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.406+0000 7f7c6affd700 1 -- 172.21.15.66:0/2818620950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c10b710 con 0x7f7c6c108de0 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.406+0000 7f7c6affd700 1 --2- 172.21.15.66:0/2818620950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c10b1d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c540098a0 tx=0x7f7c54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffd3d0655716da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.406+0000 7f7c69ffb700 1 -- 172.21.15.66:0/2818620950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c54004160 con 0x7f7c6c108de0 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.406+0000 7f7c69ffb700 1 -- 172.21.15.66:0/2818620950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c54004300 con 0x7f7c6c108de0 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.406+0000 7f7c69ffb700 1 -- 172.21.15.66:0/2818620950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c54011a20 con 0x7f7c6c108de0 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 -- 172.21.15.66:0/2818620950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 msgr2=0x7f7c6c10b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 --2- 172.21.15.66:0/2818620950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c10b1d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c540098a0 tx=0x7f7c54009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 -- 172.21.15.66:0/2818620950 shutdown_connections 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 --2- 172.21.15.66:0/2818620950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c10b1d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 -- 172.21.15.66:0/2818620950 >> 172.21.15.66:0/2818620950 conn(0x7f7c6c1027d0 msgr2=0x7f7c6c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:40.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 -- 172.21.15.66:0/2818620950 shutdown_connections 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 -- 172.21.15.66:0/2818620950 wait complete. 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.407+0000 7f7c71997700 1 Processor -- start 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c71997700 1 -- start start 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c71997700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c108660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c71997700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c10c710 con 0x7f7c6c108de0 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c6affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c108660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c6affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c108660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44354/0 (socket says 172.21.15.66:44354) 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c6affd700 1 -- 172.21.15.66:0/3873211554 learned_addr learned my addr 172.21.15.66:0/3873211554 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c6affd700 1 -- 172.21.15.66:0/3873211554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c108ba0 con 0x7f7c6c108de0 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.408+0000 7f7c6affd700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c108660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c54000c00 tx=0x7f7c54011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.409+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c54003aa0 con 0x7f7c6c108de0 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.409+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c106d90 con 0x7f7c6c108de0 2024-04-21T08:58:40.045 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.409+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5401b8f0 con 0x7f7c6c108de0 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.409+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c54023870 con 0x7f7c6c108de0 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.409+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c1072d0 con 0x7f7c6c108de0 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.410+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7c54023b70 con 0x7f7c6c108de0 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.410+0000 7f7c70995700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.410+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 --> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c6c069b50 con 0x7f7c580396c0 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.410+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c5404ede0 con 0x7f7c6c108de0 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.410+0000 7f7c6a7fc700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 msgr2=0x7f7c5803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/763565357 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.410+0000 7f7c6a7fc700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.611+0000 7f7c6a7fc700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 msgr2=0x7f7c5803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/763565357 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:33.611+0000 7f7c6a7fc700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:34.012+0000 7f7c6a7fc700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 msgr2=0x7f7c5803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/763565357 2024-04-21T08:58:40.046 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:34.012+0000 7f7c6a7fc700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:34.812+0000 7f7c6a7fc700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 msgr2=0x7f7c5803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/763565357 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:34.812+0000 7f7c6a7fc700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:36.413+0000 7f7c6a7fc700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 msgr2=0x7f7c5803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/763565357 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:36.413+0000 7f7c6a7fc700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:38.212+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f7c5401b440 con 0x7f7c6c108de0 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:38.212+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 msgr2=0x7f7c5803bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:38.212+0000 7f7c70995700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.214+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f7c5404fe30 con 0x7f7c6c108de0 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.214+0000 7f7c70995700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7c5803cf50 0x7f7c5803f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.214+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c5402de50 con 0x7f7c5803cf50 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.214+0000 7f7c6a7fc700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7c5803cf50 0x7f7c5803f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.215+0000 7f7c6a7fc700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7c5803cf50 0x7f7c5803f340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c003a10 tx=0x7f7c5c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.216+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7c5402de50 con 0x7f7c5803cf50 2024-04-21T08:58:40.047 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.222+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7c6c103980 con 0x7f7c5803cf50 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c70995700 1 -- 172.21.15.66:0/3873211554 <== mgr.14152 v2:172.21.15.66:6800/3032531159 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7c6c103980 con 0x7f7c5803cf50 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7c5803cf50 msgr2=0x7f7c5803f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7c5803cf50 0x7f7c5803f340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c003a10 tx=0x7f7c5c0092b0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 msgr2=0x7f7c6c108660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c108660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c54000c00 tx=0x7f7c54011aa0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 shutdown_connections 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7c5803cf50 0x7f7c5803f340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c108de0 0x7f7c6c108660 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 --2- 172.21.15.66:0/3873211554 >> [v2:172.21.15.66:6800/763565357,v1:172.21.15.66:6801/763565357] conn(0x7f7c580396c0 0x7f7c5803bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 >> 172.21.15.66:0/3873211554 conn(0x7f7c6c1027d0 msgr2=0x7f7c6c103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 shutdown_connections 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:39.223+0000 7f7c71997700 1 -- 172.21.15.66:0/3873211554 wait complete. 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 9 is available 2024-04-21T08:58:40.048 INFO:teuthology.orchestra.run.smithi066.stdout:Generating a dashboard self-signed certificate... 2024-04-21T08:58:41.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:40 smithi066 ceph-mon[27953]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:58:41.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:40 smithi066 ceph-mon[27953]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:58:41.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:41.471 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.532+0000 7fe6081d8700 1 Processor -- start 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.533+0000 7fe6081d8700 1 -- start start 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.533+0000 7fe6081d8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60010b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.533+0000 7fe6081d8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe60007be20 con 0x7fe600108e30 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.533+0000 7fe605f74700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60010b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.533+0000 7fe605f74700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60010b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42040/0 (socket says 172.21.15.66:42040) 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.533+0000 7fe605f74700 1 -- 172.21.15.66:0/1416562652 learned_addr learned my addr 172.21.15.66:0/1416562652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.534+0000 7fe605f74700 1 -- 172.21.15.66:0/1416562652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60010b760 con 0x7fe600108e30 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.534+0000 7fe605f74700 1 --2- 172.21.15.66:0/1416562652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60010b220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0098a0 tx=0x7fe5fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67a4648d1527478 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.534+0000 7fe604f72700 1 -- 172.21.15.66:0/1416562652 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5fc004160 con 0x7fe600108e30 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.534+0000 7fe604f72700 1 -- 172.21.15.66:0/1416562652 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5fc004300 con 0x7fe600108e30 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.534+0000 7fe604f72700 1 -- 172.21.15.66:0/1416562652 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5fc011a20 con 0x7fe600108e30 2024-04-21T08:58:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 -- 172.21.15.66:0/1416562652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 msgr2=0x7fe60010b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 --2- 172.21.15.66:0/1416562652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60010b220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0098a0 tx=0x7fe5fc009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 -- 172.21.15.66:0/1416562652 shutdown_connections 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 --2- 172.21.15.66:0/1416562652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60010b220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 -- 172.21.15.66:0/1416562652 >> 172.21.15.66:0/1416562652 conn(0x7fe600102820 msgr2=0x7fe600104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 -- 172.21.15.66:0/1416562652 shutdown_connections 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 -- 172.21.15.66:0/1416562652 wait complete. 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.535+0000 7fe6081d8700 1 Processor -- start 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe6081d8700 1 -- start start 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe6081d8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60019c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe6081d8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe60010c700 con 0x7fe600108e30 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe605f74700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60019c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe605f74700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60019c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42048/0 (socket says 172.21.15.66:42048) 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe605f74700 1 -- 172.21.15.66:0/3259574190 learned_addr learned my addr 172.21.15.66:0/3259574190 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:41.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe605f74700 1 -- 172.21.15.66:0/3259574190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60019c6c0 con 0x7fe600108e30 2024-04-21T08:58:41.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.536+0000 7fe605f74700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60019c180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0116e0 tx=0x7fe5fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:41.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.537+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5fc003a60 con 0x7fe600108e30 2024-04-21T08:58:41.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.537+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5fc01b8f0 con 0x7fe600108e30 2024-04-21T08:58:41.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.537+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60019c950 con 0x7fe600108e30 2024-04-21T08:58:41.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.537+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5fc019860 con 0x7fe600108e30 2024-04-21T08:58:41.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.537+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60019f640 con 0x7fe600108e30 2024-04-21T08:58:41.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.538+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fe5fc019b30 con 0x7fe600108e30 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.538+0000 7fe5f6ffd700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe5ec0395b0 0x7fe5ec03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.538+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe5fc04dc10 con 0x7fe600108e30 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.538+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6000640a0 con 0x7fe600108e30 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.538+0000 7fe605773700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe5ec0395b0 0x7fe5ec03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.539+0000 7fe605773700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe5ec0395b0 0x7fe5ec03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006fd0 tx=0x7fe5f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.548+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5fc01f030 con 0x7fe600108e30 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.761+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe600103910 con 0x7fe5ec0395b0 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.774+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fe5fc01a8f0 con 0x7fe600108e30 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.872+0000 7fe5f6ffd700 1 -- 172.21.15.66:0/3259574190 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe600103910 con 0x7fe5ec0395b0 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe5ec0395b0 msgr2=0x7fe5ec03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:41.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe5ec0395b0 0x7fe5ec03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006fd0 tx=0x7fe5f0006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 msgr2=0x7fe60019c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60019c180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0116e0 tx=0x7fe5fc011aa0 comp rx=0 tx=0).stop 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 shutdown_connections 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe5ec0395b0 0x7fe5ec03ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 --2- 172.21.15.66:0/3259574190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe600108e30 0x7fe60019c180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 >> 172.21.15.66:0/3259574190 conn(0x7fe600102820 msgr2=0x7fe600103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 shutdown_connections 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:40.874+0000 7fe6081d8700 1 -- 172.21.15.66:0/3259574190 wait complete. 2024-04-21T08:58:41.476 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial admin user... 2024-04-21T08:58:42.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:40] ENGINE Bus STARTING 2024-04-21T08:58:42.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:42.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: mgrmap e12: x(active, since 2s) 2024-04-21T08:58:42.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:40] ENGINE Serving on https://172.21.15.66:7150 2024-04-21T08:58:42.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:42.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:42.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:40] ENGINE Serving on http://172.21.15.66:8765 2024-04-21T08:58:42.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: [21/Apr/2024:08:58:40] ENGINE Bus STARTED 2024-04-21T08:58:42.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:43.040 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$p4ZU.tYgh1lSsdQ9ImepHO6MmPOmOlRwsJGogy.7tb/fwTbybniDa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713689922, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-21T08:58:43.040 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.864+0000 7f2c85fba700 1 Processor -- start 2024-04-21T08:58:43.040 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.864+0000 7f2c85fba700 1 -- start start 2024-04-21T08:58:43.040 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.864+0000 7f2c85fba700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c80108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:43.040 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.864+0000 7f2c85fba700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80108d30 con 0x7f2c80108340 2024-04-21T08:58:43.040 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.864+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c80108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:43.040 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.864+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c80108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42050/0 (socket says 172.21.15.66:42050) 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.865+0000 7f2c7f7fe700 1 -- 172.21.15.66:0/3872211136 learned_addr learned my addr 172.21.15.66:0/3872211136 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.865+0000 7f2c7f7fe700 1 -- 172.21.15.66:0/3872211136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c80109540 con 0x7f2c80108340 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.865+0000 7f2c7f7fe700 1 --2- 172.21.15.66:0/3872211136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c80108760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009e80 tx=0x7f2c7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5759967a193ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.865+0000 7f2c7e7fc700 1 -- 172.21.15.66:0/3872211136 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70017070 con 0x7f2c80108340 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.865+0000 7f2c7e7fc700 1 -- 172.21.15.66:0/3872211136 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7000b8c0 con 0x7f2c80108340 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.865+0000 7f2c7e7fc700 1 -- 172.21.15.66:0/3872211136 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70004490 con 0x7f2c80108340 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.866+0000 7f2c85fba700 1 -- 172.21.15.66:0/3872211136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 msgr2=0x7f2c80108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.866+0000 7f2c85fba700 1 --2- 172.21.15.66:0/3872211136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c80108760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009e80 tx=0x7f2c7000b330 comp rx=0 tx=0).stop 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.866+0000 7f2c85fba700 1 -- 172.21.15.66:0/3872211136 shutdown_connections 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.866+0000 7f2c85fba700 1 --2- 172.21.15.66:0/3872211136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c80108760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.866+0000 7f2c85fba700 1 -- 172.21.15.66:0/3872211136 >> 172.21.15.66:0/3872211136 conn(0x7f2c801038c0 msgr2=0x7f2c80105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:43.041 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.866+0000 7f2c85fba700 1 -- 172.21.15.66:0/3872211136 shutdown_connections 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.866+0000 7f2c85fba700 1 -- 172.21.15.66:0/3872211136 wait complete. 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.867+0000 7f2c85fba700 1 Processor -- start 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.867+0000 7f2c85fba700 1 -- start start 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.867+0000 7f2c85fba700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c8019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.867+0000 7f2c85fba700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8010c8f0 con 0x7f2c80108340 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c8019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c8019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42064/0 (socket says 172.21.15.66:42064) 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7f7fe700 1 -- 172.21.15.66:0/2151895309 learned_addr learned my addr 172.21.15.66:0/2151895309 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7f7fe700 1 -- 172.21.15.66:0/2151895309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8019c6f0 con 0x7f2c80108340 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7f7fe700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c8019c1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c70004220 tx=0x7f2c7000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7cff9700 1 -- 172.21.15.66:0/2151895309 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70020020 con 0x7f2c80108340 2024-04-21T08:58:43.042 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7cff9700 1 -- 172.21.15.66:0/2151895309 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c70003b50 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8019c980 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c7cff9700 1 -- 172.21.15.66:0/2151895309 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70017040 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.868+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8019f670 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.869+0000 7f2c7cff9700 1 -- 172.21.15.66:0/2151895309 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f2c70023020 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.869+0000 7f2c7cff9700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2c6c0396d0 0x7f2c6c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.869+0000 7f2c7cff9700 1 -- 172.21.15.66:0/2151895309 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c7000f050 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.870+0000 7f2c7effd700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2c6c0396d0 0x7f2c6c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.870+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c800640a0 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.870+0000 7f2c7effd700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2c6c0396d0 0x7f2c6c03bb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c68006fd0 tx=0x7f2c68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:41.875+0000 7f2c7cff9700 1 -- 172.21.15.66:0/2151895309 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c7002a020 con 0x7f2c80108340 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.110+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2c8019f840 con 0x7f2c6c0396d0 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.356+0000 7f2c7cff9700 1 -- 172.21.15.66:0/2151895309 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2c8019f840 con 0x7f2c6c0396d0 2024-04-21T08:58:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2c6c0396d0 msgr2=0x7f2c6c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2c6c0396d0 0x7f2c6c03bb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c68006fd0 tx=0x7f2c68006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 msgr2=0x7f2c8019c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c8019c1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c70004220 tx=0x7f2c7000bd30 comp rx=0 tx=0).stop 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 shutdown_connections 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2c6c0396d0 0x7f2c6c03bb90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 --2- 172.21.15.66:0/2151895309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c80108340 0x7f2c8019c1b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.358+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 >> 172.21.15.66:0/2151895309 conn(0x7f2c801038c0 msgr2=0x7f2c80104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.359+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 shutdown_connections 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:42.359+0000 7f2c85fba700 1 -- 172.21.15.66:0/2151895309 wait complete. 2024-04-21T08:58:43.044 INFO:teuthology.orchestra.run.smithi066.stdout:Fetching dashboard port number... 2024-04-21T08:58:43.501 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:43 smithi066 ceph-mon[27953]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:43.501 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:44.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 8443 2024-04-21T08:58:44.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0853235700 1 Processor -- start 2024-04-21T08:58:44.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0853235700 1 -- start start 2024-04-21T08:58:44.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0853235700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0853235700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f084c108d30 con 0x7f084c108340 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0850fd1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0850fd1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42074/0 (socket says 172.21.15.66:42074) 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0850fd1700 1 -- 172.21.15.66:0/3653990769 learned_addr learned my addr 172.21.15.66:0/3653990769 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0850fd1700 1 -- 172.21.15.66:0/3653990769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c109540 con 0x7f084c108340 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.514+0000 7f0850fd1700 1 --2- 172.21.15.66:0/3653990769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c108760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08400098a0 tx=0x7f0840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378c3a9bd087815f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f084b7fe700 1 -- 172.21.15.66:0/3653990769 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0840004160 con 0x7f084c108340 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f084b7fe700 1 -- 172.21.15.66:0/3653990769 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0840004300 con 0x7f084c108340 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f084b7fe700 1 -- 172.21.15.66:0/3653990769 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0840011a20 con 0x7f084c108340 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f0853235700 1 -- 172.21.15.66:0/3653990769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 msgr2=0x7f084c108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f0853235700 1 --2- 172.21.15.66:0/3653990769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c108760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08400098a0 tx=0x7f0840009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:44.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f0853235700 1 -- 172.21.15.66:0/3653990769 shutdown_connections 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f0853235700 1 --2- 172.21.15.66:0/3653990769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c108760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f0853235700 1 -- 172.21.15.66:0/3653990769 >> 172.21.15.66:0/3653990769 conn(0x7f084c1038c0 msgr2=0x7f084c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f0853235700 1 -- 172.21.15.66:0/3653990769 shutdown_connections 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.515+0000 7f0853235700 1 -- 172.21.15.66:0/3653990769 wait complete. 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0853235700 1 Processor -- start 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0853235700 1 -- start start 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0853235700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0853235700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10c8f0 con 0x7f084c108340 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0850fd1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0850fd1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c19e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42086/0 (socket says 172.21.15.66:42086) 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0850fd1700 1 -- 172.21.15.66:0/2187609338 learned_addr learned my addr 172.21.15.66:0/2187609338 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.516+0000 7f0850fd1700 1 -- 172.21.15.66:0/2187609338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c19e8f0 con 0x7f084c108340 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.517+0000 7f0850fd1700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c19e3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f08400116e0 tx=0x7f0840011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:44.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.517+0000 7f0849ffb700 1 -- 172.21.15.66:0/2187609338 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08400039e0 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.517+0000 7f0849ffb700 1 -- 172.21.15.66:0/2187609338 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f084001b8f0 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.517+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c19eb80 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.517+0000 7f0849ffb700 1 -- 172.21.15.66:0/2187609338 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0840023760 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.517+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a1870 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.518+0000 7f0849ffb700 1 -- 172.21.15.66:0/2187609338 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f084001b440 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.518+0000 7f0849ffb700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f083c039670 0x7f083c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.518+0000 7f0849ffb700 1 -- 172.21.15.66:0/2187609338 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f084004e040 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.518+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c0640a0 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.518+0000 7f084bfff700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f083c039670 0x7f083c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.519+0000 7f084bfff700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f083c039670 0x7f083c03bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0838006fd0 tx=0x7f0838006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.523+0000 7f0849ffb700 1 -- 172.21.15.66:0/2187609338 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f084001f030 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.732+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f084c1a1a40 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.732+0000 7f0849ffb700 1 -- 172.21.15.66:0/2187609338 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f08400292a0 con 0x7f084c108340 2024-04-21T08:58:44.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f083c039670 msgr2=0x7f083c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f083c039670 0x7f083c03bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0838006fd0 tx=0x7f0838006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 msgr2=0x7f084c19e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c19e3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f08400116e0 tx=0x7f0840011aa0 comp rx=0 tx=0).stop 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 shutdown_connections 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f083c039670 0x7f083c03bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 --2- 172.21.15.66:0/2187609338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f084c108340 0x7f084c19e3b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 >> 172.21.15.66:0/2187609338 conn(0x7f084c1038c0 msgr2=0x7f084c104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 shutdown_connections 2024-04-21T08:58:44.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:43.735+0000 7f0853235700 1 -- 172.21.15.66:0/2187609338 wait complete. 2024-04-21T08:58:44.481 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:58:44.481 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-04-21T08:58:44.488 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:58:44.488 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-04-21T08:58:44.488 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-04-21T08:58:44.488 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-21T08:58:44.490 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph Dashboard is now available at: 2024-04-21T08:58:44.491 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:44.491 INFO:teuthology.orchestra.run.smithi066.stdout: URL: https://smithi066:8443/ 2024-04-21T08:58:44.491 INFO:teuthology.orchestra.run.smithi066.stdout: User: admin 2024-04-21T08:58:44.491 INFO:teuthology.orchestra.run.smithi066.stdout: Password: nwuqpjoy6e 2024-04-21T08:58:44.491 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:44.491 INFO:teuthology.orchestra.run.smithi066.stdout:Saving cluster configuration to /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config directory 2024-04-21T08:58:44.491 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling autotune for osd_memory_target 2024-04-21T08:58:44.614 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:44 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2187609338' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-21T08:58:45.587 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:45 smithi066 ceph-mon[27953]: mgrmap e13: x(active, since 6s) 2024-04-21T08:58:45.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e92842700 1 Processor -- start 2024-04-21T08:58:45.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e92842700 1 -- start start 2024-04-21T08:58:45.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e92842700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:45.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e92842700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c108d30 con 0x7f1e8c108340 2024-04-21T08:58:45.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:45.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51724/0 (socket says 172.21.15.66:51724) 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e8bfff700 1 -- 172.21.15.66:0/742753501 learned_addr learned my addr 172.21.15.66:0/742753501 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.002+0000 7f1e8bfff700 1 -- 172.21.15.66:0/742753501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c109540 con 0x7f1e8c108340 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e8bfff700 1 --2- 172.21.15.66:0/742753501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c108760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0098a0 tx=0x7f1e7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36104be1f7bf1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e8affd700 1 -- 172.21.15.66:0/742753501 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e7c004160 con 0x7f1e8c108340 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e8affd700 1 -- 172.21.15.66:0/742753501 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c004300 con 0x7f1e8c108340 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e8affd700 1 -- 172.21.15.66:0/742753501 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e7c011a20 con 0x7f1e8c108340 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e92842700 1 -- 172.21.15.66:0/742753501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 msgr2=0x7f1e8c108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e92842700 1 --2- 172.21.15.66:0/742753501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c108760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0098a0 tx=0x7f1e7c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e92842700 1 -- 172.21.15.66:0/742753501 shutdown_connections 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e92842700 1 --2- 172.21.15.66:0/742753501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c108760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e92842700 1 -- 172.21.15.66:0/742753501 >> 172.21.15.66:0/742753501 conn(0x7f1e8c1038c0 msgr2=0x7f1e8c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e92842700 1 -- 172.21.15.66:0/742753501 shutdown_connections 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.003+0000 7f1e92842700 1 -- 172.21.15.66:0/742753501 wait complete. 2024-04-21T08:58:45.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.004+0000 7f1e92842700 1 Processor -- start 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.004+0000 7f1e92842700 1 -- start start 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.004+0000 7f1e92842700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.004+0000 7f1e92842700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c10c8f0 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.004+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.004+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51728/0 (socket says 172.21.15.66:51728) 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.004+0000 7f1e8bfff700 1 -- 172.21.15.66:0/2411485394 learned_addr learned my addr 172.21.15.66:0/2411485394 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.005+0000 7f1e8bfff700 1 -- 172.21.15.66:0/2411485394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c19c6f0 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.005+0000 7f1e8bfff700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c19c1b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009520 tx=0x7f1e7c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.005+0000 7f1e897fa700 1 -- 172.21.15.66:0/2411485394 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e7c0039e0 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.005+0000 7f1e897fa700 1 -- 172.21.15.66:0/2411485394 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c01b8f0 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.005+0000 7f1e897fa700 1 -- 172.21.15.66:0/2411485394 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e7c019770 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.005+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c19c980 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.005+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c19f670 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.006+0000 7f1e897fa700 1 -- 172.21.15.66:0/2411485394 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1e7c019a70 con 0x7f1e8c108340 2024-04-21T08:58:45.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.006+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c0640a0 con 0x7f1e8c108340 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.006+0000 7f1e897fa700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e74039760 0x7f1e7403bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.006+0000 7f1e897fa700 1 -- 172.21.15.66:0/2411485394 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1e7c04e260 con 0x7f1e8c108340 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.006+0000 7f1e8b7fe700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e74039760 0x7f1e7403bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.007+0000 7f1e8b7fe700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e74039760 0x7f1e7403bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e80006fd0 tx=0x7f1e80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.011+0000 7f1e897fa700 1 -- 172.21.15.66:0/2411485394 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e7c01f030 con 0x7f1e8c108340 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.220+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f1e8c19fcd0 con 0x7f1e8c108340 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.221+0000 7f1e897fa700 1 -- 172.21.15.66:0/2411485394 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f1e7c01baa0 con 0x7f1e8c108340 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e74039760 msgr2=0x7f1e7403bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e74039760 0x7f1e7403bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e80006fd0 tx=0x7f1e80006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 msgr2=0x7f1e8c19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c19c1b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009520 tx=0x7f1e7c011aa0 comp rx=0 tx=0).stop 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 shutdown_connections 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e74039760 0x7f1e7403bc20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:45.891 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 --2- 172.21.15.66:0/2411485394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e8c108340 0x7f1e8c19c1b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:45.892 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 >> 172.21.15.66:0/2411485394 conn(0x7f1e8c1038c0 msgr2=0x7f1e8c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:45.892 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 shutdown_connections 2024-04-21T08:58:45.892 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:45.223+0000 7f1e92842700 1 -- 172.21.15.66:0/2411485394 wait complete. 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.377+0000 7f8be54f4700 1 Processor -- start 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.377+0000 7f8be54f4700 1 -- start start 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.377+0000 7f8be54f4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be0106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.377+0000 7f8be54f4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0107020 con 0x7f8be0106630 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bdeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be0106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bdeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be0106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51740/0 (socket says 172.21.15.66:51740) 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bdeffd700 1 -- 172.21.15.66:0/2243867033 learned_addr learned my addr 172.21.15.66:0/2243867033 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bdeffd700 1 -- 172.21.15.66:0/2243867033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01095c0 con 0x7f8be0106630 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bdeffd700 1 --2- 172.21.15.66:0/2243867033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be0106a50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009e80 tx=0x7f8bd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a295dd85afb00d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bddffb700 1 -- 172.21.15.66:0/2243867033 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bd8017070 con 0x7f8be0106630 2024-04-21T08:58:47.328 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bddffb700 1 -- 172.21.15.66:0/2243867033 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd800b8c0 con 0x7f8be0106630 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.378+0000 7f8bddffb700 1 -- 172.21.15.66:0/2243867033 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bd8004490 con 0x7f8be0106630 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.379+0000 7f8be54f4700 1 -- 172.21.15.66:0/2243867033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 msgr2=0x7f8be0106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.379+0000 7f8be54f4700 1 --2- 172.21.15.66:0/2243867033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be0106a50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009e80 tx=0x7f8bd800b330 comp rx=0 tx=0).stop 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.379+0000 7f8be54f4700 1 -- 172.21.15.66:0/2243867033 shutdown_connections 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.379+0000 7f8be54f4700 1 --2- 172.21.15.66:0/2243867033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be0106a50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.379+0000 7f8be54f4700 1 -- 172.21.15.66:0/2243867033 >> 172.21.15.66:0/2243867033 conn(0x7f8be0101fa0 msgr2=0x7f8be01043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.379+0000 7f8be54f4700 1 -- 172.21.15.66:0/2243867033 shutdown_connections 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.379+0000 7f8be54f4700 1 -- 172.21.15.66:0/2243867033 wait complete. 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8be54f4700 1 Processor -- start 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8be54f4700 1 -- start start 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8be54f4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8be54f4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8be010c970 con 0x7f8be0106630 2024-04-21T08:58:47.329 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8bdeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8bdeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be019c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51744/0 (socket says 172.21.15.66:51744) 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8bdeffd700 1 -- 172.21.15.66:0/4030236329 learned_addr learned my addr 172.21.15.66:0/4030236329 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.380+0000 7f8bdeffd700 1 -- 172.21.15.66:0/4030236329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be019c740 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.381+0000 7f8bdeffd700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be019c200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b7b0 tx=0x7f8bd800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.381+0000 7f8bd7fff700 1 -- 172.21.15.66:0/4030236329 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bd8020020 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.381+0000 7f8bd7fff700 1 -- 172.21.15.66:0/4030236329 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd8003b50 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.381+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be019c9d0 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.381+0000 7f8bd7fff700 1 -- 172.21.15.66:0/4030236329 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bd8017040 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.381+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be019f6c0 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.382+0000 7f8bd7fff700 1 -- 172.21.15.66:0/4030236329 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8bd8023020 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.382+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be00640a0 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.382+0000 7f8bd7fff700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bcc0397b0 0x7f8bcc03bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.382+0000 7f8bd7fff700 1 -- 172.21.15.66:0/4030236329 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8bd800f050 con 0x7f8be0106630 2024-04-21T08:58:47.330 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.382+0000 7f8bde7fc700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bcc0397b0 0x7f8bcc03bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.383+0000 7f8bde7fc700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bcc0397b0 0x7f8bcc03bc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8006fd0 tx=0x7f8bc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.387+0000 7f8bd7fff700 1 -- 172.21.15.66:0/4030236329 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bd802a020 con 0x7f8be0106630 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.711+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8be019f890 con 0x7f8be0106630 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.713+0000 7f8bd7fff700 1 -- 172.21.15.66:0/4030236329 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f8be019f890 con 0x7f8be0106630 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.715+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bcc0397b0 msgr2=0x7f8bcc03bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.715+0000 7f8be54f4700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bcc0397b0 0x7f8bcc03bc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8006fd0 tx=0x7f8bc8006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.715+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 msgr2=0x7f8be019c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.715+0000 7f8be54f4700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be019c200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b7b0 tx=0x7f8bd800bd30 comp rx=0 tx=0).stop 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.716+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 shutdown_connections 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.716+0000 7f8be54f4700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bcc0397b0 0x7f8bcc03bc70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.716+0000 7f8be54f4700 1 --2- 172.21.15.66:0/4030236329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be0106630 0x7f8be019c200 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.716+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 >> 172.21.15.66:0/4030236329 conn(0x7f8be0101fa0 msgr2=0x7f8be01029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.716+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 shutdown_connections 2024-04-21T08:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2024-04-21T08:58:46.716+0000 7f8be54f4700 1 -- 172.21.15.66:0/4030236329 wait complete. 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout:Or, if you are only running a single cluster on this host: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph telemetry on 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout:For more information see: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-21T08:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout:Bootstrap complete. 2024-04-21T08:58:47.366 INFO:tasks.cephadm:Fetching config... 2024-04-21T08:58:47.367 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:58:47.367 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-21T08:58:47.383 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-21T08:58:47.383 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:58:47.383 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-21T08:58:47.439 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-21T08:58:47.439 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:58:47.439 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-21T08:58:47.505 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-21T08:58:47.575 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:58:47.576 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-21T08:58:47.592 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-21T08:58:47.592 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHxzkvE3r8M/y9o8yG6CgqqAjHFuQ7Ot1M4wL+tSufdAL1VRwX/aMHcEt8sd7G5WZjjuyl1XuqME8tTrEnLsbKMgx3gncXfqAW9cBYdSiwhnU2FVpPahCo+Ify/YjjEntyiMkOBW7wajCmk4zVj1BDiiEtuoCuzJNtHiqAAGtKrXmXkfCYjQBaCpsoZUultKw4qgXlHfFrDOYPWpqQkiOyng/574Lsxcx6/72BjW8ZKZVm9/SdfYCzUR/841YRY2e9ILR4I0HvQ9p+cpsWb6bVJkuGfcLRoMyvUHR49MH1Kn3jMcFcGO9pjrPBbYXhsiV6g4hEvUAVhf6Y0xmBpB1STcDgbCORvVXCXkfQIwJFfoH754omPRRmNbN0Y+wOQ0urL5irMh+HRBGXwM8WEzdKDHS/JP1xyBey0xiprpjEASjElQ2pJdnjS9Vf5eM/TmTzahyxNi2e4iVLcnEDCUJEJqWGqwnDDsaOHFAvdOtogz5Bz3OF5zCblPHpxmrp6c= ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:58:47.669 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHxzkvE3r8M/y9o8yG6CgqqAjHFuQ7Ot1M4wL+tSufdAL1VRwX/aMHcEt8sd7G5WZjjuyl1XuqME8tTrEnLsbKMgx3gncXfqAW9cBYdSiwhnU2FVpPahCo+Ify/YjjEntyiMkOBW7wajCmk4zVj1BDiiEtuoCuzJNtHiqAAGtKrXmXkfCYjQBaCpsoZUultKw4qgXlHfFrDOYPWpqQkiOyng/574Lsxcx6/72BjW8ZKZVm9/SdfYCzUR/841YRY2e9ILR4I0HvQ9p+cpsWb6bVJkuGfcLRoMyvUHR49MH1Kn3jMcFcGO9pjrPBbYXhsiV6g4hEvUAVhf6Y0xmBpB1STcDgbCORvVXCXkfQIwJFfoH754omPRRmNbN0Y+wOQ0urL5irMh+HRBGXwM8WEzdKDHS/JP1xyBey0xiprpjEASjElQ2pJdnjS9Vf5eM/TmTzahyxNi2e4iVLcnEDCUJEJqWGqwnDDsaOHFAvdOtogz5Bz3OF5zCblPHpxmrp6c= ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:58:47.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHxzkvE3r8M/y9o8yG6CgqqAjHFuQ7Ot1M4wL+tSufdAL1VRwX/aMHcEt8sd7G5WZjjuyl1XuqME8tTrEnLsbKMgx3gncXfqAW9cBYdSiwhnU2FVpPahCo+Ify/YjjEntyiMkOBW7wajCmk4zVj1BDiiEtuoCuzJNtHiqAAGtKrXmXkfCYjQBaCpsoZUultKw4qgXlHfFrDOYPWpqQkiOyng/574Lsxcx6/72BjW8ZKZVm9/SdfYCzUR/841YRY2e9ILR4I0HvQ9p+cpsWb6bVJkuGfcLRoMyvUHR49MH1Kn3jMcFcGO9pjrPBbYXhsiV6g4hEvUAVhf6Y0xmBpB1STcDgbCORvVXCXkfQIwJFfoH754omPRRmNbN0Y+wOQ0urL5irMh+HRBGXwM8WEzdKDHS/JP1xyBey0xiprpjEASjElQ2pJdnjS9Vf5eM/TmTzahyxNi2e4iVLcnEDCUJEJqWGqwnDDsaOHFAvdOtogz5Bz3OF5zCblPHpxmrp6c= ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:58:47.721 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHxzkvE3r8M/y9o8yG6CgqqAjHFuQ7Ot1M4wL+tSufdAL1VRwX/aMHcEt8sd7G5WZjjuyl1XuqME8tTrEnLsbKMgx3gncXfqAW9cBYdSiwhnU2FVpPahCo+Ify/YjjEntyiMkOBW7wajCmk4zVj1BDiiEtuoCuzJNtHiqAAGtKrXmXkfCYjQBaCpsoZUultKw4qgXlHfFrDOYPWpqQkiOyng/574Lsxcx6/72BjW8ZKZVm9/SdfYCzUR/841YRY2e9ILR4I0HvQ9p+cpsWb6bVJkuGfcLRoMyvUHR49MH1Kn3jMcFcGO9pjrPBbYXhsiV6g4hEvUAVhf6Y0xmBpB1STcDgbCORvVXCXkfQIwJFfoH754omPRRmNbN0Y+wOQ0urL5irMh+HRBGXwM8WEzdKDHS/JP1xyBey0xiprpjEASjElQ2pJdnjS9Vf5eM/TmTzahyxNi2e4iVLcnEDCUJEJqWGqwnDDsaOHFAvdOtogz5Bz3OF5zCblPHpxmrp6c= ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:58:47.733 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHxzkvE3r8M/y9o8yG6CgqqAjHFuQ7Ot1M4wL+tSufdAL1VRwX/aMHcEt8sd7G5WZjjuyl1XuqME8tTrEnLsbKMgx3gncXfqAW9cBYdSiwhnU2FVpPahCo+Ify/YjjEntyiMkOBW7wajCmk4zVj1BDiiEtuoCuzJNtHiqAAGtKrXmXkfCYjQBaCpsoZUultKw4qgXlHfFrDOYPWpqQkiOyng/574Lsxcx6/72BjW8ZKZVm9/SdfYCzUR/841YRY2e9ILR4I0HvQ9p+cpsWb6bVJkuGfcLRoMyvUHR49MH1Kn3jMcFcGO9pjrPBbYXhsiV6g4hEvUAVhf6Y0xmBpB1STcDgbCORvVXCXkfQIwJFfoH754omPRRmNbN0Y+wOQ0urL5irMh+HRBGXwM8WEzdKDHS/JP1xyBey0xiprpjEASjElQ2pJdnjS9Vf5eM/TmTzahyxNi2e4iVLcnEDCUJEJqWGqwnDDsaOHFAvdOtogz5Bz3OF5zCblPHpxmrp6c= ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:58:47.771 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHxzkvE3r8M/y9o8yG6CgqqAjHFuQ7Ot1M4wL+tSufdAL1VRwX/aMHcEt8sd7G5WZjjuyl1XuqME8tTrEnLsbKMgx3gncXfqAW9cBYdSiwhnU2FVpPahCo+Ify/YjjEntyiMkOBW7wajCmk4zVj1BDiiEtuoCuzJNtHiqAAGtKrXmXkfCYjQBaCpsoZUultKw4qgXlHfFrDOYPWpqQkiOyng/574Lsxcx6/72BjW8ZKZVm9/SdfYCzUR/841YRY2e9ILR4I0HvQ9p+cpsWb6bVJkuGfcLRoMyvUHR49MH1Kn3jMcFcGO9pjrPBbYXhsiV6g4hEvUAVhf6Y0xmBpB1STcDgbCORvVXCXkfQIwJFfoH754omPRRmNbN0Y+wOQ0urL5irMh+HRBGXwM8WEzdKDHS/JP1xyBey0xiprpjEASjElQ2pJdnjS9Vf5eM/TmTzahyxNi2e4iVLcnEDCUJEJqWGqwnDDsaOHFAvdOtogz5Bz3OF5zCblPHpxmrp6c= ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:58:47.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-21T08:58:47.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:47 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4030236329' entity='client.admin' 2024-04-21T08:58:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:58:50.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.134+0000 7f99cee7d700 1 -- 172.21.15.66:0/1735362048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 msgr2=0x7f99c81017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.134+0000 7f99cee7d700 1 --2- 172.21.15.66:0/1735362048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81017c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009910 tx=0x7f99bc009c20 comp rx=0 tx=0).stop 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.134+0000 7f99cee7d700 1 -- 172.21.15.66:0/1735362048 shutdown_connections 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.134+0000 7f99cee7d700 1 --2- 172.21.15.66:0/1735362048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81017c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.134+0000 7f99cee7d700 1 -- 172.21.15.66:0/1735362048 >> 172.21.15.66:0/1735362048 conn(0x7f99c80fc940 msgr2=0x7f99c80fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.134+0000 7f99cee7d700 1 -- 172.21.15.66:0/1735362048 shutdown_connections 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.134+0000 7f99cee7d700 1 -- 172.21.15.66:0/1735362048 wait complete. 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99cee7d700 1 Processor -- start 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99cee7d700 1 -- start start 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99cee7d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99cee7d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81072e0 con 0x7f99c81013a0 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99ccc19700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:50.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99ccc19700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51756/0 (socket says 172.21.15.66:51756) 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99ccc19700 1 -- 172.21.15.66:0/1821107999 learned_addr learned my addr 172.21.15.66:0/1821107999 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99ccc19700 1 -- 172.21.15.66:0/1821107999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c8196f20 con 0x7f99c81013a0 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.135+0000 7f99ccc19700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81969e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005800 tx=0x7f99bc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.136+0000 7f99c5ffb700 1 -- 172.21.15.66:0/1821107999 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc027070 con 0x7f99c81013a0 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.136+0000 7f99c5ffb700 1 -- 172.21.15.66:0/1821107999 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99bc00be10 con 0x7f99c81013a0 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.136+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c81971b0 con 0x7f99c81013a0 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.136+0000 7f99c5ffb700 1 -- 172.21.15.66:0/1821107999 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc00f460 con 0x7f99c81013a0 2024-04-21T08:58:50.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.136+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c8199ea0 con 0x7f99c81013a0 2024-04-21T08:58:50.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.137+0000 7f99c5ffb700 1 -- 172.21.15.66:0/1821107999 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f99bc02a070 con 0x7f99c81013a0 2024-04-21T08:58:50.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.137+0000 7f99c5ffb700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f99b80396b0 0x7f99b803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:50.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.137+0000 7f99c5ffb700 1 -- 172.21.15.66:0/1821107999 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f99bc01e030 con 0x7f99c81013a0 2024-04-21T08:58:50.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.137+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c8107a60 con 0x7f99c81013a0 2024-04-21T08:58:50.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.137+0000 7f99c7fff700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f99b80396b0 0x7f99b803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:50.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.137+0000 7f99c7fff700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f99b80396b0 0x7f99b803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006fd0 tx=0x7f99b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:50.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.142+0000 7f99c5ffb700 1 -- 172.21.15.66:0/1821107999 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99bc031070 con 0x7f99c81013a0 2024-04-21T08:58:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.349+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f99c802df80 con 0x7f99c81013a0 2024-04-21T08:58:50.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.352+0000 7f99c5ffb700 1 -- 172.21.15.66:0/1821107999 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f99bc013070 con 0x7f99c81013a0 2024-04-21T08:58:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f99b80396b0 msgr2=0x7f99b803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f99b80396b0 0x7f99b803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006fd0 tx=0x7f99b4006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 msgr2=0x7f99c81969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81969e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005800 tx=0x7f99bc009f60 comp rx=0 tx=0).stop 2024-04-21T08:58:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 shutdown_connections 2024-04-21T08:58:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f99b80396b0 0x7f99b803bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 --2- 172.21.15.66:0/1821107999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99c81013a0 0x7f99c81969e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 >> 172.21.15.66:0/1821107999 conn(0x7f99c80fc940 msgr2=0x7f99c80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 shutdown_connections 2024-04-21T08:58:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:50.355+0000 7f99cee7d700 1 -- 172.21.15.66:0/1821107999 wait complete. 2024-04-21T08:58:50.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:58:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:50.917 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-21T08:58:50.917 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-21T08:58:51.156 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:58:51.550 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:51 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1821107999' entity='client.admin' 2024-04-21T08:58:51.550 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:51.550 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:52.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.674+0000 7f5abd365700 1 -- 172.21.15.66:0/3204871428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 msgr2=0x7f5ab8103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:52.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.674+0000 7f5abd365700 1 --2- 172.21.15.66:0/3204871428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8103fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009910 tx=0x7f5aa8009c20 comp rx=0 tx=0).stop 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.674+0000 7f5abd365700 1 -- 172.21.15.66:0/3204871428 shutdown_connections 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.674+0000 7f5abd365700 1 --2- 172.21.15.66:0/3204871428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8103fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.674+0000 7f5abd365700 1 -- 172.21.15.66:0/3204871428 >> 172.21.15.66:0/3204871428 conn(0x7f5ab80fda90 msgr2=0x7f5ab80ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.674+0000 7f5abd365700 1 -- 172.21.15.66:0/3204871428 shutdown_connections 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.674+0000 7f5abd365700 1 -- 172.21.15.66:0/3204871428 wait complete. 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.675+0000 7f5abd365700 1 Processor -- start 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.675+0000 7f5abd365700 1 -- start start 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.675+0000 7f5abd365700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.675+0000 7f5abd365700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8108520 con 0x7f5ab8103bb0 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.675+0000 7f5ab6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.675+0000 7f5ab6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51784/0 (socket says 172.21.15.66:51784) 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.675+0000 7f5ab6ffd700 1 -- 172.21.15.66:0/792682877 learned_addr learned my addr 172.21.15.66:0/792682877 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.676+0000 7f5ab6ffd700 1 -- 172.21.15.66:0/792682877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab8198150 con 0x7f5ab8103bb0 2024-04-21T08:58:52.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.676+0000 7f5ab6ffd700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8197c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005800 tx=0x7f5aa8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:52.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.676+0000 7f5a9ffff700 1 -- 172.21.15.66:0/792682877 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa8027070 con 0x7f5ab8103bb0 2024-04-21T08:58:52.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.676+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab81983e0 con 0x7f5ab8103bb0 2024-04-21T08:58:52.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.676+0000 7f5a9ffff700 1 -- 172.21.15.66:0/792682877 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa800be10 con 0x7f5ab8103bb0 2024-04-21T08:58:52.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.676+0000 7f5a9ffff700 1 -- 172.21.15.66:0/792682877 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa800f460 con 0x7f5ab8103bb0 2024-04-21T08:58:52.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.676+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab819b0d0 con 0x7f5ab8103bb0 2024-04-21T08:58:52.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.677+0000 7f5a9ffff700 1 -- 172.21.15.66:0/792682877 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5aa802a070 con 0x7f5ab8103bb0 2024-04-21T08:58:52.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.677+0000 7f5a9ffff700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5aa0039650 0x7f5aa003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:52.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.677+0000 7f5a9ffff700 1 -- 172.21.15.66:0/792682877 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5aa801e030 con 0x7f5ab8103bb0 2024-04-21T08:58:52.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.677+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab8108ca0 con 0x7f5ab8103bb0 2024-04-21T08:58:52.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.677+0000 7f5ab67fc700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5aa0039650 0x7f5aa003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:52.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.678+0000 7f5ab67fc700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5aa0039650 0x7f5aa003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006fd0 tx=0x7f5aac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:52.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.682+0000 7f5a9ffff700 1 -- 172.21.15.66:0/792682877 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5aa8031070 con 0x7f5ab8103bb0 2024-04-21T08:58:52.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.894+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f5ab8062e80 con 0x7f5aa0039650 2024-04-21T08:58:52.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.898+0000 7f5a9ffff700 1 -- 172.21.15.66:0/792682877 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ab8062e80 con 0x7f5aa0039650 2024-04-21T08:58:52.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5aa0039650 msgr2=0x7f5aa003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:52.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5aa0039650 0x7f5aa003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006fd0 tx=0x7f5aac006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:52.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 msgr2=0x7f5ab8197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:52.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8197c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005800 tx=0x7f5aa8009f60 comp rx=0 tx=0).stop 2024-04-21T08:58:52.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 shutdown_connections 2024-04-21T08:58:52.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5aa0039650 0x7f5aa003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:52.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 --2- 172.21.15.66:0/792682877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab8103bb0 0x7f5ab8197c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:52.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 >> 172.21.15.66:0/792682877 conn(0x7f5ab80fda90 msgr2=0x7f5ab80fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:52.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 shutdown_connections 2024-04-21T08:58:52.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:52.900+0000 7f5abd365700 1 -- 172.21.15.66:0/792682877 wait complete. 2024-04-21T08:58:53.645 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi083 2024-04-21T08:58:53.645 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:58:53.645 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:58:53.663 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:58:53.663 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:58:53.721 INFO:tasks.cephadm:Adding host smithi083 to orchestrator... 2024-04-21T08:58:53.721 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch host add smithi083 2024-04-21T08:58:53.967 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:58:54.215 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:54.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:58:55.073 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:54 smithi066 ceph-mon[27953]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:58:55.074 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:54 smithi066 ceph-mon[27953]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:58:55.074 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:54 smithi066 ceph-mon[27953]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.424+0000 7f54dc09b700 1 -- 172.21.15.66:0/2789680352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 msgr2=0x7f54d41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.424+0000 7f54dc09b700 1 --2- 172.21.15.66:0/2789680352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d41088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009910 tx=0x7f54d0009c20 comp rx=0 tx=0).stop 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.424+0000 7f54dc09b700 1 -- 172.21.15.66:0/2789680352 shutdown_connections 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.424+0000 7f54dc09b700 1 --2- 172.21.15.66:0/2789680352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d41088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.424+0000 7f54dc09b700 1 -- 172.21.15.66:0/2789680352 >> 172.21.15.66:0/2789680352 conn(0x7f54d40fdf60 msgr2=0x7f54d4100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.425+0000 7f54dc09b700 1 -- 172.21.15.66:0/2789680352 shutdown_connections 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.425+0000 7f54dc09b700 1 -- 172.21.15.66:0/2789680352 wait complete. 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.425+0000 7f54dc09b700 1 Processor -- start 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.425+0000 7f54dc09b700 1 -- start start 2024-04-21T08:58:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.425+0000 7f54dc09b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.425+0000 7f54dc09b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54d410c940 con 0x7f54d41084e0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54d9e37700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54d9e37700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60278/0 (socket says 172.21.15.66:60278) 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54d9e37700 1 -- 172.21.15.66:0/1943207390 learned_addr learned my addr 172.21.15.66:0/1943207390 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54d9e37700 1 -- 172.21.15.66:0/1943207390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d419a5d0 con 0x7f54d41084e0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54d9e37700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d419a090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54d0006010 tx=0x7f54d0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54caffd700 1 -- 172.21.15.66:0/1943207390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54d0026070 con 0x7f54d41084e0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54caffd700 1 -- 172.21.15.66:0/1943207390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d000bba0 con 0x7f54d41084e0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d419a8c0 con 0x7f54d41084e0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54caffd700 1 -- 172.21.15.66:0/1943207390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54d0003910 con 0x7f54d41084e0 2024-04-21T08:58:55.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.426+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d419e960 con 0x7f54d41084e0 2024-04-21T08:58:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.427+0000 7f54caffd700 1 -- 172.21.15.66:0/1943207390 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f54d0029070 con 0x7f54d41084e0 2024-04-21T08:58:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.427+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d4103b90 con 0x7f54d41084e0 2024-04-21T08:58:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.427+0000 7f54caffd700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54c00396b0 0x7f54c003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.427+0000 7f54caffd700 1 -- 172.21.15.66:0/1943207390 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f54d001e070 con 0x7f54d41084e0 2024-04-21T08:58:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.427+0000 7f54d9636700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54c00396b0 0x7f54c003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:55.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.428+0000 7f54d9636700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54c00396b0 0x7f54c003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006fd0 tx=0x7f54c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:55.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.433+0000 7f54caffd700 1 -- 172.21.15.66:0/1943207390 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54d0013070 con 0x7f54d41084e0 2024-04-21T08:58:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:55.643+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}) -- 0x7f54d419b3f0 con 0x7f54c00396b0 2024-04-21T08:58:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:56 smithi066 ceph-mon[27953]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:58:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:57 smithi066 ceph-mon[27953]: Deploying cephadm binary to smithi083 2024-04-21T08:58:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:58:59 smithi066 ceph-mon[27953]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:58:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.984+0000 7f54caffd700 1 -- 172.21.15.66:0/1943207390 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f54d419b3f0 con 0x7f54c00396b0 2024-04-21T08:58:59.985 INFO:teuthology.orchestra.run.smithi066.stdout:Added host 'smithi083' with addr '172.21.15.83' 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54c00396b0 msgr2=0x7f54c003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54c00396b0 0x7f54c003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006fd0 tx=0x7f54c4006e40 comp rx=0 tx=0).stop 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 msgr2=0x7f54d419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d419a090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54d0006010 tx=0x7f54d0005780 comp rx=0 tx=0).stop 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 shutdown_connections 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54c00396b0 0x7f54c003bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 --2- 172.21.15.66:0/1943207390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54d41084e0 0x7f54d419a090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.986+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 >> 172.21.15.66:0/1943207390 conn(0x7f54d40fdf60 msgr2=0x7f54d410a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:59.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.987+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 shutdown_connections 2024-04-21T08:58:59.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:58:59.987+0000 7f54dc09b700 1 -- 172.21.15.66:0/1943207390 wait complete. 2024-04-21T08:59:00.609 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:59:00.855 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:59:01.296 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:00 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:01.296 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:00 smithi066 ceph-mon[27953]: Added host smithi083 2024-04-21T08:59:01.296 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:00 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:01.296 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:00 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:02.049 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:02 smithi066 ceph-mon[27953]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:02.049 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:02 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.417+0000 7fb58c67f700 1 -- 172.21.15.66:0/3122678915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 msgr2=0x7fb5841089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.417+0000 7fb58c67f700 1 --2- 172.21.15.66:0/3122678915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb5841089a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb580009910 tx=0x7fb580009c20 comp rx=0 tx=0).stop 2024-04-21T08:59:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.417+0000 7fb58c67f700 1 -- 172.21.15.66:0/3122678915 shutdown_connections 2024-04-21T08:59:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.417+0000 7fb58c67f700 1 --2- 172.21.15.66:0/3122678915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb5841089a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.417+0000 7fb58c67f700 1 -- 172.21.15.66:0/3122678915 >> 172.21.15.66:0/3122678915 conn(0x7fb5840fc5f0 msgr2=0x7fb5840fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.417+0000 7fb58c67f700 1 -- 172.21.15.66:0/3122678915 shutdown_connections 2024-04-21T08:59:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.417+0000 7fb58c67f700 1 -- 172.21.15.66:0/3122678915 wait complete. 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58c67f700 1 Processor -- start 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58c67f700 1 -- start start 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58c67f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb584101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58c67f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb58410c9a0 con 0x7fb5841085c0 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58a41b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb584101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58a41b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb584101a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60290/0 (socket says 172.21.15.66:60290) 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58a41b700 1 -- 172.21.15.66:0/887760742 learned_addr learned my addr 172.21.15.66:0/887760742 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.418+0000 7fb58a41b700 1 -- 172.21.15.66:0/887760742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb584103980 con 0x7fb5841085c0 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.419+0000 7fb58a41b700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb584101a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5800056d0 tx=0x7fb5800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.419+0000 7fb57b7fe700 1 -- 172.21.15.66:0/887760742 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb580027070 con 0x7fb5841085c0 2024-04-21T08:59:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.419+0000 7fb57b7fe700 1 -- 172.21.15.66:0/887760742 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb58000be10 con 0x7fb5841085c0 2024-04-21T08:59:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.419+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb584101fd0 con 0x7fb5841085c0 2024-04-21T08:59:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.419+0000 7fb57b7fe700 1 -- 172.21.15.66:0/887760742 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb580022d20 con 0x7fb5841085c0 2024-04-21T08:59:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.419+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb584102490 con 0x7fb5841085c0 2024-04-21T08:59:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.420+0000 7fb57b7fe700 1 -- 172.21.15.66:0/887760742 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb58002a070 con 0x7fb5841085c0 2024-04-21T08:59:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.420+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5840640a0 con 0x7fb5841085c0 2024-04-21T08:59:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.420+0000 7fb57b7fe700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb570039650 0x7fb57003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.420+0000 7fb57b7fe700 1 -- 172.21.15.66:0/887760742 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb58001e030 con 0x7fb5841085c0 2024-04-21T08:59:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.420+0000 7fb589c1a700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb570039650 0x7fb57003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.420+0000 7fb589c1a700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb570039650 0x7fb57003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb574006fd0 tx=0x7fb574006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:02.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.425+0000 7fb57b7fe700 1 -- 172.21.15.66:0/887760742 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb580013070 con 0x7fb5841085c0 2024-04-21T08:59:02.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.642+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5841026c0 con 0x7fb570039650 2024-04-21T08:59:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.643+0000 7fb57b7fe700 1 -- 172.21.15.66:0/887760742 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fb5841026c0 con 0x7fb570039650 2024-04-21T08:59:02.643 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:59:02.643 INFO:teuthology.orchestra.run.smithi066.stdout:[{"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}] 2024-04-21T08:59:02.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb570039650 msgr2=0x7fb57003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb570039650 0x7fb57003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb574006fd0 tx=0x7fb574006e40 comp rx=0 tx=0).stop 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 msgr2=0x7fb584101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb584101a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5800056d0 tx=0x7fb5800057b0 comp rx=0 tx=0).stop 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 shutdown_connections 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb570039650 0x7fb57003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 --2- 172.21.15.66:0/887760742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5841085c0 0x7fb584101a90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 >> 172.21.15.66:0/887760742 conn(0x7fb5840fc5f0 msgr2=0x7fb5840fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 shutdown_connections 2024-04-21T08:59:02.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:02.645+0000 7fb58c67f700 1 -- 172.21.15.66:0/887760742 wait complete. 2024-04-21T08:59:03.220 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-04-21T08:59:03.220 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:59:03.220 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:59:03.239 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:59:03.239 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:59:03.297 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-04-21T08:59:03.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch host add smithi186 2024-04-21T08:59:03.540 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:59:03.858 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:03 smithi066 ceph-mon[27953]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:03.858 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:03 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:03.858 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:03 smithi066 ceph-mon[27953]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:59:04.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.919+0000 7f4ad495f700 1 -- 172.21.15.66:0/3302341992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 msgr2=0x7f4ad01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:04.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.919+0000 7f4ad495f700 1 --2- 172.21.15.66:0/3302341992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad01019d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009910 tx=0x7f4ac0009c20 comp rx=0 tx=0).stop 2024-04-21T08:59:04.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.919+0000 7f4ad495f700 1 -- 172.21.15.66:0/3302341992 shutdown_connections 2024-04-21T08:59:04.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.919+0000 7f4ad495f700 1 --2- 172.21.15.66:0/3302341992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad01019d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:04.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.919+0000 7f4ad495f700 1 -- 172.21.15.66:0/3302341992 >> 172.21.15.66:0/3302341992 conn(0x7f4ad00fcea0 msgr2=0x7f4ad00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:04.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.919+0000 7f4ad495f700 1 -- 172.21.15.66:0/3302341992 shutdown_connections 2024-04-21T08:59:04.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.919+0000 7f4ad495f700 1 -- 172.21.15.66:0/3302341992 wait complete. 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.920+0000 7f4ad495f700 1 Processor -- start 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.920+0000 7f4ad495f700 1 -- start start 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.920+0000 7f4ad495f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.920+0000 7f4ad495f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010b870 con 0x7f4ad01015f0 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4ace59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4ace59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad0197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51792/0 (socket says 172.21.15.66:51792) 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4ace59c700 1 -- 172.21.15.66:0/3905210324 learned_addr learned my addr 172.21.15.66:0/3905210324 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4ace59c700 1 -- 172.21.15.66:0/3905210324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad0198420 con 0x7f4ad01015f0 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4ace59c700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad0197ee0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00056d0 tx=0x7f4ac00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4abf7fe700 1 -- 172.21.15.66:0/3905210324 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac0026070 con 0x7f4ad01015f0 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4abf7fe700 1 -- 172.21.15.66:0/3905210324 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac000bba0 con 0x7f4ad01015f0 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad0198710 con 0x7f4ad01015f0 2024-04-21T08:59:04.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4abf7fe700 1 -- 172.21.15.66:0/3905210324 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac0003920 con 0x7f4ad01015f0 2024-04-21T08:59:04.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.921+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad019c7b0 con 0x7f4ad01015f0 2024-04-21T08:59:04.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.922+0000 7f4abf7fe700 1 -- 172.21.15.66:0/3905210324 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4ac0029070 con 0x7f4ad01015f0 2024-04-21T08:59:04.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.922+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad0102ad0 con 0x7f4ad01015f0 2024-04-21T08:59:04.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.922+0000 7f4abf7fe700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4ab80396b0 0x7f4ab803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:04.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.923+0000 7f4abf7fe700 1 -- 172.21.15.66:0/3905210324 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4ac001e070 con 0x7f4ad01015f0 2024-04-21T08:59:04.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.923+0000 7f4acdd9b700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4ab80396b0 0x7f4ab803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:04.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.923+0000 7f4acdd9b700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4ab80396b0 0x7f4ab803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4006fd0 tx=0x7f4ac4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:04.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:04.928+0000 7f4abf7fe700 1 -- 172.21.15.66:0/3905210324 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ac0013070 con 0x7f4ad01015f0 2024-04-21T08:59:05.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:05.139+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) -- 0x7f4ad0199240 con 0x7f4ab80396b0 2024-04-21T08:59:05.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:05 smithi066 ceph-mon[27953]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:07.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: Deploying cephadm binary to smithi186 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:07.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:07 smithi066 ceph-mon[27953]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-21T08:59:08.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:08 smithi066 ceph-mon[27953]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:08.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:08 smithi066 ceph-mon[27953]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:09.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.375+0000 7f4abf7fe700 1 -- 172.21.15.66:0/3905210324 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4ad0199240 con 0x7f4ab80396b0 2024-04-21T08:59:09.376 INFO:teuthology.orchestra.run.smithi066.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4ab80396b0 msgr2=0x7f4ab803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4ab80396b0 0x7f4ab803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4006fd0 tx=0x7f4ac4006e40 comp rx=0 tx=0).stop 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 msgr2=0x7f4ad0197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad0197ee0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00056d0 tx=0x7f4ac00057b0 comp rx=0 tx=0).stop 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 shutdown_connections 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4ab80396b0 0x7f4ab803bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 --2- 172.21.15.66:0/3905210324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ad01015f0 0x7f4ad0197ee0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:09.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 >> 172.21.15.66:0/3905210324 conn(0x7f4ad00fcea0 msgr2=0x7f4ad006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:09.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.377+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 shutdown_connections 2024-04-21T08:59:09.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:09.378+0000 7f4ad495f700 1 -- 172.21.15.66:0/3905210324 wait complete. 2024-04-21T08:59:09.930 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:59:10.170 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:10.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:11.331 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:11 smithi066 ceph-mon[27953]: Added host smithi186 2024-04-21T08:59:11.331 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:11 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:11.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.873+0000 7fba81576700 1 -- 172.21.15.66:0/2757316746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 msgr2=0x7fba7c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:11.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.873+0000 7fba81576700 1 --2- 172.21.15.66:0/2757316746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c108930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009910 tx=0x7fba6c009c20 comp rx=0 tx=0).stop 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.873+0000 7fba81576700 1 -- 172.21.15.66:0/2757316746 shutdown_connections 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.873+0000 7fba81576700 1 --2- 172.21.15.66:0/2757316746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c108930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.873+0000 7fba81576700 1 -- 172.21.15.66:0/2757316746 >> 172.21.15.66:0/2757316746 conn(0x7fba7c0fdf70 msgr2=0x7fba7c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.873+0000 7fba81576700 1 -- 172.21.15.66:0/2757316746 shutdown_connections 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.873+0000 7fba81576700 1 -- 172.21.15.66:0/2757316746 wait complete. 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba81576700 1 Processor -- start 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba81576700 1 -- start start 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba81576700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba81576700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c10c9c0 con 0x7fba7c108550 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba7affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:11.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba7affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51814/0 (socket says 172.21.15.66:51814) 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba7affd700 1 -- 172.21.15.66:0/1501171329 learned_addr learned my addr 172.21.15.66:0/1501171329 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.874+0000 7fba7affd700 1 -- 172.21.15.66:0/1501171329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c076be0 con 0x7fba7c108550 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.875+0000 7fba7affd700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c074d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0056d0 tx=0x7fba6c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.875+0000 7fba63fff700 1 -- 172.21.15.66:0/1501171329 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c026070 con 0x7fba7c108550 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.875+0000 7fba63fff700 1 -- 172.21.15.66:0/1501171329 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6c00bc00 con 0x7fba7c108550 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.875+0000 7fba63fff700 1 -- 172.21.15.66:0/1501171329 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c003920 con 0x7fba7c108550 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.875+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c075280 con 0x7fba7c108550 2024-04-21T08:59:11.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.875+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c075740 con 0x7fba7c108550 2024-04-21T08:59:11.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.876+0000 7fba63fff700 1 -- 172.21.15.66:0/1501171329 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fba6c029070 con 0x7fba7c108550 2024-04-21T08:59:11.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.876+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c103bc0 con 0x7fba7c108550 2024-04-21T08:59:11.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.876+0000 7fba63fff700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fba640396b0 0x7fba6403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:11.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.876+0000 7fba63fff700 1 -- 172.21.15.66:0/1501171329 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fba6c01e070 con 0x7fba7c108550 2024-04-21T08:59:11.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.876+0000 7fba7a7fc700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fba640396b0 0x7fba6403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:11.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.876+0000 7fba7a7fc700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fba640396b0 0x7fba6403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba70006fd0 tx=0x7fba70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:11.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:11.881+0000 7fba63fff700 1 -- 172.21.15.66:0/1501171329 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba6c013070 con 0x7fba7c108550 2024-04-21T08:59:12.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.113+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba7c0762a0 con 0x7fba640396b0 2024-04-21T08:59:12.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.114+0000 7fba63fff700 1 -- 172.21.15.66:0/1501171329 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fba7c0762a0 con 0x7fba640396b0 2024-04-21T08:59:12.115 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T08:59:12.115 INFO:teuthology.orchestra.run.smithi066.stdout:[{"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-04-21T08:59:12.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.116+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fba640396b0 msgr2=0x7fba6403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:12.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fba640396b0 0x7fba6403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba70006fd0 tx=0x7fba70006e40 comp rx=0 tx=0).stop 2024-04-21T08:59:12.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 msgr2=0x7fba7c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:12.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c074d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0056d0 tx=0x7fba6c0057b0 comp rx=0 tx=0).stop 2024-04-21T08:59:12.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 shutdown_connections 2024-04-21T08:59:12.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fba640396b0 0x7fba6403bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:12.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 --2- 172.21.15.66:0/1501171329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba7c108550 0x7fba7c074d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:12.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 >> 172.21.15.66:0/1501171329 conn(0x7fba7c0fdf70 msgr2=0x7fba7c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:12.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 shutdown_connections 2024-04-21T08:59:12.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:12.117+0000 7fba81576700 1 -- 172.21.15.66:0/1501171329 wait complete. 2024-04-21T08:59:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:12 smithi066 ceph-mon[27953]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:12 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:12.664 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-21T08:59:12.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-21T08:59:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:59:14.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:14 smithi066 ceph-mon[27953]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:59:14.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:14 smithi066 ceph-mon[27953]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:14.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.252+0000 7f74df1b6700 1 -- 172.21.15.66:0/1035502416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 msgr2=0x7f74d81039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:14.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.252+0000 7f74df1b6700 1 --2- 172.21.15.66:0/1035502416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d81039f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009910 tx=0x7f74cc009c20 comp rx=0 tx=0).stop 2024-04-21T08:59:14.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.252+0000 7f74df1b6700 1 -- 172.21.15.66:0/1035502416 shutdown_connections 2024-04-21T08:59:14.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.252+0000 7f74df1b6700 1 --2- 172.21.15.66:0/1035502416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d81039f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:14.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.252+0000 7f74df1b6700 1 -- 172.21.15.66:0/1035502416 >> 172.21.15.66:0/1035502416 conn(0x7f74d80fce60 msgr2=0x7f74d80ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:14.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.252+0000 7f74df1b6700 1 -- 172.21.15.66:0/1035502416 shutdown_connections 2024-04-21T08:59:14.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.252+0000 7f74df1b6700 1 -- 172.21.15.66:0/1035502416 wait complete. 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.253+0000 7f74df1b6700 1 Processor -- start 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.253+0000 7f74df1b6700 1 -- start start 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.253+0000 7f74df1b6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.253+0000 7f74df1b6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810c9b0 con 0x7f74d8103610 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.253+0000 7f74dcf52700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.253+0000 7f74dcf52700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51834/0 (socket says 172.21.15.66:51834) 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.253+0000 7f74dcf52700 1 -- 172.21.15.66:0/781161763 learned_addr learned my addr 172.21.15.66:0/781161763 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.254+0000 7f74dcf52700 1 -- 172.21.15.66:0/781161763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d819a6a0 con 0x7f74d8103610 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.254+0000 7f74dcf52700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d819a160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0056d0 tx=0x7f74cc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.254+0000 7f74d5ffb700 1 -- 172.21.15.66:0/781161763 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74cc026070 con 0x7f74d8103610 2024-04-21T08:59:14.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.254+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d819a930 con 0x7f74d8103610 2024-04-21T08:59:14.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.254+0000 7f74d5ffb700 1 -- 172.21.15.66:0/781161763 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc004370 con 0x7f74d8103610 2024-04-21T08:59:14.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.254+0000 7f74d5ffb700 1 -- 172.21.15.66:0/781161763 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74cc0039b0 con 0x7f74d8103610 2024-04-21T08:59:14.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.254+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d819ad90 con 0x7f74d8103610 2024-04-21T08:59:14.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.255+0000 7f74d5ffb700 1 -- 172.21.15.66:0/781161763 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f74cc029070 con 0x7f74d8103610 2024-04-21T08:59:14.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.255+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d806b1b0 con 0x7f74d8103610 2024-04-21T08:59:14.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.255+0000 7f74d5ffb700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74c8039650 0x7f74c803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:14.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.255+0000 7f74d5ffb700 1 -- 172.21.15.66:0/781161763 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f74cc01e070 con 0x7f74d8103610 2024-04-21T08:59:14.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.255+0000 7f74d7fff700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74c8039650 0x7f74c803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:14.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.256+0000 7f74d7fff700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74c8039650 0x7f74c803bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006fd0 tx=0x7f74c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:14.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.260+0000 7f74d5ffb700 1 -- 172.21.15.66:0/781161763 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74cc013070 con 0x7f74d8103610 2024-04-21T08:59:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:14.476+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f74d819b470 con 0x7f74d8103610 2024-04-21T08:59:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.043+0000 7f74d5ffb700 1 -- 172.21.15.66:0/781161763 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f74cc030090 con 0x7f74d8103610 2024-04-21T08:59:15.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74c8039650 msgr2=0x7f74c803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:15.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74c8039650 0x7f74c803bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006fd0 tx=0x7f74c4006e40 comp rx=0 tx=0).stop 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 msgr2=0x7f74d819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d819a160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0056d0 tx=0x7f74cc0057b0 comp rx=0 tx=0).stop 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 shutdown_connections 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74c8039650 0x7f74c803bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 --2- 172.21.15.66:0/781161763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74d8103610 0x7f74d819a160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 >> 172.21.15.66:0/781161763 conn(0x7f74d80fce60 msgr2=0x7f74d81087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 shutdown_connections 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:15.046+0000 7f74df1b6700 1 -- 172.21.15.66:0/781161763 wait complete. 2024-04-21T08:59:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/781161763' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:15.777 INFO:tasks.cephadm:Adding mon.a on smithi066 2024-04-21T08:59:15.777 INFO:tasks.cephadm:Adding mon.b on smithi083 2024-04-21T08:59:15.777 INFO:tasks.cephadm:Adding mon.c on smithi186 2024-04-21T08:59:15.777 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch apply mon '3;smithi066:172.21.15.66=a;smithi083:172.21.15.83=b;smithi186:172.21.15.186=c' 2024-04-21T08:59:16.003 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:16 smithi066 ceph-mon[27953]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:16 smithi066 ceph-mon[27953]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-21T08:59:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/781161763' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T08:59:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:16 smithi066 ceph-mon[27953]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:17.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:17 smithi066 ceph-mon[27953]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:17.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:17 smithi066 ceph-mon[27953]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:59:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:17 smithi066 ceph-mon[27953]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:59:17.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 -- 172.21.15.186:0/453087031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 msgr2=0x7fe4841019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:17.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 --2- 172.21.15.186:0/453087031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe4841019d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe474009910 tx=0x7fe474009c20 comp rx=0 tx=0).stop 2024-04-21T08:59:17.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 -- 172.21.15.186:0/453087031 shutdown_connections 2024-04-21T08:59:17.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 --2- 172.21.15.186:0/453087031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe4841019d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:17.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 -- 172.21.15.186:0/453087031 >> 172.21.15.186:0/453087031 conn(0x7fe484076ea0 msgr2=0x7fe4840772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:17.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 -- 172.21.15.186:0/453087031 shutdown_connections 2024-04-21T08:59:17.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 -- 172.21.15.186:0/453087031 wait complete. 2024-04-21T08:59:17.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.408+0000 7fe48a1eb700 1 Processor -- start 2024-04-21T08:59:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.409+0000 7fe48a1eb700 1 -- start start 2024-04-21T08:59:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.409+0000 7fe48a1eb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe484197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.409+0000 7fe48a1eb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410b820 con 0x7fe4841015f0 2024-04-21T08:59:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.409+0000 7fe4837fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe484197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.409+0000 7fe4837fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe484197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.186:58558/0 (socket says 172.21.15.186:58558) 2024-04-21T08:59:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.409+0000 7fe4837fe700 1 -- 172.21.15.186:0/761090870 learned_addr learned my addr 172.21.15.186:0/761090870 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T08:59:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.410+0000 7fe4837fe700 1 -- 172.21.15.186:0/761090870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4841983c0 con 0x7fe4841015f0 2024-04-21T08:59:17.412 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.410+0000 7fe4837fe700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe484197e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4740056d0 tx=0x7fe4740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:17.412 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.410+0000 7fe4817fa700 1 -- 172.21.15.186:0/761090870 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474026070 con 0x7fe4841015f0 2024-04-21T08:59:17.412 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.410+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4841986b0 con 0x7fe4841015f0 2024-04-21T08:59:17.412 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.410+0000 7fe4817fa700 1 -- 172.21.15.186:0/761090870 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47400bba0 con 0x7fe4841015f0 2024-04-21T08:59:17.412 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.410+0000 7fe4817fa700 1 -- 172.21.15.186:0/761090870 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474003920 con 0x7fe4841015f0 2024-04-21T08:59:17.412 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.410+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48419c750 con 0x7fe4841015f0 2024-04-21T08:59:17.413 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.411+0000 7fe4817fa700 1 -- 172.21.15.186:0/761090870 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe474029070 con 0x7fe4841015f0 2024-04-21T08:59:17.413 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.411+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe484102a80 con 0x7fe4841015f0 2024-04-21T08:59:17.413 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.411+0000 7fe4817fa700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe470039650 0x7fe47003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:17.413 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.411+0000 7fe4817fa700 1 -- 172.21.15.186:0/761090870 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe47401e070 con 0x7fe4841015f0 2024-04-21T08:59:17.413 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.412+0000 7fe47bfff700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe470039650 0x7fe47003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:17.413 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.412+0000 7fe47bfff700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe470039650 0x7fe47003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe46c006fd0 tx=0x7fe46c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:17.418 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.417+0000 7fe4817fa700 1 -- 172.21.15.186:0/761090870 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe474013070 con 0x7fe4841015f0 2024-04-21T08:59:17.626 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.625+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi083:172.21.15.83=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) -- 0x7fe4841991e0 con 0x7fe470039650 2024-04-21T08:59:17.632 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.632+0000 7fe4817fa700 1 -- 172.21.15.186:0/761090870 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe4841991e0 con 0x7fe470039650 2024-04-21T08:59:17.632 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-04-21T08:59:17.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe470039650 msgr2=0x7fe47003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe470039650 0x7fe47003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe46c006fd0 tx=0x7fe46c006e40 comp rx=0 tx=0).stop 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 msgr2=0x7fe484197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe484197e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe4740056d0 tx=0x7fe4740057b0 comp rx=0 tx=0).stop 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 shutdown_connections 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe470039650 0x7fe47003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 --2- 172.21.15.186:0/761090870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4841015f0 0x7fe484197e80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 >> 172.21.15.186:0/761090870 conn(0x7fe484076ea0 msgr2=0x7fe484069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 shutdown_connections 2024-04-21T08:59:17.635 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:17.634+0000 7fe48a1eb700 1 -- 172.21.15.186:0/761090870 wait complete. 2024-04-21T08:59:18.215 DEBUG:teuthology.orchestra.run.smithi083:mon.b> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.b.service 2024-04-21T08:59:18.218 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.c.service 2024-04-21T08:59:18.220 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:59:18.220 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:59:18.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:18.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:18.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:18.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:18.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:18.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:18.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:18.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:19.040 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:19.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:19 smithi066 ceph-mon[27953]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi083:172.21.15.83=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:19.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:19 smithi066 ceph-mon[27953]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi083:172.21.15.83=b;smithi186:172.21.15.186=c;count:3 2024-04-21T08:59:19.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:19 smithi066 ceph-mon[27953]: Deploying daemon mon.c on smithi186 2024-04-21T08:59:20.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:20 smithi066 ceph-mon[27953]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:21.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.461+0000 7f7ce1210700 1 -- 172.21.15.186:0/1893280046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 msgr2=0x7f7cdc102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:21.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.461+0000 7f7ce1210700 1 --2- 172.21.15.186:0/1893280046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc102280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009910 tx=0x7f7ccc009c20 comp rx=0 tx=0).stop 2024-04-21T08:59:21.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.461+0000 7f7ce1210700 1 -- 172.21.15.186:0/1893280046 shutdown_connections 2024-04-21T08:59:21.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.461+0000 7f7ce1210700 1 --2- 172.21.15.186:0/1893280046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc102280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:21.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.461+0000 7f7ce1210700 1 -- 172.21.15.186:0/1893280046 >> 172.21.15.186:0/1893280046 conn(0x7f7cdc0788d0 msgr2=0x7f7cdc078ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:21.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.461+0000 7f7ce1210700 1 -- 172.21.15.186:0/1893280046 shutdown_connections 2024-04-21T08:59:21.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.461+0000 7f7ce1210700 1 -- 172.21.15.186:0/1893280046 wait complete. 2024-04-21T08:59:21.463 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.462+0000 7f7ce1210700 1 Processor -- start 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.462+0000 7f7ce1210700 1 -- start start 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.462+0000 7f7ce1210700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.462+0000 7f7ce1210700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc10b8b0 con 0x7f7cdc101ea0 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.463+0000 7f7cdad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.463+0000 7f7cdad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.186:58576/0 (socket says 172.21.15.186:58576) 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.463+0000 7f7cdad9d700 1 -- 172.21.15.186:0/3499593903 learned_addr learned my addr 172.21.15.186:0/3499593903 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.463+0000 7f7cdad9d700 1 -- 172.21.15.186:0/3499593903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cdc101930 con 0x7f7cdc101ea0 2024-04-21T08:59:21.464 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.463+0000 7f7cdad9d700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc196ad0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0056d0 tx=0x7f7ccc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:21.465 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.463+0000 7f7cd8d99700 1 -- 172.21.15.186:0/3499593903 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ccc026070 con 0x7f7cdc101ea0 2024-04-21T08:59:21.465 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.464+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cdc197010 con 0x7f7cdc101ea0 2024-04-21T08:59:21.465 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.464+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cdc1974d0 con 0x7f7cdc101ea0 2024-04-21T08:59:21.465 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.464+0000 7f7cd8d99700 1 -- 172.21.15.186:0/3499593903 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc00bba0 con 0x7f7cdc101ea0 2024-04-21T08:59:21.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.464+0000 7f7cd8d99700 1 -- 172.21.15.186:0/3499593903 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ccc0037f0 con 0x7f7cdc101ea0 2024-04-21T08:59:21.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.464+0000 7f7cd8d99700 1 -- 172.21.15.186:0/3499593903 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7ccc029070 con 0x7f7cdc101ea0 2024-04-21T08:59:21.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.464+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cbc005330 con 0x7f7cdc101ea0 2024-04-21T08:59:21.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.465+0000 7f7cd8d99700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7cc80396b0 0x7f7cc803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:21.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.465+0000 7f7cd8d99700 1 -- 172.21.15.186:0/3499593903 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ccc01e070 con 0x7f7cdc101ea0 2024-04-21T08:59:21.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.465+0000 7f7cd259c700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7cc80396b0 0x7f7cc803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:21.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.465+0000 7f7cd259c700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7cc80396b0 0x7f7cc803bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006fd0 tx=0x7f7cc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:21.471 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.470+0000 7f7cd8d99700 1 -- 172.21.15.186:0/3499593903 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ccc013070 con 0x7f7cdc101ea0 2024-04-21T08:59:21.755 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.754+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7cbc005620 con 0x7f7cdc101ea0 2024-04-21T08:59:21.756 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.755+0000 7f7cd8d99700 1 -- 172.21.15.186:0/3499593903 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f7ccc015580 con 0x7f7cdc101ea0 2024-04-21T08:59:21.756 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:59:21.756 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","modified":"2024-04-21T08:57:31.934215Z","created":"2024-04-21T08:57:31.934215Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-21T08:59:21.758 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7cc80396b0 msgr2=0x7f7cc803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:21.758 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7cc80396b0 0x7f7cc803bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006fd0 tx=0x7f7cc4006e40 comp rx=0 tx=0).stop 2024-04-21T08:59:21.758 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 msgr2=0x7f7cdc196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:21.758 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc196ad0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0056d0 tx=0x7f7ccc0057b0 comp rx=0 tx=0).stop 2024-04-21T08:59:21.759 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 shutdown_connections 2024-04-21T08:59:21.759 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7cc80396b0 0x7f7cc803bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:21.759 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 --2- 172.21.15.186:0/3499593903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cdc101ea0 0x7f7cdc196ad0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:21.759 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 >> 172.21.15.186:0/3499593903 conn(0x7f7cdc0788d0 msgr2=0x7f7cdc102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:21.759 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 shutdown_connections 2024-04-21T08:59:21.759 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:21.757+0000 7f7ce1210700 1 -- 172.21.15.186:0/3499593903 wait complete. 2024-04-21T08:59:21.759 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-21T08:59:22.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:22 smithi066 ceph-mon[27953]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/3499593903' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:59:23.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:23 smithi186 systemd[1]: Started Ceph mon.c for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T08:59:23.657 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:59:23.657 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:59:23.891 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:59:28.478 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.476+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/1771155707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 msgr2=0x7f8bd4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:28.478 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.477+0000 7f8bf1fdd700 1 --2- 172.21.15.186:0/1771155707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bd4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8bec101380 tx=0x7f8bdc009e80 comp rx=0 tx=0).stop 2024-04-21T08:59:28.478 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.477+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/1771155707 shutdown_connections 2024-04-21T08:59:28.478 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.477+0000 7f8bf1fdd700 1 --2- 172.21.15.186:0/1771155707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bd4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:28.478 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.477+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/1771155707 >> 172.21.15.186:0/1771155707 conn(0x7f8bec0fcf40 msgr2=0x7f8bec0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:28.478 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.477+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/1771155707 shutdown_connections 2024-04-21T08:59:28.478 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.477+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/1771155707 wait complete. 2024-04-21T08:59:28.480 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.478+0000 7f8bf1fdd700 1 Processor -- start 2024-04-21T08:59:28.480 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.478+0000 7f8bf1fdd700 1 -- start start 2024-04-21T08:59:28.480 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.478+0000 7f8bf1fdd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bec198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:28.480 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.478+0000 7f8bf1fdd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 0x7f8bec19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.478+0000 7f8bf1fdd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1093a0 con 0x7f8bd40035c0 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.478+0000 7f8bf1fdd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec109220 con 0x7f8bec198570 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 0x7f8bec19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bec198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 0x7f8bec19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:40898/0 (socket says 172.21.15.186:40898) 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bec198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.186:57374/0 (socket says 172.21.15.186:57374) 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beaffd700 1 -- 172.21.15.186:0/2387303753 learned_addr learned my addr 172.21.15.186:0/2387303753 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beaffd700 1 -- 172.21.15.186:0/2387303753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 msgr2=0x7f8bec19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beaffd700 1 -- 172.21.15.186:0/2387303753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 msgr2=0x7f8bec19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beaffd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 0x7f8bec19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beaffd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 0x7f8bec19c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beb7fe700 1 -- 172.21.15.186:0/2387303753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 msgr2=0x7f8bec19c900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:59:28.481 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beb7fe700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 0x7f8bec19c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:28.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beb7fe700 1 -- 172.21.15.186:0/2387303753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec198bb0 con 0x7f8bd40035c0 2024-04-21T08:59:28.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8beb7fe700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bec198030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8bec101380 tx=0x7f8bdc017750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:28.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.479+0000 7f8be8ff9700 1 -- 172.21.15.186:0/2387303753 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8bdc02a470 con 0x7f8bd40035c0 2024-04-21T08:59:28.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.480+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec19ce40 con 0x7f8bd40035c0 2024-04-21T08:59:28.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.480+0000 7f8be8ff9700 1 -- 172.21.15.186:0/2387303753 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc025070 con 0x7f8bd40035c0 2024-04-21T08:59:28.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.480+0000 7f8be8ff9700 1 -- 172.21.15.186:0/2387303753 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8bdc0219c0 con 0x7f8bd40035c0 2024-04-21T08:59:28.482 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.480+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec19d3a0 con 0x7f8bd40035c0 2024-04-21T08:59:28.483 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.481+0000 7f8be8ff9700 1 -- 172.21.15.186:0/2387303753 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8bdc01e070 con 0x7f8bd40035c0 2024-04-21T08:59:28.483 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.481+0000 7f8be8ff9700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bd8039410 0x7f8bd803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:28.483 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.481+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bec102b90 con 0x7f8bd40035c0 2024-04-21T08:59:28.483 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.481+0000 7f8be8ff9700 1 -- 172.21.15.186:0/2387303753 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8bdc01c070 con 0x7f8bd40035c0 2024-04-21T08:59:28.483 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.481+0000 7f8beaffd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bd8039410 0x7f8bd803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:28.483 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.482+0000 7f8beaffd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bd8039410 0x7f8bd803b8d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009180 tx=0x7f8be0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:28.487 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.487+0000 7f8be8ff9700 1 -- 172.21.15.186:0/2387303753 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bdc029c90 con 0x7f8bd40035c0 2024-04-21T08:59:28.770 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.770+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8bec19d650 con 0x7f8bd40035c0 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: mon.a calling monitor election 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: mon.c calling monitor election 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: monmap epoch 2 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: last_changed 2024-04-21T08:59:23.443828+0000 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: min_mon_release 19 (squid) 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: election_strategy: 1 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: fsmap 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: mgrmap e13: x(active, since 50s) 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: overall HEALTH_OK 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:28.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:28.773 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:28.773 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:28.773 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:28.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.770+0000 7f8be8ff9700 1 -- 172.21.15.186:0/2387303753 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8bdc039090 con 0x7f8bd40035c0 2024-04-21T08:59:28.773 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:59:28.773 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","modified":"2024-04-21T08:59:23.443828Z","created":"2024-04-21T08:57:31.934215Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:59:28.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bd8039410 msgr2=0x7f8bd803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:28.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bd8039410 0x7f8bd803b8d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009180 tx=0x7f8be0008040 comp rx=0 tx=0).stop 2024-04-21T08:59:28.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 msgr2=0x7f8bec198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:28.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bec198030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8bec101380 tx=0x7f8bdc017750 comp rx=0 tx=0).stop 2024-04-21T08:59:28.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 shutdown_connections 2024-04-21T08:59:28.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8bd8039410 0x7f8bd803b8d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:28.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bd40035c0 0x7f8bec198030 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:28.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 --2- 172.21.15.186:0/2387303753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec198570 0x7f8bec19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:28.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 >> 172.21.15.186:0/2387303753 conn(0x7f8bec0fcf40 msgr2=0x7f8bec06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:28.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.772+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 shutdown_connections 2024-04-21T08:59:28.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:28.773+0000 7f8bf1fdd700 1 -- 172.21.15.186:0/2387303753 wait complete. 2024-04-21T08:59:28.775 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: mon.a calling monitor election 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: mon.c calling monitor election 2024-04-21T08:59:28.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: monmap epoch 2 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: last_changed 2024-04-21T08:59:23.443828+0000 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: min_mon_release 19 (squid) 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: election_strategy: 1 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: fsmap 2024-04-21T08:59:28.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:28.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: mgrmap e13: x(active, since 50s) 2024-04-21T08:59:28.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: overall HEALTH_OK 2024-04-21T08:59:28.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:28.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:28.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:28.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:28.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:29.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:29 smithi186 ceph-mon[31505]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:29.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:29 smithi186 ceph-mon[31505]: Deploying daemon mon.b on smithi083 2024-04-21T08:59:29.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:29 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/2387303753' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:59:29.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:29.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:29 smithi066 ceph-mon[27953]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:29.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:29 smithi066 ceph-mon[27953]: Deploying daemon mon.b on smithi083 2024-04-21T08:59:29.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:29 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/2387303753' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:59:29.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:30.385 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:59:30.385 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:59:30.629 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:59:31.691 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:31 smithi186 ceph-mon[31505]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:31.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:31 smithi066 ceph-mon[27953]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:32.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 -- 172.21.15.186:0/1198635509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628108640 msgr2=0x7f7628108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:32.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 --2- 172.21.15.186:0/1198635509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628108640 0x7f7628108a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7618009910 tx=0x7f7618009c20 comp rx=0 tx=0).stop 2024-04-21T08:59:32.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 -- 172.21.15.186:0/1198635509 shutdown_connections 2024-04-21T08:59:32.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 --2- 172.21.15.186:0/1198635509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628108640 0x7f7628108a20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:32.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 -- 172.21.15.186:0/1198635509 >> 172.21.15.186:0/1198635509 conn(0x7f76280fe060 msgr2=0x7f7628100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:32.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 -- 172.21.15.186:0/1198635509 shutdown_connections 2024-04-21T08:59:32.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 -- 172.21.15.186:0/1198635509 wait complete. 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.144+0000 7f762d914700 1 Processor -- start 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f762d914700 1 -- start start 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f762d914700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 0x7f7628074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f762d914700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628075280 0x7f7628075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f762d914700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f762810cab0 con 0x7f7628075280 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f762d914700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f762810c930 con 0x7f7628108640 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 0x7f7628074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 0x7f7628074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:40902/0 (socket says 172.21.15.186:40902) 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f7626ffd700 1 -- 172.21.15.186:0/893996227 learned_addr learned my addr 172.21.15.186:0/893996227 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f76267fc700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628075280 0x7f7628075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:32.147 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f7626ffd700 1 -- 172.21.15.186:0/893996227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 msgr2=0x7f7628074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f7626ffd700 1 -- 172.21.15.186:0/893996227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 msgr2=0x7f7628074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f7626ffd700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 0x7f7628074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.145+0000 7f7626ffd700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 0x7f7628074d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f76267fc700 1 -- 172.21.15.186:0/893996227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 msgr2=0x7f7628074d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f76267fc700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 0x7f7628074d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f76267fc700 1 -- 172.21.15.186:0/893996227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7628076c60 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f76267fc700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628075280 0x7f7628075700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f761c00b4d0 tx=0x7f761c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f762c912700 1 -- 172.21.15.186:0/893996227 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f761c011630 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f762c912700 1 -- 172.21.15.186:0/893996227 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761c011c80 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76281a43d0 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f762c912700 1 -- 172.21.15.186:0/893996227 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f761c010da0 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.146+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281a4910 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.147+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7628103cb0 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.147+0000 7f762c912700 1 -- 172.21.15.186:0/893996227 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f761c018020 con 0x7f7628075280 2024-04-21T08:59:32.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.147+0000 7f762c912700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76100393f0 0x7f761003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:32.149 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.147+0000 7f762c912700 1 -- 172.21.15.186:0/893996227 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f761c04c1c0 con 0x7f7628075280 2024-04-21T08:59:32.149 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.147+0000 7f7626ffd700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76100393f0 0x7f761003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:32.149 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.148+0000 7f7626ffd700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76100393f0 0x7f761003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7618009750 tx=0x7f761801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:32.152 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.153+0000 7f762c912700 1 -- 172.21.15.186:0/893996227 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f761c01f020 con 0x7f7628075280 2024-04-21T08:59:32.428 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.429+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f76281a4cc0 con 0x7f7628075280 2024-04-21T08:59:32.429 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.430+0000 7f762c912700 1 -- 172.21.15.186:0/893996227 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f761c01a920 con 0x7f7628075280 2024-04-21T08:59:32.429 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:59:32.429 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","modified":"2024-04-21T08:59:23.443828Z","created":"2024-04-21T08:57:31.934215Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:59:32.431 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76100393f0 msgr2=0x7f761003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:32.431 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76100393f0 0x7f761003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7618009750 tx=0x7f761801a040 comp rx=0 tx=0).stop 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628075280 msgr2=0x7f7628075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628075280 0x7f7628075700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f761c00b4d0 tx=0x7f761c00b890 comp rx=0 tx=0).stop 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 shutdown_connections 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76100393f0 0x7f761003b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7628108640 0x7f7628074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 --2- 172.21.15.186:0/893996227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7628075280 0x7f7628075700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 >> 172.21.15.186:0/893996227 conn(0x7f76280fe060 msgr2=0x7f76280fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 shutdown_connections 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:32.432+0000 7f762d914700 1 -- 172.21.15.186:0/893996227 wait complete. 2024-04-21T08:59:32.432 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-04-21T08:59:34.049 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:59:34.049 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:59:34.293 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:59:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: mon.c calling monitor election 2024-04-21T08:59:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:59:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: mon.a calling monitor election 2024-04-21T08:59:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: monmap epoch 3 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: last_changed 2024-04-21T08:59:32.490234+0000 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: min_mon_release 19 (squid) 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: election_strategy: 1 2024-04-21T08:59:37.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: fsmap 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: mgrmap e13: x(active, since 59s) 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: overall HEALTH_OK 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:37.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: mon.c calling monitor election 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: mon.a calling monitor election 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: monmap epoch 3 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: last_changed 2024-04-21T08:59:32.490234+0000 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: min_mon_release 19 (squid) 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: election_strategy: 1 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: fsmap 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:37.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: mgrmap e13: x(active, since 59s) 2024-04-21T08:59:37.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: overall HEALTH_OK 2024-04-21T08:59:37.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:37.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:37.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:37.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:38.797 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.797+0000 7f9607f4f700 1 -- 172.21.15.186:0/1610963545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9600108620 msgr2=0x7f95ec005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:38.798 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.797+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1610963545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9600108620 0x7f95ec005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009bf0 tx=0x7f95fc009f00 comp rx=0 tx=0).stop 2024-04-21T08:59:38.798 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.798+0000 7f9607f4f700 1 -- 172.21.15.186:0/1610963545 shutdown_connections 2024-04-21T08:59:38.798 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.798+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1610963545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9600108620 0x7f95ec005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:38.798 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.798+0000 7f9607f4f700 1 -- 172.21.15.186:0/1610963545 >> 172.21.15.186:0/1610963545 conn(0x7f96000fe060 msgr2=0x7f9600100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:38.798 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:38 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:38.798 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:38 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:38.798 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:38 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:38.801 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.798+0000 7f9607f4f700 1 -- 172.21.15.186:0/1610963545 shutdown_connections 2024-04-21T08:59:38.801 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.798+0000 7f9607f4f700 1 -- 172.21.15.186:0/1610963545 wait complete. 2024-04-21T08:59:38.801 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.798+0000 7f9607f4f700 1 Processor -- start 2024-04-21T08:59:38.801 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.798+0000 7f9607f4f700 1 -- start start 2024-04-21T08:59:38.801 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9607f4f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 0x7f9600198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:38.801 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9607f4f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9600198570 0x7f960019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9607f4f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960019ce40 0x7f960019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9607f4f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f960010a450 con 0x7f960019ce40 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9607f4f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f960010a2d0 con 0x7f9600108620 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9607f4f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f960010a5d0 con 0x7f9600198570 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9600198570 0x7f960019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9600198570 0x7f960019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:52018/0 (socket says 172.21.15.186:52018) 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 -- 172.21.15.186:0/1950182929 learned_addr learned my addr 172.21.15.186:0/1950182929 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96064ec700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960019ce40 0x7f960019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9605ceb700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 0x7f9600198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9605ceb700 1 -- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 msgr2=0x7f9600198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9605ceb700 1 -- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 msgr2=0x7f9600198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9605ceb700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 0x7f9600198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:59:38.802 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f9605ceb700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 0x7f9600198030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 -- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 msgr2=0x7f9600198030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 0x7f9600198030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 -- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960019ce40 msgr2=0x7f960019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960019ce40 0x7f960019d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 -- 172.21.15.186:0/1950182929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96001a4bb0 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f96054ea700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9600198570 0x7f960019c900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b780 tx=0x7f95f000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.799+0000 7f95f6ffd700 1 -- 172.21.15.186:0/1950182929 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0004230 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.800+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96001a4de0 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.800+0000 7f95f6ffd700 1 -- 172.21.15.186:0/1950182929 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f00043d0 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.800+0000 7f95f6ffd700 1 -- 172.21.15.186:0/1950182929 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0011ad0 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.800+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001a5290 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.801+0000 7f95f6ffd700 1 -- 172.21.15.186:0/1950182929 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f95f0011c90 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.801+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96000640a0 con 0x7f9600198570 2024-04-21T08:59:38.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.801+0000 7f95f6ffd700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f95e4039ae0 0x7f95e403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:38.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.801+0000 7f95f6ffd700 1 -- 172.21.15.186:0/1950182929 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f95f004c480 con 0x7f9600198570 2024-04-21T08:59:38.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.801+0000 7f9605ceb700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f95e4039ae0 0x7f95e403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:38.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.802+0000 7f9605ceb700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f95e4039ae0 0x7f95e403bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007f50 tx=0x7f95fc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:38.805 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:38.806+0000 7f95f6ffd700 1 -- 172.21.15.186:0/1950182929 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95f0003b20 con 0x7f9600198570 2024-04-21T08:59:38.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:38.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:38.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:39.098 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.098+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f960002e140 con 0x7f9600198570 2024-04-21T08:59:39.099 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.099+0000 7f95f6ffd700 1 -- 172.21.15.186:0/1950182929 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f95f00036a0 con 0x7f9600198570 2024-04-21T08:59:39.099 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-21T08:59:39.099 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","modified":"2024-04-21T08:59:32.490234Z","created":"2024-04-21T08:57:31.934215Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:59:39.101 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f95e4039ae0 msgr2=0x7f95e403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:39.101 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f95e4039ae0 0x7f95e403bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007f50 tx=0x7f95fc00b040 comp rx=0 tx=0).stop 2024-04-21T08:59:39.101 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9600198570 msgr2=0x7f960019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:39.101 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9600198570 0x7f960019c900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b780 tx=0x7f95f000ba90 comp rx=0 tx=0).stop 2024-04-21T08:59:39.101 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 shutdown_connections 2024-04-21T08:59:39.101 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f95e4039ae0 0x7f95e403bfa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:39.101 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9600108620 0x7f9600198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:39.102 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960019ce40 0x7f960019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:39.102 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 --2- 172.21.15.186:0/1950182929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9600198570 0x7f960019c900 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:39.102 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 >> 172.21.15.186:0/1950182929 conn(0x7f96000fe060 msgr2=0x7f9600107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:39.102 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 shutdown_connections 2024-04-21T08:59:39.102 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:39.101+0000 7f9607f4f700 1 -- 172.21.15.186:0/1950182929 wait complete. 2024-04-21T08:59:39.102 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2024-04-21T08:59:39.849 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-21T08:59:39.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-21T08:59:39.862 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: mon.b calling monitor election 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: mon.b calling monitor election 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: mon.c calling monitor election 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: mon.a calling monitor election 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:59:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: monmap epoch 3 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: last_changed 2024-04-21T08:59:32.490234+0000 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: min_mon_release 19 (squid) 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: election_strategy: 1 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: fsmap 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: mgrmap e13: x(active, since 61s) 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: overall HEALTH_OK 2024-04-21T08:59:40.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:40.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:40 smithi186 ceph-mon[31505]: Reconfiguring daemon mon.a on smithi066 2024-04-21T08:59:40.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: mon.b calling monitor election 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: mon.b calling monitor election 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: mon.c calling monitor election 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: mon.a calling monitor election 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: monmap epoch 3 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: last_changed 2024-04-21T08:59:32.490234+0000 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: min_mon_release 19 (squid) 2024-04-21T08:59:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: election_strategy: 1 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: fsmap 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: mgrmap e13: x(active, since 61s) 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: overall HEALTH_OK 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:40.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:40.862 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:40.862 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:40 smithi083 ceph-mon[31413]: Reconfiguring daemon mon.a on smithi066 2024-04-21T08:59:40.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: mon.b calling monitor election 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: mon.b calling monitor election 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: mon.c calling monitor election 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: mon.a calling monitor election 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: monmap epoch 3 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: last_changed 2024-04-21T08:59:32.490234+0000 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: created 2024-04-21T08:57:31.934215+0000 2024-04-21T08:59:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: min_mon_release 19 (squid) 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: election_strategy: 1 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: 0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: fsmap 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: mgrmap e13: x(active, since 61s) 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: overall HEALTH_OK 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:59:40.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:40.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:40.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:40.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:40 smithi066 ceph-mon[27953]: Reconfiguring daemon mon.a on smithi066 2024-04-21T08:59:41.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:41 smithi066 ceph-mon[27953]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:41.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:41 smithi083 ceph-mon[31413]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:41.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:41.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:41.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:41.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:41.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:41.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:41.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:41 smithi186 ceph-mon[31505]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:59:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:42.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 -- 172.21.15.66:0/193331085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20481022f0 msgr2=0x7f204810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:42.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 --2- 172.21.15.66:0/193331085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20481022f0 0x7f204810e7e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2038009990 tx=0x7f2038009ca0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 -- 172.21.15.66:0/193331085 shutdown_connections 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 --2- 172.21.15.66:0/193331085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2048107980 0x7f2048107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 --2- 172.21.15.66:0/193331085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20481022f0 0x7f204810e7e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 --2- 172.21.15.66:0/193331085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2048101930 0x7f2048101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 -- 172.21.15.66:0/193331085 >> 172.21.15.66:0/193331085 conn(0x7f20480788b0 msgr2=0x7f2048078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 -- 172.21.15.66:0/193331085 shutdown_connections 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.587+0000 7f20504f2700 1 -- 172.21.15.66:0/193331085 wait complete. 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 Processor -- start 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 -- start start 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2048101930 0x7f2048197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20481022f0 0x7f2048198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:42.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2048107980 0x7f20481987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2048111b20 con 0x7f2048101930 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20481119a0 con 0x7f2048107980 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f20504f2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2048111ca0 con 0x7f20481022f0 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204e28e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2048101930 0x7f2048197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204e28e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2048101930 0x7f2048197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55408/0 (socket says 172.21.15.66:55408) 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204e28e700 1 -- 172.21.15.66:0/3441570060 learned_addr learned my addr 172.21.15.66:0/3441570060 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204ea8f700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2048107980 0x7f20481987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204da8d700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20481022f0 0x7f2048198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204e28e700 1 -- 172.21.15.66:0/3441570060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20481022f0 msgr2=0x7f2048198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204e28e700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20481022f0 0x7f2048198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204e28e700 1 -- 172.21.15.66:0/3441570060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2048107980 msgr2=0x7f20481987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.588+0000 7f204e28e700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2048107980 0x7f20481987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.589+0000 7f204e28e700 1 -- 172.21.15.66:0/3441570060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20481a7560 con 0x7f2048101930 2024-04-21T08:59:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.589+0000 7f204e28e700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2048101930 0x7f2048197d50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2040007f10 tx=0x7f204000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:42.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.589+0000 7f20477fe700 1 -- 172.21.15.66:0/3441570060 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040004db0 con 0x7f2048101930 2024-04-21T08:59:42.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.589+0000 7f20477fe700 1 -- 172.21.15.66:0/3441570060 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2040003a00 con 0x7f2048101930 2024-04-21T08:59:42.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.589+0000 7f20477fe700 1 -- 172.21.15.66:0/3441570060 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040004610 con 0x7f2048101930 2024-04-21T08:59:42.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.589+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20481a7850 con 0x7f2048101930 2024-04-21T08:59:42.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.589+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481a7d90 con 0x7f2048101930 2024-04-21T08:59:42.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.590+0000 7f20477fe700 1 -- 172.21.15.66:0/3441570060 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f204001c020 con 0x7f2048101930 2024-04-21T08:59:42.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.590+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2048069730 con 0x7f2048101930 2024-04-21T08:59:42.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.590+0000 7f20477fe700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2030039890 0x7f203003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:42.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.590+0000 7f20477fe700 1 -- 172.21.15.66:0/3441570060 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f204004c120 con 0x7f2048101930 2024-04-21T08:59:42.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.590+0000 7f204da8d700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2030039890 0x7f203003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:42.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.591+0000 7f204da8d700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2030039890 0x7f203003bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2048199eb0 tx=0x7f203c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:42.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.595+0000 7f20477fe700 1 -- 172.21.15.66:0/3441570060 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2040010070 con 0x7f2048101930 2024-04-21T08:59:42.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:42 smithi066 ceph-mon[27953]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:59:42.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:42 smithi066 ceph-mon[27953]: Reconfiguring daemon mon.b on smithi083 2024-04-21T08:59:42.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.806+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f204802df80 con 0x7f2048101930 2024-04-21T08:59:42.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.806+0000 7f20477fe700 1 -- 172.21.15.66:0/3441570060 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f204001c570 con 0x7f2048101930 2024-04-21T08:59:42.807 INFO:teuthology.orchestra.run.smithi066.stdout:# minimal ceph.conf for 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:42.807 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-04-21T08:59:42.807 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T08:59:42.807 INFO:teuthology.orchestra.run.smithi066.stdout: mon_host = [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2030039890 msgr2=0x7f203003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2030039890 0x7f203003bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2048199eb0 tx=0x7f203c00a300 comp rx=0 tx=0).stop 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2048101930 msgr2=0x7f2048197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2048101930 0x7f2048197d50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2040007f10 tx=0x7f204000d3b0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 shutdown_connections 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2030039890 0x7f203003bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2048107980 0x7f20481987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2048101930 0x7f2048197d50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 --2- 172.21.15.66:0/3441570060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20481022f0 0x7f2048198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 >> 172.21.15.66:0/3441570060 conn(0x7f20480788b0 msgr2=0x7f20480ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 shutdown_connections 2024-04-21T08:59:42.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:42.808+0000 7f20504f2700 1 -- 172.21.15.66:0/3441570060 wait complete. 2024-04-21T08:59:42.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:42 smithi186 ceph-mon[31505]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:59:42.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:42 smithi186 ceph-mon[31505]: Reconfiguring daemon mon.b on smithi083 2024-04-21T08:59:42.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:42 smithi083 ceph-mon[31413]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:59:42.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:42 smithi083 ceph-mon[31413]: Reconfiguring daemon mon.b on smithi083 2024-04-21T08:59:43.447 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-21T08:59:43.447 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:59:43.448 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:59:43.476 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:59:43.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:59:43.542 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:59:43.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:59:43.574 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:59:43.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:59:43.643 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:59:43.643 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:59:43.669 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:59:43.669 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: Reconfiguring daemon mon.c on smithi186 2024-04-21T08:59:43.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:43 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3441570060' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:43.732 INFO:tasks.cephadm:Adding mgr.x on smithi066 2024-04-21T08:59:43.732 INFO:tasks.cephadm:Adding mgr.y on smithi083 2024-04-21T08:59:43.733 INFO:tasks.cephadm:Adding mgr.z on smithi186 2024-04-21T08:59:43.733 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi066=x;smithi083=y;smithi186=z' 2024-04-21T08:59:43.793 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:44.059 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:44.059 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:44.060 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:59:44.060 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:44.060 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:44.060 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:44.060 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: Reconfiguring daemon mon.c on smithi186 2024-04-21T08:59:44.060 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:43 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3441570060' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: Reconfiguring daemon mon.c on smithi186 2024-04-21T08:59:44.061 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:43 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3441570060' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:44.236 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:59:45.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:45 smithi186 ceph-mon[31505]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:45.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:45.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:45.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:45.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:45 smithi083 ceph-mon[31413]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:45 smithi066 ceph-mon[27953]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:45.817 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 -- 172.21.15.186:0/501599754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0104070 msgr2=0x7f84d0104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 --2- 172.21.15.186:0/501599754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0104070 0x7f84d0104490 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00b0a0 tx=0x7f84cc00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 -- 172.21.15.186:0/501599754 shutdown_connections 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 --2- 172.21.15.186:0/501599754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84d0101490 0x7f84d010b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 --2- 172.21.15.186:0/501599754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0104070 0x7f84d0104490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 --2- 172.21.15.186:0/501599754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84d0100af0 0x7f84d0100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 -- 172.21.15.186:0/501599754 >> 172.21.15.186:0/501599754 conn(0x7f84d00fc6d0 msgr2=0x7f84d00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 -- 172.21.15.186:0/501599754 shutdown_connections 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.815+0000 7f84d79a7700 1 -- 172.21.15.186:0/501599754 wait complete. 2024-04-21T08:59:45.818 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 Processor -- start 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 -- start start 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84d0100af0 0x7f84d0198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0101490 0x7f84d0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84d0104070 0x7f84d01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f84d010e590 con 0x7f84d0100af0 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84d010e410 con 0x7f84d0104070 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d79a7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84d010e710 con 0x7f84d0101490 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d4f42700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0101490 0x7f84d0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d4f42700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0101490 0x7f84d0199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:35258/0 (socket says 172.21.15.186:35258) 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d4f42700 1 -- 172.21.15.186:0/2490370664 learned_addr learned my addr 172.21.15.186:0/2490370664 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.816+0000 7f84d5743700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84d0100af0 0x7f84d0198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d4f42700 1 -- 172.21.15.186:0/2490370664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84d0104070 msgr2=0x7f84d01a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d5f44700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84d0104070 0x7f84d01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d4f42700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84d0104070 0x7f84d01a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:45.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d4f42700 1 -- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84d0100af0 msgr2=0x7f84d0198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d4f42700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84d0100af0 0x7f84d0198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d4f42700 1 -- 172.21.15.186:0/2490370664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d01a0950 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d5f44700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84d0104070 0x7f84d01a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d4f42700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0101490 0x7f84d0199240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84c000b530 tx=0x7f84c000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84c67fc700 1 -- 172.21.15.186:0/2490370664 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0015020 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84c67fc700 1 -- 172.21.15.186:0/2490370664 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84c0011ce0 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d01a0c40 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84c67fc700 1 -- 172.21.15.186:0/2490370664 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0010e40 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.817+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d01a1180 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.818+0000 7f84c67fc700 1 -- 172.21.15.186:0/2490370664 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f84c0018020 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.818+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d0069730 con 0x7f84d0101490 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.818+0000 7f84c67fc700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84bc039890 0x7f84bc03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:45.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.818+0000 7f84c67fc700 1 -- 172.21.15.186:0/2490370664 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f84c004c230 con 0x7f84d0101490 2024-04-21T08:59:45.821 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.818+0000 7f84d5743700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84bc039890 0x7f84bc03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:45.821 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.819+0000 7f84d5743700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84bc039890 0x7f84bc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84d019a1e0 tx=0x7f84cc009c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:45.823+0000 7f84c67fc700 1 -- 172.21.15.186:0/2490370664 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84c001f020 con 0x7f84d0101490 2024-04-21T08:59:46.031 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.031+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi083=y;smithi186=z", "target": ["mon-mgr", ""]}) -- 0x7f84d0062e80 con 0x7f84bc039890 2024-04-21T08:59:46.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.037+0000 7f84c67fc700 1 -- 172.21.15.186:0/2490370664 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f84d0062e80 con 0x7f84bc039890 2024-04-21T08:59:46.037 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2024-04-21T08:59:46.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84bc039890 msgr2=0x7f84bc03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:46.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84bc039890 0x7f84bc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84d019a1e0 tx=0x7f84cc009c90 comp rx=0 tx=0).stop 2024-04-21T08:59:46.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0101490 msgr2=0x7f84d0199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:46.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0101490 0x7f84d0199240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84c000b530 tx=0x7f84c000b840 comp rx=0 tx=0).stop 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 shutdown_connections 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84bc039890 0x7f84bc03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84d0104070 0x7f84d01a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84d0100af0 0x7f84d0198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 --2- 172.21.15.186:0/2490370664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84d0101490 0x7f84d0199240 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 >> 172.21.15.186:0/2490370664 conn(0x7f84d00fc6d0 msgr2=0x7f84d0107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 shutdown_connections 2024-04-21T08:59:46.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T08:59:46.040+0000 7f84d79a7700 1 -- 172.21.15.186:0/2490370664 wait complete. 2024-04-21T08:59:46.606 DEBUG:teuthology.orchestra.run.smithi083:mgr.y> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.y.service 2024-04-21T08:59:46.609 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.z.service 2024-04-21T08:59:46.648 INFO:tasks.cephadm:Deploying OSDs... 2024-04-21T08:59:46.648 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T08:59:46.648 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:59:46.667 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:59:46.668 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-21 08:58:48.320892843 +0000 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-21 08:53:12.974724086 +0000 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-21 08:53:12.974724086 +0000 2024-04-21T08:59:46.727 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-21 08:53:12.974724086 +0000 2024-04-21T08:59:46.728 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:59:46.796 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-21T08:59:46.797 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-21T08:59:46.797 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000235412 s, 2.2 MB/s 2024-04-21T08:59:46.798 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:59:46.856 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-04-21T08:59:46.913 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:59:46.913 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:46.914 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-21T08:59:46.914 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:46.914 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:46.914 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-21 08:58:48.353892594 +0000 2024-04-21T08:59:46.914 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-21 08:53:13.330721715 +0000 2024-04-21T08:59:46.914 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-21 08:53:13.330721715 +0000 2024-04-21T08:59:46.914 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-21 08:53:13.330721715 +0000 2024-04-21T08:59:46.914 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:59:46.981 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-21T08:59:46.981 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-21T08:59:46.982 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000235252 s, 2.2 MB/s 2024-04-21T08:59:46.983 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:59:47.041 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-21 08:58:48.386892346 +0000 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-21 08:53:13.671719443 +0000 2024-04-21T08:59:47.100 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-21 08:53:13.671719443 +0000 2024-04-21T08:59:47.101 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-21 08:53:13.671719443 +0000 2024-04-21T08:59:47.101 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:59:47.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi083=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: Saving service mgr spec with placement smithi066=x;smithi083=y;smithi186=z;count:3 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:47.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:47 smithi066 ceph-mon[27953]: Deploying daemon mgr.z on smithi186 2024-04-21T08:59:47.145 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-21T08:59:47.146 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-21T08:59:47.146 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000243436 s, 2.1 MB/s 2024-04-21T08:59:47.147 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:59:47.205 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-04-21T08:59:47.261 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-21 08:58:48.426892044 +0000 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-21 08:53:14.019717125 +0000 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-21 08:53:14.019717125 +0000 2024-04-21T08:59:47.262 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-21 08:53:14.019717125 +0000 2024-04-21T08:59:47.262 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:59:47.326 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-21T08:59:47.327 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-21T08:59:47.327 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000209967 s, 2.4 MB/s 2024-04-21T08:59:47.328 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi083=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: Saving service mgr spec with placement smithi066=x;smithi083=y;smithi186=z;count:3 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:47 smithi186 ceph-mon[31505]: Deploying daemon mgr.z on smithi186 2024-04-21T08:59:47.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi083=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:47.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: Saving service mgr spec with placement smithi066=x;smithi083=y;smithi186=z;count:3 2024-04-21T08:59:47.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:47.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:47 smithi083 ceph-mon[31413]: Deploying daemon mgr.z on smithi186 2024-04-21T08:59:47.386 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-21T08:59:47.386 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:59:47.405 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:59:47.405 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-21T08:59:47.462 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:59:47.462 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:47.462 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-21T08:59:47.462 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:47.462 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:47.463 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-21 08:59:31.923336279 +0000 2024-04-21T08:59:47.463 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-21 08:54:06.841294024 +0000 2024-04-21T08:59:47.463 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-21 08:54:06.841294024 +0000 2024-04-21T08:59:47.463 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-21 08:54:06.841294024 +0000 2024-04-21T08:59:47.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:59:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-21T08:59:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-21T08:59:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000351007 s, 1.5 MB/s 2024-04-21T08:59:47.530 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:59:47.587 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-21T08:59:47.643 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-21 08:59:31.923336279 +0000 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-21 08:54:07.176286123 +0000 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-21 08:54:07.176286123 +0000 2024-04-21T08:59:47.644 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-21 08:54:07.176286123 +0000 2024-04-21T08:59:47.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:59:47.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-21T08:59:47.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-21T08:59:47.708 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00022925 s, 2.2 MB/s 2024-04-21T08:59:47.709 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:59:47.766 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-21T08:59:47.822 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:59:47.822 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:47.822 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-21T08:59:47.823 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:47.823 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:47.823 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-21 08:59:31.923336279 +0000 2024-04-21T08:59:47.823 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-21 08:54:07.523277940 +0000 2024-04-21T08:59:47.823 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-21 08:54:07.523277940 +0000 2024-04-21T08:59:47.823 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-21 08:54:07.523277940 +0000 2024-04-21T08:59:47.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:59:47.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-21T08:59:47.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-21T08:59:47.886 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000224223 s, 2.3 MB/s 2024-04-21T08:59:47.887 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:59:47.944 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-21T08:59:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:59:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:48.000 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-21T08:59:48.000 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:48.000 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:48.000 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-21 08:59:31.924336256 +0000 2024-04-21T08:59:48.000 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-21 08:54:07.856270086 +0000 2024-04-21T08:59:48.000 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-21 08:54:07.856270086 +0000 2024-04-21T08:59:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-21 08:54:07.856270086 +0000 2024-04-21T08:59:48.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:59:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-21T08:59:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-21T08:59:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000228936 s, 2.2 MB/s 2024-04-21T08:59:48.064 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:59:48.121 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-21T08:59:48.121 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:59:48.138 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:59:48.138 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-04-21T08:59:48.194 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:59:48.194 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:48.194 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-21T08:59:48.195 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:48.195 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:48.195 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-21 08:59:48.008379056 +0000 2024-04-21T08:59:48.195 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-21 08:53:29.381694093 +0000 2024-04-21T08:59:48.195 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-21 08:53:29.381694093 +0000 2024-04-21T08:59:48.195 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-21 08:53:29.381694093 +0000 2024-04-21T08:59:48.195 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:59:48.259 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-21T08:59:48.259 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-21T08:59:48.259 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000261499 s, 2.0 MB/s 2024-04-21T08:59:48.260 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:59:48.315 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-04-21T08:59:48.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:48 smithi186 ceph-mon[31505]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:48.338 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 podman[33319]: 2024-04-21 08:59:48.301170374 +0000 UTC m=+0.126045254 container create 26a6274ad30b3c4972905a34b2307c7c54124b1bdd653da785647c90b76b89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:59:48.351 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:59:48.351 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:48.351 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-21T08:59:48.351 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:48.351 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:48.351 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-21 08:59:48.009379038 +0000 2024-04-21T08:59:48.352 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-21 08:53:29.721687913 +0000 2024-04-21T08:59:48.352 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-21 08:53:29.721687913 +0000 2024-04-21T08:59:48.352 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-21 08:53:29.721687913 +0000 2024-04-21T08:59:48.352 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:59:48.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:48 smithi083 ceph-mon[31413]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:48.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:48 smithi066 ceph-mon[27953]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:48.419 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-21T08:59:48.419 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-21T08:59:48.419 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000185681 s, 2.8 MB/s 2024-04-21T08:59:48.420 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:59:48.476 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-04-21T08:59:48.530 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-21 08:59:48.009379038 +0000 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-21 08:53:30.053681878 +0000 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-21 08:53:30.053681878 +0000 2024-04-21T08:59:48.531 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-21 08:53:30.053681878 +0000 2024-04-21T08:59:48.531 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:59:48.595 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-21T08:59:48.596 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-21T08:59:48.596 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000199491 s, 2.6 MB/s 2024-04-21T08:59:48.597 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:59:48.652 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-04-21T08:59:48.707 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:59:48.707 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:59:48.707 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-21T08:59:48.707 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:59:48.708 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:59:48.708 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-21 08:59:48.010379020 +0000 2024-04-21T08:59:48.708 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-21 08:53:30.398675606 +0000 2024-04-21T08:59:48.708 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-21 08:53:30.398675606 +0000 2024-04-21T08:59:48.708 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-21 08:53:30.398675606 +0000 2024-04-21T08:59:48.708 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:59:48.712 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 podman[33319]: 2024-04-21 08:59:48.554206546 +0000 UTC m=+0.379081434 container init 26a6274ad30b3c4972905a34b2307c7c54124b1bdd653da785647c90b76b89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:59:48.712 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 podman[33319]: 2024-04-21 08:59:48.55689856 +0000 UTC m=+0.381773445 container start 26a6274ad30b3c4972905a34b2307c7c54124b1bdd653da785647c90b76b89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T08:59:48.713 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 ceph-mgr[33460]: -- 172.21.15.186:0/3011134320 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f4265a5a0 con 0x559f4262ec00 2024-04-21T08:59:48.713 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 bash[33319]: 26a6274ad30b3c4972905a34b2307c7c54124b1bdd653da785647c90b76b89b1 2024-04-21T08:59:48.713 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 systemd[1]: Started Ceph mgr.z for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T08:59:48.736 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-21T08:59:48.737 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-21T08:59:48.737 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000233058 s, 2.2 MB/s 2024-04-21T08:59:48.738 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:59:48.793 INFO:tasks.cephadm:Deploying osd.0 on smithi066 with /dev/vg_nvme/lv_4... 2024-04-21T08:59:48.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:59:49.006 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:48.712+0000 7f1ac065e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:59:49.006 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:48.904+0000 7f1ac065e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:59:49.006 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:49.006+0000 7f1ac065e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:59:49.040 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:59:49.587 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:49.318+0000 7f1ac065e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:59:49.588 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:49.404+0000 7f1ac065e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:59:49.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:49 smithi066 ceph-mon[27953]: Deploying daemon mgr.y on smithi083 2024-04-21T08:59:49.972 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:49.972 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:49.973 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:49.973 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:49.973 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:49.973 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:59:49.973 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:49.973 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:49.973 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:49 smithi083 ceph-mon[31413]: Deploying daemon mgr.y on smithi083 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:50.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:49 smithi186 ceph-mon[31505]: Deploying daemon mgr.y on smithi083 2024-04-21T08:59:50.950 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:50.603+0000 7f1ac065e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:59:50.950 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:50.771+0000 7f1ac065e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:59:50.950 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:50.949+0000 7f1ac065e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:59:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:59:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:59:51.135 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-21T08:59:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-21T08:59:51.136 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111132 s, 944 MB/s 2024-04-21T08:59:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-21T08:59:51.219 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:51 smithi083 systemd[1]: Started Ceph mgr.y for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T08:59:51.515 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:51.217+0000 7fc9116e5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:59:51.515 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:51.406+0000 7fc9116e5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:59:51.515 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:51.515+0000 7fc9116e5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:59:51.570 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:51.254+0000 7f1ac065e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:59:51.570 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:51.351+0000 7f1ac065e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:59:51.570 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:51.570+0000 7f1ac065e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:59:51.743 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2024-04-21T08:59:51.837 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:51.773+0000 7f1ac065e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:59:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-mon[31505]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:51.839 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-mon[31413]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:51.839 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.839 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.839 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.839 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:51.840 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:51.982 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:59:52.104 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:51 smithi066 ceph-mon[27953]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:52.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:52.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:52.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:52.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:52.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:52.109 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:51.839+0000 7fc9116e5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:59:52.109 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:51.942+0000 7fc9116e5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:59:52.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:52.419+0000 7f1ac065e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:59:53.148 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:52.851+0000 7f1ac065e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:59:53.148 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:52.949+0000 7f1ac065e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:59:53.148 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:53.148+0000 7f1ac065e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:59:53.485 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:53.155+0000 7fc9116e5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:59:53.485 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:53.319+0000 7fc9116e5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:59:53.485 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:53.485+0000 7fc9116e5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:59:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 -- 172.21.15.66:0/838112187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 msgr2=0x7f5d7410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 --2- 172.21.15.66:0/838112187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d7410abf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00b0a0 tx=0x7f5d6c00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:59:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 -- 172.21.15.66:0/838112187 shutdown_connections 2024-04-21T08:59:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 --2- 172.21.15.66:0/838112187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d7410abf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 --2- 172.21.15.66:0/838112187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 0x7f5d74105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 --2- 172.21.15.66:0/838112187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d74105ed0 0x7f5d741082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 -- 172.21.15.66:0/838112187 >> 172.21.15.66:0/838112187 conn(0x7f5d740fcf30 msgr2=0x7f5d740ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:59:53.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 -- 172.21.15.66:0/838112187 shutdown_connections 2024-04-21T08:59:53.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.581+0000 7f5d7982b700 1 -- 172.21.15.66:0/838112187 wait complete. 2024-04-21T08:59:53.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 Processor -- start 2024-04-21T08:59:53.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 -- start start 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 0x7f5d74198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d74105ed0 0x7f5d74199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d741a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7410e510 con 0x7f5d74108800 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7410e390 con 0x7f5d74105ed0 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d7982b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7410e690 con 0x7f5d741035a0 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d737fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d741a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d72ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 0x7f5d74198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d727fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d74105ed0 0x7f5d74199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d72ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 0x7f5d74198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:43444/0 (socket says 172.21.15.66:43444) 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d72ffd700 1 -- 172.21.15.66:0/1675232036 learned_addr learned my addr 172.21.15.66:0/1675232036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.582+0000 7f5d737fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d741a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36470/0 (socket says 172.21.15.66:36470) 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d72ffd700 1 -- 172.21.15.66:0/1675232036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d74105ed0 msgr2=0x7f5d74199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d72ffd700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d74105ed0 0x7f5d74199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d72ffd700 1 -- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 msgr2=0x7f5d741a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d72ffd700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d741a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d72ffd700 1 -- 172.21.15.66:0/1675232036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d741a0780 con 0x7f5d741035a0 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d737fe700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d741a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d72ffd700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 0x7f5d74198b20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400e9f0 tx=0x7f5d6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400cbd0 con 0x7f5d741035a0 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d6400ee90 con 0x7f5d741035a0 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d741a0a70 con 0x7f5d741035a0 2024-04-21T08:59:53.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64005590 con 0x7f5d741035a0 2024-04-21T08:59:53.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.583+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d741a0fb0 con 0x7f5d741035a0 2024-04-21T08:59:53.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.584+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d74069730 con 0x7f5d741035a0 2024-04-21T08:59:53.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.584+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5d64021440 con 0x7f5d741035a0 2024-04-21T08:59:53.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.584+0000 7f5d78829700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d5c039890 0x7f5d5c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:59:53.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.584+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5d6404cc20 con 0x7f5d741035a0 2024-04-21T08:59:53.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.585+0000 7f5d727fc700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d5c039890 0x7f5d5c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:59:53.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.585+0000 7f5d727fc700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d5c039890 0x7f5d5c03bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009680 tx=0x7f5d68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:59:53.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.590+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d64016bd0 con 0x7f5d741035a0 2024-04-21T08:59:53.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:53.820+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5d74062e80 con 0x7f5d5c039890 2024-04-21T08:59:53.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:53.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:53.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: Reconfiguring daemon mgr.x on smithi066 2024-04-21T08:59:53.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:53.999 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:53.670+0000 7f1ac065e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:59:53.999 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:53.761+0000 7f1ac065e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:59:53.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:53.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:53.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: Reconfiguring daemon mgr.x on smithi066 2024-04-21T08:59:54.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.052 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:53.766+0000 7fc9116e5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:59:54.052 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:53.855+0000 7fc9116e5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:59:54.052 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:54.052+0000 7fc9116e5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:59:54.052 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:54.053 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: Reconfiguring daemon mgr.x on smithi066 2024-04-21T08:59:54.054 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.337 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:53.999+0000 7f1ac065e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:59:54.337 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:54.130+0000 7f1ac065e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:59:54.337 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:54.224+0000 7f1ac065e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:59:54.337 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:54.337+0000 7f1ac065e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:59:54.359 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:54.239+0000 7fc9116e5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:59:54.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:54 smithi066 ceph-mon[27953]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:54.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:59:54.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:59:54.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:54.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:54.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:55.087 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:54.638+0000 7f1ac065e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:59:55.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-mon[31505]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:55.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:59:55.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:59:55.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:55.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:55.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:55.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:55.109 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:54.845+0000 7fc9116e5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:59:55.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-mon[31413]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:59:55.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:59:55.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:59:55.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:55.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:55.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:55.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:59:55.556 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:55.263+0000 7fc9116e5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:59:55.556 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:55.357+0000 7fc9116e5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:59:55.556 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:55.556+0000 7fc9116e5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:59:55.588 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:55.240+0000 7f1ac065e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:59:55.588 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T08:59:55.452+0000 7f1ac065e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:59:55.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:55.765+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f5d6401f340 con 0x7f5d741035a0 2024-04-21T08:59:55.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:55.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:55.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: Standby manager daemon z started 2024-04-21T08:59:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:59:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:59:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:59:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:55 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:59:55.983 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:55.983 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:55.984 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:55.984 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:55.984 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: Standby manager daemon z started 2024-04-21T08:59:55.984 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:59:55.984 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:59:55.984 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:59:55.984 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:55 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:59:56.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: Standby manager daemon z started 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:59:56.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:55 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/29422673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:59:56.359 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:56.086+0000 7fc9116e5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:59:56.359 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:56.177+0000 7fc9116e5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:59:56.756 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:56.416+0000 7fc9116e5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:59:56.757 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:56.546+0000 7fc9116e5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:59:56.757 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:56.641+0000 7fc9116e5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:59:56.757 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:56.756+0000 7fc9116e5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:59:57.069 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-mon[31413]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-21T08:59:57.069 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:56 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:59:57.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:56 smithi186 ceph-mon[31505]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-21T08:59:57.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:56 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:59:57.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:56 smithi066 ceph-mon[27953]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-21T08:59:57.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:56 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:59:57.359 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:57.069+0000 7fc9116e5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:59:58.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:57 smithi186 ceph-mon[31505]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:57 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/525091138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5f4d872-e0ff-41aa-bd69-ac73da467920"}]: dispatch 2024-04-21T08:59:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:57 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/525091138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5f4d872-e0ff-41aa-bd69-ac73da467920"}]': finished 2024-04-21T08:59:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:57 smithi186 ceph-mon[31505]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:59:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:59:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:57 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3028585705' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:59:58.109 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:57.663+0000 7fc9116e5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:59:58.109 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T08:59:57.878+0000 7fc9116e5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:59:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-mon[31413]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/525091138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5f4d872-e0ff-41aa-bd69-ac73da467920"}]: dispatch 2024-04-21T08:59:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/525091138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5f4d872-e0ff-41aa-bd69-ac73da467920"}]': finished 2024-04-21T08:59:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-mon[31413]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:59:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:59:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:57 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3028585705' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:59:58.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:57 smithi066 ceph-mon[27953]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:59:58.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:57 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/525091138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5f4d872-e0ff-41aa-bd69-ac73da467920"}]: dispatch 2024-04-21T08:59:58.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:57 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/525091138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5f4d872-e0ff-41aa-bd69-ac73da467920"}]': finished 2024-04-21T08:59:58.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:57 smithi066 ceph-mon[27953]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:59:58.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:59:58.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:57 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3028585705' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:59:58.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T08:59:58.777+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mon.1 v2:172.21.15.186:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5d64014230 con 0x7f5d741035a0 2024-04-21T08:59:59.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:58 smithi186 ceph-mon[31505]: Standby manager daemon y started 2024-04-21T08:59:59.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:58 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:59:59.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:58 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:59:59.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:58 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:59:59.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:58 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:59:59.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:58 smithi083 ceph-mon[31413]: Standby manager daemon y started 2024-04-21T08:59:59.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:58 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:59:59.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:58 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:59:59.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:58 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:59:59.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:58 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:59:59.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:58 smithi066 ceph-mon[27953]: Standby manager daemon y started 2024-04-21T08:59:59.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:58 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:59:59.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:58 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:59:59.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:58 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:59:59.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:58 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/2393319430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T09:00:00.039 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:59 smithi083 ceph-mon[31413]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:00.039 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:59 smithi083 ceph-mon[31413]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-21T09:00:00.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 08:59:59 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T09:00:00.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:59 smithi186 ceph-mon[31505]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:00.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:59 smithi186 ceph-mon[31505]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-21T09:00:00.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 08:59:59 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T09:00:00.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:59 smithi066 ceph-mon[27953]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:00.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:59 smithi066 ceph-mon[27953]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-21T09:00:00.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 08:59:59 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T09:00:01.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:00 smithi186 ceph-mon[31505]: overall HEALTH_OK 2024-04-21T09:00:01.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:00 smithi083 ceph-mon[31413]: overall HEALTH_OK 2024-04-21T09:00:01.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:00 smithi066 ceph-mon[27953]: overall HEALTH_OK 2024-04-21T09:00:01.780 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:01 smithi186 ceph-mon[31505]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:02.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:01 smithi083 ceph-mon[31413]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:02.113 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:01 smithi066 ceph-mon[27953]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:04.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:03 smithi186 ceph-mon[31505]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:04.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:03 smithi083 ceph-mon[31413]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:04.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:03 smithi066 ceph-mon[27953]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:04.907 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:04 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T09:00:04.908 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:04 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:04.908 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:04 smithi066 ceph-mon[27953]: Deploying daemon osd.0 on smithi066 2024-04-21T09:00:05.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:04 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T09:00:05.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:04 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:05.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:04 smithi186 ceph-mon[31505]: Deploying daemon osd.0 on smithi066 2024-04-21T09:00:05.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:04 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T09:00:05.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:04 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:05.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:04 smithi083 ceph-mon[31413]: Deploying daemon osd.0 on smithi066 2024-04-21T09:00:05.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:05 smithi066 ceph-mon[27953]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:06.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:05 smithi186 ceph-mon[31505]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:06.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:05 smithi083 ceph-mon[31413]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:07.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:07 smithi066 ceph-mon[27953]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:08.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:07 smithi186 ceph-mon[31505]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:08.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:07 smithi083 ceph-mon[31413]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:10.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:09 smithi186 ceph-mon[31505]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:10.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:09 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:10.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:09 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:10.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:09 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:10.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:09 smithi083 ceph-mon[31413]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:10.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:09 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:10.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:09 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:10.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:09 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:10.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:09 smithi066 ceph-mon[27953]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:10.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:09 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:10.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:09 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:10.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:09 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:12.054 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:11 smithi066 ceph-mon[27953]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:12.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:11 smithi186 ceph-mon[31505]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:12.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:11 smithi083 ceph-mon[31413]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.925+0000 7f5d78829700 1 -- 172.21.15.66:0/1675232036 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5d74062e80 con 0x7f5d5c039890 2024-04-21T09:00:12.925 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 0 on host 'smithi066' 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d5c039890 msgr2=0x7f5d5c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d5c039890 0x7f5d5c03bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009680 tx=0x7f5d68009260 comp rx=0 tx=0).stop 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 msgr2=0x7f5d74198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 0x7f5d74198b20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400e9f0 tx=0x7f5d6400ed00 comp rx=0 tx=0).stop 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 shutdown_connections 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d5c039890 0x7f5d5c03bd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d74108800 0x7f5d741a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d741035a0 0x7f5d74198b20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 --2- 172.21.15.66:0/1675232036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d74105ed0 0x7f5d74199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.927+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 >> 172.21.15.66:0/1675232036 conn(0x7f5d740fcf30 msgr2=0x7f5d74103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:00:12.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.928+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 shutdown_connections 2024-04-21T09:00:12.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:00:12.928+0000 7f5d7982b700 1 -- 172.21.15.66:0/1675232036 wait complete. 2024-04-21T09:00:13.583 DEBUG:teuthology.orchestra.run.smithi066:osd.0> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.0.service 2024-04-21T09:00:13.585 INFO:tasks.cephadm:Deploying osd.1 on smithi083 with /dev/vg_nvme/lv_4... 2024-04-21T09:00:13.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T09:00:13.826 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:00:14.085 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:13 smithi066 ceph-mon[27953]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:14.086 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.086 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.086 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.086 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.086 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:13 smithi066 ceph-mon[27953]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T09:00:14.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:13 smithi083 ceph-mon[31413]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:14.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.186 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.186 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:13 smithi083 ceph-mon[31413]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T09:00:14.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:13 smithi186 ceph-mon[31505]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:14.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:14.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:13 smithi186 ceph-mon[31505]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T09:00:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:14 smithi066 ceph-mon[27953]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T09:00:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:14 smithi066 ceph-mon[27953]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T09:00:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:14 smithi066 ceph-mon[27953]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:00:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:15.098 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:14 smithi083 ceph-mon[31413]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T09:00:15.098 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:14 smithi083 ceph-mon[31413]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T09:00:15.098 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:14 smithi083 ceph-mon[31413]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:00:15.098 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:15.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:14 smithi186 ceph-mon[31505]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T09:00:15.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:14 smithi186 ceph-mon[31505]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T09:00:15.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:14 smithi186 ceph-mon[31505]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:00:15.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T09:00:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T09:00:15.649 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-21T09:00:15.649 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-21T09:00:15.649 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114267 s, 918 MB/s 2024-04-21T09:00:15.649 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-21T09:00:15.870 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0[40464]: 2024-04-21T09:00:15.469+0000 7fe21aa13700 -1 osd.0 0 waiting for initial osdmap 2024-04-21T09:00:15.870 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0[40464]: 2024-04-21T09:00:15.473+0000 7fe2127fe700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:00:16.244 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-mon[27953]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:16.245 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-mon[27953]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:00:16.245 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-mon[27953]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T09:00:16.245 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:16.245 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:16.245 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:15 smithi066 ceph-mon[27953]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' 2024-04-21T09:00:16.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:15 smithi083 ceph-mon[31413]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:16.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:15 smithi083 ceph-mon[31413]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:00:16.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:15 smithi083 ceph-mon[31413]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T09:00:16.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:16.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:16.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:15 smithi083 ceph-mon[31413]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' 2024-04-21T09:00:16.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-04-21T09:00:16.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:15 smithi186 ceph-mon[31505]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:16.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:15 smithi186 ceph-mon[31505]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:00:16.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:15 smithi186 ceph-mon[31505]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T09:00:16.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:16.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:16.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:15 smithi186 ceph-mon[31505]: from='osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786]' entity='osd.0' 2024-04-21T09:00:16.539 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:00:17.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:00:17.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:00:17.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:17.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] boot 2024-04-21T09:00:17.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T09:00:17.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:17.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:00:17.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:17.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:17.340 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.340 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:17.340 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:00:17.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:00:17.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:17.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] boot 2024-04-21T09:00:17.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T09:00:17.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:17.342 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.342 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.342 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:00:17.342 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.342 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:17.343 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:17.343 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.343 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: osd.0 [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] boot 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:17.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:17.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:18.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:17 smithi066 ceph-mon[27953]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:18.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:17 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi066 2024-04-21T09:00:18.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:17 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi066 to 17088M 2024-04-21T09:00:18.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:17 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:18.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:17 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:18.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:17 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.138+0000 7f83ad2c8700 1 -- 172.21.15.83:0/2974031644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81019d0 msgr2=0x7f83a8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.138+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/2974031644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81019d0 0x7f83a8101df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f83980099c0 tx=0x7f8398009cd0 comp rx=0 tx=0).stop 2024-04-21T09:00:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.138+0000 7f83ad2c8700 1 -- 172.21.15.83:0/2974031644 shutdown_connections 2024-04-21T09:00:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.138+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/2974031644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81019d0 0x7f83a8101df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.138+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/2974031644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83a8103570 0x7f83a8109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.138+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/2974031644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a8102bd0 0x7f83a8103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.138+0000 7f83ad2c8700 1 -- 172.21.15.83:0/2974031644 >> 172.21.15.83:0/2974031644 conn(0x7f83a80fcf90 msgr2=0x7f83a80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 -- 172.21.15.83:0/2974031644 shutdown_connections 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 -- 172.21.15.83:0/2974031644 wait complete. 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 Processor -- start 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 -- start start 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a81019d0 0x7f83a8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83a8102bd0 0x7f83a8199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a8103570 0x7f83a81a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83a810d440 con 0x7f83a8102bd0 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83a810d2c0 con 0x7f83a8103570 2024-04-21T09:00:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.139+0000 7f83ad2c8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83a810d5c0 con 0x7f83a81019d0 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a8103570 0x7f83a81a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a8103570 0x7f83a81a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42886/0 (socket says 172.21.15.83:42886) 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 -- 172.21.15.83:0/582367105 learned_addr learned my addr 172.21.15.83:0/582367105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a67fc700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83a8102bd0 0x7f83a8199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a6ffd700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a81019d0 0x7f83a8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 -- 172.21.15.83:0/582367105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a81019d0 msgr2=0x7f83a8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a81019d0 0x7f83a8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 -- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83a8102bd0 msgr2=0x7f83a8199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83a8102bd0 0x7f83a8199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 -- 172.21.15.83:0/582367105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a81a0730 con 0x7f83a8103570 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83a77fe700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a8103570 0x7f83a81a0090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83a0007d70 tx=0x7f83a000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f838ffff700 1 -- 172.21.15.83:0/582367105 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0016020 con 0x7f83a8103570 2024-04-21T09:00:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a81a0a20 con 0x7f83a8103570 2024-04-21T09:00:18.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f838ffff700 1 -- 172.21.15.83:0/582367105 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a000ad40 con 0x7f83a8103570 2024-04-21T09:00:18.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f838ffff700 1 -- 172.21.15.83:0/582367105 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00115e0 con 0x7f83a8103570 2024-04-21T09:00:18.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.140+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81a0f10 con 0x7f83a8103570 2024-04-21T09:00:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.141+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a8069730 con 0x7f83a8103570 2024-04-21T09:00:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.142+0000 7f838ffff700 1 -- 172.21.15.83:0/582367105 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f83a0019020 con 0x7f83a8103570 2024-04-21T09:00:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.143+0000 7f838ffff700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f83900a3bb0 0x7f83900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.143+0000 7f838ffff700 1 -- 172.21.15.83:0/582367105 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f83a00ce8c0 con 0x7f83a8103570 2024-04-21T09:00:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.143+0000 7f83a6ffd700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f83900a3bb0 0x7f83900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.143+0000 7f83a6ffd700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f83900a3bb0 0x7f83900a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f839800b580 tx=0x7f8398005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:00:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.147+0000 7f838ffff700 1 -- 172.21.15.83:0/582367105 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83a0020020 con 0x7f83a8103570 2024-04-21T09:00:18.253 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:17 smithi186 ceph-mon[31505]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:18.253 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:17 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi066 2024-04-21T09:00:18.253 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:17 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi066 to 17088M 2024-04-21T09:00:18.253 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:17 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:18.253 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:17 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:18.253 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:17 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:18.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:18.349+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f83a80ff050 con 0x7f83900a3bb0 2024-04-21T09:00:18.349 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:17 smithi083 ceph-mon[31413]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T09:00:18.349 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:17 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi066 2024-04-21T09:00:18.349 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:17 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi066 to 17088M 2024-04-21T09:00:18.349 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:17 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:18.350 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:17 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:18.350 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:17 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:19.305 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:19 smithi083 ceph-mon[31413]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T09:00:19.306 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:00:19.306 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:00:19.306 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:19.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:19 smithi186 ceph-mon[31505]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T09:00:19.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:00:19.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:00:19.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:19.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:19 smithi066 ceph-mon[27953]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T09:00:19.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:00:19.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:00:19.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:20.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:20 smithi083 ceph-mon[31413]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:20.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:20 smithi083 ceph-mon[31413]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:00:20.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:20 smithi186 ceph-mon[31505]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:20.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:20 smithi186 ceph-mon[31505]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:00:20.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:20 smithi066 ceph-mon[27953]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:20.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:20 smithi066 ceph-mon[27953]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:00:22.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:22 smithi186 ceph-mon[31505]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:22.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:22 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/3502562466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]: dispatch 2024-04-21T09:00:22.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:22 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]: dispatch 2024-04-21T09:00:22.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:22 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]': finished 2024-04-21T09:00:22.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:22 smithi186 ceph-mon[31505]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T09:00:22.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:22.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:22 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/1755575958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:00:22.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:22 smithi083 ceph-mon[31413]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:22.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:22 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/3502562466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]: dispatch 2024-04-21T09:00:22.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:22 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]: dispatch 2024-04-21T09:00:22.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:22 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]': finished 2024-04-21T09:00:22.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:22 smithi083 ceph-mon[31413]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T09:00:22.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:22.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:22 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/1755575958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:00:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:22 smithi066 ceph-mon[27953]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/3502562466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]: dispatch 2024-04-21T09:00:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:22 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]: dispatch 2024-04-21T09:00:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:22 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f582ca-69a5-425c-a239-3f46fb57c283"}]': finished 2024-04-21T09:00:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:22 smithi066 ceph-mon[27953]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T09:00:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/1755575958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:00:24.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:24 smithi186 ceph-mon[31505]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:24.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:24 smithi083 ceph-mon[31413]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:24.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:24 smithi066 ceph-mon[27953]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:26.295 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:26 smithi083 ceph-mon[31413]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:26.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:26 smithi186 ceph-mon[31505]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:26.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:26 smithi066 ceph-mon[27953]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:28.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:28 smithi186 ceph-mon[31505]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:28.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:28 smithi083 ceph-mon[31413]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:28.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:28 smithi066 ceph-mon[27953]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:29.108 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T09:00:29.108 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:29.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T09:00:29.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:29.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T09:00:29.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:30.297 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:30 smithi083 ceph-mon[31413]: Deploying daemon osd.1 on smithi083 2024-04-21T09:00:30.297 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:30 smithi083 ceph-mon[31413]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:30.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:30 smithi186 ceph-mon[31505]: Deploying daemon osd.1 on smithi083 2024-04-21T09:00:30.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:30 smithi186 ceph-mon[31505]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:30.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:30 smithi066 ceph-mon[27953]: Deploying daemon osd.1 on smithi083 2024-04-21T09:00:30.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:30 smithi066 ceph-mon[27953]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:32.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:32 smithi186 ceph-mon[31505]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:32.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:32 smithi083 ceph-mon[31413]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:32.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:32 smithi066 ceph-mon[27953]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:34.223 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:34 smithi083 ceph-mon[31413]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:34.223 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:34 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:34.223 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:34 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:34.223 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:34 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:34.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:34 smithi186 ceph-mon[31505]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:34.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:34 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:34.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:34 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:34.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:34 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:34.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:34 smithi066 ceph-mon[27953]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:34.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:34 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:34.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:34 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:34.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:34 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:36.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:36 smithi186 ceph-mon[31505]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:36.357 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:36 smithi083 ceph-mon[31413]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:36.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:36 smithi066 ceph-mon[27953]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:37.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.264+0000 7f838ffff700 1 -- 172.21.15.83:0/582367105 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f83a80ff050 con 0x7f83900a3bb0 2024-04-21T09:00:37.264 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 1 on host 'smithi083' 2024-04-21T09:00:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f83900a3bb0 msgr2=0x7f83900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f83900a3bb0 0x7f83900a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f839800b580 tx=0x7f8398005fb0 comp rx=0 tx=0).stop 2024-04-21T09:00:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a8103570 msgr2=0x7f83a81a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a8103570 0x7f83a81a0090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83a0007d70 tx=0x7f83a000a340 comp rx=0 tx=0).stop 2024-04-21T09:00:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 shutdown_connections 2024-04-21T09:00:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f83900a3bb0 0x7f83900a6070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a8103570 0x7f83a81a0090 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a81019d0 0x7f83a8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 --2- 172.21.15.83:0/582367105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83a8102bd0 0x7f83a8199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 >> 172.21.15.83:0/582367105 conn(0x7f83a80fcf90 msgr2=0x7f83a80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:00:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 shutdown_connections 2024-04-21T09:00:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:00:37.266+0000 7f83ad2c8700 1 -- 172.21.15.83:0/582367105 wait complete. 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.565 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:37.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:37.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:37.910 DEBUG:teuthology.orchestra.run.smithi083:osd.1> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.1.service 2024-04-21T09:00:37.912 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2024-04-21T09:00:37.912 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T09:00:38.159 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:00:38.565 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:38 smithi186 ceph-mon[31505]: from='osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T09:00:38.565 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:38 smithi186 ceph-mon[31505]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T09:00:38.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:38 smithi083 ceph-mon[31413]: from='osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T09:00:38.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:38 smithi083 ceph-mon[31413]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T09:00:38.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:38 smithi066 ceph-mon[27953]: from='osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T09:00:38.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:38 smithi066 ceph-mon[27953]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T09:00:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:39 smithi186 ceph-mon[31505]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:39 smithi186 ceph-mon[31505]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T09:00:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:39 smithi186 ceph-mon[31505]: from='osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:00:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:39 smithi186 ceph-mon[31505]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T09:00:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:39.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:39 smithi186 ceph-mon[31505]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:00:39.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-mon[31413]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-mon[31413]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T09:00:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-mon[31413]: from='osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:00:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-mon[31413]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T09:00:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-mon[31413]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:00:39.860 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1[36037]: 2024-04-21T09:00:39.474+0000 7fae5f967700 -1 osd.1 0 waiting for initial osdmap 2024-04-21T09:00:39.860 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:00:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1[36037]: 2024-04-21T09:00:39.478+0000 7fae5b75a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:00:39.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:39 smithi066 ceph-mon[27953]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:39.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:39 smithi066 ceph-mon[27953]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T09:00:39.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:39 smithi066 ceph-mon[27953]: from='osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:00:39.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:39 smithi066 ceph-mon[27953]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T09:00:39.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:39.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:39 smithi066 ceph-mon[27953]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:00:40.009 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T09:00:40.009 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T09:00:40.009 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-21T09:00:40.009 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-21T09:00:40.009 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114168 s, 918 MB/s 2024-04-21T09:00:40.010 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-21T09:00:40.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:40 smithi186 ceph-mon[31505]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:00:40.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:40 smithi186 ceph-mon[31505]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T09:00:40.584 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.584 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.584 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.612 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-04-21T09:00:40.829 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:40 smithi083 ceph-mon[31413]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:00:40.829 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:40 smithi083 ceph-mon[31413]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T09:00:40.829 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.830 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.830 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.848 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:00:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:40 smithi066 ceph-mon[27953]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:00:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:40 smithi066 ceph-mon[27953]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T09:00:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:40.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] boot 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi083 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:00:41.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-21T09:00:41.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:41.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:41.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] boot 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi083 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:41.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:41.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: osd.1 [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] boot 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi083 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:00:41.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:42.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- 172.21.15.186:0/1149742638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 msgr2=0x7ff34c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:42.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 --2- 172.21.15.186:0/1149742638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c102eb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff348009990 tx=0x7ff348009ca0 comp rx=0 tx=0).stop 2024-04-21T09:00:42.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- 172.21.15.186:0/1149742638 shutdown_connections 2024-04-21T09:00:42.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 --2- 172.21.15.186:0/1149742638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff34c104630 0x7ff34c10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 --2- 172.21.15.186:0/1149742638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c102eb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 --2- 172.21.15.186:0/1149742638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34c103c90 0x7ff34c1040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- 172.21.15.186:0/1149742638 >> 172.21.15.186:0/1149742638 conn(0x7ff34c0fe030 msgr2=0x7ff34c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- 172.21.15.186:0/1149742638 shutdown_connections 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- 172.21.15.186:0/1149742638 wait complete. 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 Processor -- start 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- start start 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34c103c90 0x7ff34c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff34c104630 0x7ff34c1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10e500 con 0x7ff34c103c90 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10e380 con 0x7ff34c104630 2024-04-21T09:00:42.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.380+0000 7ff3534a5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10e680 con 0x7ff34c102a90 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:48292/0 (socket says 172.21.15.186:48292) 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 -- 172.21.15.186:0/2690515845 learned_addr learned my addr 172.21.15.186:0/2690515845 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351a42700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff34c104630 0x7ff34c1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff350a40700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34c103c90 0x7ff34c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 -- 172.21.15.186:0/2690515845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff34c104630 msgr2=0x7ff34c1a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff34c104630 0x7ff34c1a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 -- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34c103c90 msgr2=0x7ff34c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34c103c90 0x7ff34c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 -- 172.21.15.186:0/2690515845 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c1a2a50 con 0x7ff34c102a90 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff351241700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c19adf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff3480047b0 tx=0x7ff348009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff3427fc700 1 -- 172.21.15.186:0/2690515845 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348026070 con 0x7ff34c102a90 2024-04-21T09:00:42.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff3427fc700 1 -- 172.21.15.186:0/2690515845 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3480047e0 con 0x7ff34c102a90 2024-04-21T09:00:42.384 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c1a2ce0 con 0x7ff34c102a90 2024-04-21T09:00:42.384 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff3427fc700 1 -- 172.21.15.186:0/2690515845 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348005320 con 0x7ff34c102a90 2024-04-21T09:00:42.384 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.381+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a3130 con 0x7ff34c102a90 2024-04-21T09:00:42.385 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.382+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c069760 con 0x7ff34c102a90 2024-04-21T09:00:42.385 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.383+0000 7ff3427fc700 1 -- 172.21.15.186:0/2690515845 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff348029070 con 0x7ff34c102a90 2024-04-21T09:00:42.385 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.384+0000 7ff3427fc700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3380a3bb0 0x7ff3380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:00:42.385 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.384+0000 7ff3427fc700 1 -- 172.21.15.186:0/2690515845 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7ff3480cf660 con 0x7ff34c102a90 2024-04-21T09:00:42.385 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.384+0000 7ff350a40700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3380a3bb0 0x7ff3380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:00:42.385 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.384+0000 7ff350a40700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3380a3bb0 0x7ff3380a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff33c005950 tx=0x7ff33c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:00:42.388 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.388+0000 7ff3427fc700 1 -- 172.21.15.186:0/2690515845 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff34805dff0 con 0x7ff34c102a90 2024-04-21T09:00:42.592 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:00:42.593+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff34c10ab50 con 0x7ff3380a3bb0 2024-04-21T09:00:43.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:42 smithi186 ceph-mon[31505]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T09:00:43.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:42 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:00:43.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:42 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:00:43.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:42 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:43.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:42 smithi083 ceph-mon[31413]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T09:00:43.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:42 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:00:43.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:42 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:00:43.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:42 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:43.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:42 smithi066 ceph-mon[27953]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T09:00:43.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:42 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:00:43.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:42 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:00:43.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:42 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:44.040 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:43 smithi186 ceph-mon[31505]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:44.041 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:43 smithi186 ceph-mon[31505]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:00:44.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:43 smithi083 ceph-mon[31413]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:43 smithi083 ceph-mon[31413]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:00:44.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:43 smithi066 ceph-mon[27953]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:44.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:43 smithi066 ceph-mon[27953]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:00:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:45 smithi186 ceph-mon[31505]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:45 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/2438879117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]: dispatch 2024-04-21T09:00:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:45 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]: dispatch 2024-04-21T09:00:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:45 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]': finished 2024-04-21T09:00:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:45 smithi186 ceph-mon[31505]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T09:00:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:00:46.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:45 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/3156436094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:00:46.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:45 smithi083 ceph-mon[31413]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:46.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:45 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/2438879117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]: dispatch 2024-04-21T09:00:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:45 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]: dispatch 2024-04-21T09:00:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:45 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]': finished 2024-04-21T09:00:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:45 smithi083 ceph-mon[31413]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T09:00:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:00:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:45 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/3156436094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:00:46.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:45 smithi066 ceph-mon[27953]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:46.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:45 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/2438879117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]: dispatch 2024-04-21T09:00:46.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:45 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]: dispatch 2024-04-21T09:00:46.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:45 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0462467-e4ef-4fc1-8f53-5ce7742ff9af"}]': finished 2024-04-21T09:00:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:45 smithi066 ceph-mon[27953]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T09:00:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:00:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:45 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/3156436094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:00:48.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:47 smithi186 ceph-mon[31505]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:48.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:47 smithi083 ceph-mon[31413]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:48.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:47 smithi066 ceph-mon[27953]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:50.235 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:49 smithi186 ceph-mon[31505]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:50.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:49 smithi083 ceph-mon[31413]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:50.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:49 smithi066 ceph-mon[27953]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:52.254 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:51 smithi186 ceph-mon[31505]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:52.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:51 smithi083 ceph-mon[31413]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:52.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:51 smithi066 ceph-mon[27953]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:53.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:52 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T09:00:53.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:52 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:53.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:52 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T09:00:53.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:52 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:52 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T09:00:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:52 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:00:54.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:53 smithi186 ceph-mon[31505]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:54.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:53 smithi186 ceph-mon[31505]: Deploying daemon osd.2 on smithi186 2024-04-21T09:00:54.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:53 smithi083 ceph-mon[31413]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:54.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:53 smithi083 ceph-mon[31413]: Deploying daemon osd.2 on smithi186 2024-04-21T09:00:54.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:53 smithi066 ceph-mon[27953]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:54.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:53 smithi066 ceph-mon[27953]: Deploying daemon osd.2 on smithi186 2024-04-21T09:00:56.180 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:55 smithi186 ceph-mon[31505]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:56.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:55 smithi083 ceph-mon[31413]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:56.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:55 smithi066 ceph-mon[27953]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:58.236 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:57 smithi186 ceph-mon[31505]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:58.236 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:58.236 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:58.237 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:58.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:57 smithi083 ceph-mon[31413]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:58.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:58.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:58.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:00:58.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:57 smithi066 ceph-mon[27953]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:00:58.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:58.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:00:58.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:00.284 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:00:59 smithi186 ceph-mon[31505]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:00.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:00:59 smithi083 ceph-mon[31413]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:00.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:00:59 smithi066 ceph-mon[27953]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:01.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:01 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:01 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:01 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:01.361 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:01 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:01.361 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:01 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:01 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:01 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:01 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:01.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:01 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:01.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:01 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:01 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:01 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:01 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:01.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:01 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:01.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:01 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:01.516 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.516+0000 7ff3427fc700 1 -- 172.21.15.186:0/2690515845 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff34c10ab50 con 0x7ff3380a3bb0 2024-04-21T09:01:01.516 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2024-04-21T09:01:01.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3380a3bb0 msgr2=0x7ff3380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:01.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3380a3bb0 0x7ff3380a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff33c005950 tx=0x7ff33c0058e0 comp rx=0 tx=0).stop 2024-04-21T09:01:01.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 msgr2=0x7ff34c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:01.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c19adf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff3480047b0 tx=0x7ff348009580 comp rx=0 tx=0).stop 2024-04-21T09:01:01.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 shutdown_connections 2024-04-21T09:01:01.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3380a3bb0 0x7ff3380a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:01.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff34c104630 0x7ff34c1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:01.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34c102a90 0x7ff34c19adf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:01.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 --2- 172.21.15.186:0/2690515845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34c103c90 0x7ff34c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:01.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 >> 172.21.15.186:0/2690515845 conn(0x7ff34c0fe030 msgr2=0x7ff34c109430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:01:01.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 shutdown_connections 2024-04-21T09:01:01.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:01.518+0000 7ff3534a5700 1 -- 172.21.15.186:0/2690515845 wait complete. 2024-04-21T09:01:02.182 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.2.service 2024-04-21T09:01:02.184 INFO:tasks.cephadm:Deploying osd.3 on smithi066 with /dev/vg_nvme/lv_3... 2024-04-21T09:01:02.184 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T09:01:02.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:02 smithi066 ceph-mon[27953]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:02.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:02 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:02.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:02 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:02.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:02 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:02.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:02 smithi066 ceph-mon[27953]: from='osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T09:01:02.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:02 smithi066 ceph-mon[27953]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T09:01:02.383 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:02 smithi186 ceph-mon[31505]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:02.383 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:02 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:02.383 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:02 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:02.383 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:02 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:02.383 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:02 smithi186 ceph-mon[31505]: from='osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T09:01:02.383 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:02 smithi186 ceph-mon[31505]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T09:01:02.443 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:01:02.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:02 smithi083 ceph-mon[31413]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:02 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:02 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:02 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:02 smithi083 ceph-mon[31413]: from='osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T09:01:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:02 smithi083 ceph-mon[31413]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T09:01:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-mon[31505]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-mon[31505]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T09:01:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-mon[31505]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T09:01:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-mon[31505]: from='osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:01:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-mon[31505]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:01:03.589 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2[36820]: 2024-04-21T09:01:03.513+0000 7febfe022700 -1 osd.2 0 waiting for initial osdmap 2024-04-21T09:01:03.589 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:03 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2[36820]: 2024-04-21T09:01:03.518+0000 7febf9e15700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:01:03.849 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:03 smithi066 ceph-mon[27953]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:03.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:03 smithi066 ceph-mon[27953]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T09:01:03.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:03 smithi066 ceph-mon[27953]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T09:01:03.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:03 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:03.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:03 smithi066 ceph-mon[27953]: from='osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:01:03.851 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:03 smithi066 ceph-mon[27953]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:01:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:03 smithi083 ceph-mon[31413]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:03 smithi083 ceph-mon[31413]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T09:01:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:03 smithi083 ceph-mon[31413]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T09:01:03.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:03 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:03.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:03 smithi083 ceph-mon[31413]: from='osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:01:03.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:03 smithi083 ceph-mon[31413]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:01:04.761 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T09:01:04.761 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T09:01:04.761 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-21T09:01:04.761 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-21T09:01:04.761 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116297 s, 902 MB/s 2024-04-21T09:01:04.761 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-21T09:01:04.762 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:04 smithi066 ceph-mon[27953]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:01:04.762 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:04 smithi066 ceph-mon[27953]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T09:01:04.762 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:04 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:04.762 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:04 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:04.762 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:04 smithi186 ceph-mon[31505]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:01:04.762 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:04 smithi186 ceph-mon[31505]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T09:01:04.763 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:04 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:04.763 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:04 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:04.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:04 smithi083 ceph-mon[31413]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:01:04.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:04 smithi083 ceph-mon[31413]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T09:01:04.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:04 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:04.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:04 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:05.396 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] boot 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:05.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:05.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:05.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.641 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] boot 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:05.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:05.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:05.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: osd.2 [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] boot 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:05.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:05.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:05.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:05.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:06.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:06 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi186 2024-04-21T09:01:06.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:06 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-21T09:01:06.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:06 smithi066 ceph-mon[27953]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T09:01:06.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:01:06.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:06 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi186 2024-04-21T09:01:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:06 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-21T09:01:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:06 smithi186 ceph-mon[31505]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T09:01:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:01:06.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:06 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi186 2024-04-21T09:01:06.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:06 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-21T09:01:06.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:06 smithi083 ceph-mon[31413]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T09:01:06.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:01:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 -- 172.21.15.66:0/1856610809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a81035b0 msgr2=0x7f87a81059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/1856610809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a81035b0 0x7f87a81059a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009990 tx=0x7f87a4009ca0 comp rx=0 tx=0).stop 2024-04-21T09:01:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 -- 172.21.15.66:0/1856610809 shutdown_connections 2024-04-21T09:01:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/1856610809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a8108810 0x7f87a810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/1856610809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a81035b0 0x7f87a81059a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/1856610809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87a8105ee0 0x7f87a81082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 -- 172.21.15.66:0/1856610809 >> 172.21.15.66:0/1856610809 conn(0x7f87a80fcf60 msgr2=0x7f87a80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 -- 172.21.15.66:0/1856610809 shutdown_connections 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.316+0000 7f87b0768700 1 -- 172.21.15.66:0/1856610809 wait complete. 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 Processor -- start 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 -- start start 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87a81035b0 0x7f87a8102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a8105ee0 0x7f87a8101560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a8108810 0x7f87a8101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810e550 con 0x7f87a8108810 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810e3d0 con 0x7f87a81035b0 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a8108810 0x7f87a8101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87b0768700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810e6d0 con 0x7f87a8105ee0 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a8108810 0x7f87a8101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47992/0 (socket says 172.21.15.66:47992) 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 -- 172.21.15.66:0/554931957 learned_addr learned my addr 172.21.15.66:0/554931957 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87ae504700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87a81035b0 0x7f87a8102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87add03700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a8105ee0 0x7f87a8101560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 -- 172.21.15.66:0/554931957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a8105ee0 msgr2=0x7f87a8101560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a8105ee0 0x7f87a8101560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 -- 172.21.15.66:0/554931957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87a81035b0 msgr2=0x7f87a8102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87a81035b0 0x7f87a8102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.317+0000 7f87aed05700 1 -- 172.21.15.66:0/554931957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a8102350 con 0x7f87a8108810 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.318+0000 7f87ae504700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87a81035b0 0x7f87a8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.318+0000 7f87aed05700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a8108810 0x7f87a8101aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87a0007d40 tx=0x7f87a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.318+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0016020 con 0x7f87a8108810 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.318+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a000ac90 con 0x7f87a8108810 2024-04-21T09:01:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.318+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a00115e0 con 0x7f87a8108810 2024-04-21T09:01:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.318+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a81a2cf0 con 0x7f87a8108810 2024-04-21T09:01:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.318+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a81a31b0 con 0x7f87a8108810 2024-04-21T09:01:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.319+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a8069730 con 0x7f87a8108810 2024-04-21T09:01:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.320+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f87a0019020 con 0x7f87a8108810 2024-04-21T09:01:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.321+0000 7f879f7fe700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87940a3c30 0x7f87940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.321+0000 7f87ae504700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87940a3c30 0x7f87940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.321+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f87a00cf2a0 con 0x7f87a8108810 2024-04-21T09:01:07.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.321+0000 7f87ae504700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87940a3c30 0x7f87940a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87a400b5c0 tx=0x7f87a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:01:07.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.325+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87a0020020 con 0x7f87a8108810 2024-04-21T09:01:07.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:07.544+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f87a8062e80 con 0x7f87940a3c30 2024-04-21T09:01:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 ceph-mon[27953]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:07.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T09:01:07.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 ceph-mon[27953]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T09:01:07.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:01:07.791 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T09:01:07.791 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:07.791 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40719]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:07.791 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40719]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:07.792 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 ceph-mon[31505]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:07.792 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T09:01:07.792 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 ceph-mon[31505]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T09:01:07.792 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:01:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 ceph-mon[31413]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T09:01:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 ceph-mon[31413]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T09:01:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:01:07.860 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40161]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T09:01:07.860 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40161]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:07.860 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40161]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:07.860 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40161]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:07.861 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40164]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T09:01:07.861 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:07.861 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40164]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:07.861 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40164]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.088 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T09:01:08.088 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.088 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40722]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.088 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40722]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T09:01:08.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40725]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40725]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40728]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T09:01:08.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40728]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:07 smithi186 sudo[40728]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.119 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45132]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T09:01:08.120 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45132]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.120 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45132]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.120 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45132]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.120 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45135]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T09:01:08.120 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45135]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.120 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45135]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.120 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45135]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45138]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T09:01:08.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45138]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45138]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45141]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T09:01:08.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45141]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45141]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:07 smithi066 sudo[45141]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T09:01:08.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40167]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40167]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40170]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T09:01:08.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T09:01:08.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40170]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T09:01:08.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:07 smithi083 sudo[40170]: pam_unix(sudo:session): session closed for user root 2024-04-21T09:01:08.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:08.835+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f87a00990b0 con 0x7f87a8108810 2024-04-21T09:01:08.836 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T09:01:08.836 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.837 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:08 smithi066 ceph-mon[27953]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.840 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:08 smithi186 ceph-mon[31505]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:01:08.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:08 smithi083 ceph-mon[31413]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T09:01:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:09 smithi186 ceph-mon[31505]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:09 smithi186 ceph-mon[31505]: osdmap e22: 3 total, 3 up, 3 in 2024-04-21T09:01:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:09 smithi186 ceph-mon[31505]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T09:01:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:09 smithi083 ceph-mon[31413]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:09.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:09 smithi083 ceph-mon[31413]: osdmap e22: 3 total, 3 up, 3 in 2024-04-21T09:01:09.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:09 smithi083 ceph-mon[31413]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T09:01:09.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:09 smithi066 ceph-mon[27953]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:09.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:09 smithi066 ceph-mon[27953]: osdmap e22: 3 total, 3 up, 3 in 2024-04-21T09:01:09.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:09 smithi066 ceph-mon[27953]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T09:01:10.835 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1267173408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6c91e9b-d431-49b2-8b56-562e85e16010"}]: dispatch 2024-04-21T09:01:10.835 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1267173408' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6c91e9b-d431-49b2-8b56-562e85e16010"}]': finished 2024-04-21T09:01:10.835 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:10 smithi066 ceph-mon[27953]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T09:01:10.835 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:10.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1267173408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6c91e9b-d431-49b2-8b56-562e85e16010"}]: dispatch 2024-04-21T09:01:10.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1267173408' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6c91e9b-d431-49b2-8b56-562e85e16010"}]': finished 2024-04-21T09:01:10.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:10 smithi186 ceph-mon[31505]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T09:01:10.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:10.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1267173408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6c91e9b-d431-49b2-8b56-562e85e16010"}]: dispatch 2024-04-21T09:01:10.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1267173408' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6c91e9b-d431-49b2-8b56-562e85e16010"}]': finished 2024-04-21T09:01:10.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:10 smithi083 ceph-mon[31413]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T09:01:10.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:11 smithi186 ceph-mon[31505]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/695306311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:01:11.862 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:11 smithi083 ceph-mon[31413]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:11.862 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/695306311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:01:11.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:11 smithi066 ceph-mon[27953]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:11.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/695306311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:01:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:13 smithi186 ceph-mon[31505]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:13.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:13 smithi083 ceph-mon[31413]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:13.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:13 smithi066 ceph-mon[27953]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:15.765 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:15 smithi066 ceph-mon[27953]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:15.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:15 smithi186 ceph-mon[31505]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:15 smithi083 ceph-mon[31413]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:17 smithi186 ceph-mon[31505]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:17.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:17 smithi066 ceph-mon[27953]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:17 smithi083 ceph-mon[31413]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:18.769 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T09:01:18.770 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:18.770 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:18 smithi066 ceph-mon[27953]: Deploying daemon osd.3 on smithi066 2024-04-21T09:01:18.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:18 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T09:01:18.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:18 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:18.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:18 smithi186 ceph-mon[31505]: Deploying daemon osd.3 on smithi066 2024-04-21T09:01:18.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:18 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T09:01:18.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:18 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:18.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:18 smithi083 ceph-mon[31413]: Deploying daemon osd.3 on smithi066 2024-04-21T09:01:19.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:19 smithi066 ceph-mon[27953]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:19.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:19 smithi186 ceph-mon[31505]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:19.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:19 smithi083 ceph-mon[31413]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:21.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:21 smithi186 ceph-mon[31505]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:21.856 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:21 smithi066 ceph-mon[27953]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:21.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:21 smithi083 ceph-mon[31413]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:23.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:23 smithi066 ceph-mon[27953]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:23 smithi186 ceph-mon[31505]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:23 smithi083 ceph-mon[31413]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:25.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:25 smithi066 ceph-mon[27953]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:25.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:25 smithi186 ceph-mon[31505]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:25.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:25 smithi083 ceph-mon[31413]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:27.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.314+0000 7f879f7fe700 1 -- 172.21.15.66:0/554931957 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87a8062e80 con 0x7f87940a3c30 2024-04-21T09:01:27.316 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 3 on host 'smithi066' 2024-04-21T09:01:27.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87940a3c30 msgr2=0x7f87940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:27.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87940a3c30 0x7f87940a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87a400b5c0 tx=0x7f87a4005fb0 comp rx=0 tx=0).stop 2024-04-21T09:01:27.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a8108810 msgr2=0x7f87a8101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:27.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a8108810 0x7f87a8101aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87a0007d40 tx=0x7f87a000a340 comp rx=0 tx=0).stop 2024-04-21T09:01:27.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 shutdown_connections 2024-04-21T09:01:27.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87940a3c30 0x7f87940a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:27.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87a81035b0 0x7f87a8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:27.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87a8108810 0x7f87a8101aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:27.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 --2- 172.21.15.66:0/554931957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a8105ee0 0x7f87a8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:27.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 >> 172.21.15.66:0/554931957 conn(0x7f87a80fcf60 msgr2=0x7f87a8103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:01:27.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 shutdown_connections 2024-04-21T09:01:27.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:01:27.316+0000 7f87b0768700 1 -- 172.21.15.66:0/554931957 wait complete. 2024-04-21T09:01:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:27 smithi066 ceph-mon[27953]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:27 smithi186 ceph-mon[31505]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T09:01:27.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:27.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:27.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:27.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:27.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:27.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:27 smithi083 ceph-mon[31413]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T09:01:27.943 DEBUG:teuthology.orchestra.run.smithi066:osd.3> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.3.service 2024-04-21T09:01:27.946 INFO:tasks.cephadm:Deploying osd.4 on smithi083 with /dev/vg_nvme/lv_3... 2024-04-21T09:01:27.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T09:01:28.201 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:01:29.431 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:29 smithi066 ceph-mon[27953]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:29.431 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:29 smithi066 ceph-mon[27953]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T09:01:29.431 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:29 smithi066 ceph-mon[27953]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T09:01:29.431 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:29 smithi066 ceph-mon[27953]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:01:29.431 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:29.431 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:01:29 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:01:29.302+0000 7f0e31e00700 -1 osd.3 0 waiting for initial osdmap 2024-04-21T09:01:29.432 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:01:29 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:01:29.310+0000 7f0e2dbf3700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:01:29.566 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:29 smithi083 ceph-mon[31413]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:29.566 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:29 smithi083 ceph-mon[31413]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T09:01:29.566 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:29 smithi083 ceph-mon[31413]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T09:01:29.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:29 smithi083 ceph-mon[31413]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:01:29.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:29.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:29 smithi186 ceph-mon[31505]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:29.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:29 smithi186 ceph-mon[31505]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T09:01:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:29 smithi186 ceph-mon[31505]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T09:01:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:29 smithi186 ceph-mon[31505]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:01:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.161 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T09:01:30.162 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T09:01:30.162 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-21T09:01:30.162 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-21T09:01:30.162 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115077 s, 911 MB/s 2024-04-21T09:01:30.162 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-21T09:01:30.464 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:30 smithi083 ceph-mon[31413]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:01:30.464 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:30 smithi083 ceph-mon[31413]: osdmap e25: 4 total, 3 up, 4 in 2024-04-21T09:01:30.464 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:30 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.464 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:30 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.464 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:30 smithi083 ceph-mon[31413]: osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] boot 2024-04-21T09:01:30.464 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:30 smithi083 ceph-mon[31413]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T09:01:30.464 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:30 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:30 smithi186 ceph-mon[31505]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:01:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:30 smithi186 ceph-mon[31505]: osdmap e25: 4 total, 3 up, 4 in 2024-04-21T09:01:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:30 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:30 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:30 smithi186 ceph-mon[31505]: osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] boot 2024-04-21T09:01:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:30 smithi186 ceph-mon[31505]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T09:01:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:30 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:30 smithi066 ceph-mon[27953]: from='osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:01:30.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:30 smithi066 ceph-mon[27953]: osdmap e25: 4 total, 3 up, 4 in 2024-04-21T09:01:30.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:30 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:30 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:30 smithi066 ceph-mon[27953]: osd.3 [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] boot 2024-04-21T09:01:30.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:30 smithi066 ceph-mon[27953]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T09:01:30.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:30 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:01:30.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-04-21T09:01:31.143 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi066 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:31.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:31.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi066 to 8544M 2024-04-21T09:01:31.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:31.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:31.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:31 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi066 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi066 to 8544M 2024-04-21T09:01:31.603 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.604 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:31.604 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:31.604 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:31 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi066 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi066 to 8544M 2024-04-21T09:01:31.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:31.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:31.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:31.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:31 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 -- 172.21.15.83:0/2935002880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18103ca0 msgr2=0x7fab18104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 --2- 172.21.15.83:0/2935002880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18103ca0 0x7fab18104100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0099c0 tx=0x7fab0c009cd0 comp rx=0 tx=0).stop 2024-04-21T09:01:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 -- 172.21.15.83:0/2935002880 shutdown_connections 2024-04-21T09:01:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 --2- 172.21.15.83:0/2935002880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab18104640 0x7fab1810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 --2- 172.21.15.83:0/2935002880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab18102aa0 0x7fab18102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 --2- 172.21.15.83:0/2935002880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18103ca0 0x7fab18104100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 -- 172.21.15.83:0/2935002880 >> 172.21.15.83:0/2935002880 conn(0x7fab180fe040 msgr2=0x7fab18100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 -- 172.21.15.83:0/2935002880 shutdown_connections 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 -- 172.21.15.83:0/2935002880 wait complete. 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 Processor -- start 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.731+0000 7fab1df56700 1 -- start start 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18102aa0 0x7fab18198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab18103ca0 0x7fab181990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab18104640 0x7fab181a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810e510 con 0x7fab18103ca0 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810e390 con 0x7fab18102aa0 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810e690 con 0x7fab18104640 2024-04-21T09:01:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18102aa0 0x7fab18198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18102aa0 0x7fab18198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51796/0 (socket says 172.21.15.83:51796) 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 -- 172.21.15.83:0/445121677 learned_addr learned my addr 172.21.15.83:0/445121677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab16ffd700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab18103ca0 0x7fab181990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab17fff700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab18104640 0x7fab181a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 -- 172.21.15.83:0/445121677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab18104640 msgr2=0x7fab181a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab18104640 0x7fab181a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 -- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab18103ca0 msgr2=0x7fab181990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab18103ca0 0x7fab181990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 -- 172.21.15.83:0/445121677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab181a07c0 con 0x7fab18102aa0 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab177fe700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18102aa0 0x7fab18198b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d980 tx=0x7fab0800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab14ff9700 1 -- 172.21.15.83:0/445121677 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08017030 con 0x7fab18102aa0 2024-04-21T09:01:32.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab14ff9700 1 -- 172.21.15.83:0/445121677 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab080044d0 con 0x7fab18102aa0 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab181a0ab0 con 0x7fab18102aa0 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab14ff9700 1 -- 172.21.15.83:0/445121677 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080039e0 con 0x7fab18102aa0 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.732+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab181a0ff0 con 0x7fab18102aa0 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.733+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab18069730 con 0x7fab18102aa0 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.734+0000 7fab14ff9700 1 -- 172.21.15.83:0/445121677 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab0801a070 con 0x7fab18102aa0 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.735+0000 7fab14ff9700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fab000a3c40 0x7fab000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.735+0000 7fab14ff9700 1 -- 172.21.15.83:0/445121677 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fab080ceb00 con 0x7fab18102aa0 2024-04-21T09:01:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.735+0000 7fab16ffd700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fab000a3c40 0x7fab000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:32.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.736+0000 7fab16ffd700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fab000a3c40 0x7fab000a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009990 tx=0x7fab0c00b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:01:32.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.740+0000 7fab14ff9700 1 -- 172.21.15.83:0/445121677 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab0809c700 con 0x7fab18102aa0 2024-04-21T09:01:32.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:32 smithi083 ceph-mon[31413]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T09:01:32.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:32 smithi066 ceph-mon[27953]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T09:01:32.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:32.945+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fab18062e80 con 0x7fab000a3c40 2024-04-21T09:01:33.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:32 smithi186 ceph-mon[31505]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T09:01:33.842 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:33 smithi083 ceph-mon[31413]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:33.842 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:33 smithi083 ceph-mon[31413]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T09:01:33.842 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:33 smithi083 ceph-mon[31413]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:33.842 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:33.842 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:33.842 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:33.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:33 smithi066 ceph-mon[27953]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:33.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:33 smithi066 ceph-mon[27953]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T09:01:33.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:33 smithi066 ceph-mon[27953]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:33.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:33.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:33.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:34.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:33 smithi186 ceph-mon[31505]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:34.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:33 smithi186 ceph-mon[31505]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T09:01:34.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:33 smithi186 ceph-mon[31505]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:34.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:34.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:34.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:34.868 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:34 smithi083 ceph-mon[31413]: osdmap e30: 4 total, 4 up, 4 in 2024-04-21T09:01:34.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:34 smithi066 ceph-mon[27953]: osdmap e30: 4 total, 4 up, 4 in 2024-04-21T09:01:35.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:34 smithi186 ceph-mon[31505]: osdmap e30: 4 total, 4 up, 4 in 2024-04-21T09:01:36.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:35 smithi186 ceph-mon[31505]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:36.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:35 smithi083 ceph-mon[31413]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:36.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:35 smithi066 ceph-mon[27953]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:36 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/467837956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]: dispatch 2024-04-21T09:01:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:36 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]: dispatch 2024-04-21T09:01:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:36 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]': finished 2024-04-21T09:01:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:36 smithi186 ceph-mon[31505]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T09:01:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:36 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/3411653906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:01:37.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:36 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/467837956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]: dispatch 2024-04-21T09:01:37.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:36 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]: dispatch 2024-04-21T09:01:37.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:36 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]': finished 2024-04-21T09:01:37.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:36 smithi083 ceph-mon[31413]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T09:01:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:36 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/3411653906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:01:37.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:36 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/467837956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]: dispatch 2024-04-21T09:01:37.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:36 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]: dispatch 2024-04-21T09:01:37.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:36 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46"}]': finished 2024-04-21T09:01:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:36 smithi066 ceph-mon[27953]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T09:01:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:36 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/3411653906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:01:38.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:37 smithi186 ceph-mon[31505]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:38.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:37 smithi083 ceph-mon[31413]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:38.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:37 smithi066 ceph-mon[27953]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:40.275 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:39 smithi083 ceph-mon[31413]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:01:40.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:39 smithi186 ceph-mon[31505]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:01:40.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:39 smithi066 ceph-mon[27953]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:01:42.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:41 smithi083 ceph-mon[31413]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-21T09:01:42.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:41 smithi186 ceph-mon[31505]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-21T09:01:42.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:41 smithi066 ceph-mon[27953]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-21T09:01:44.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:43 smithi083 ceph-mon[31413]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-21T09:01:44.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T09:01:44.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:44.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:43 smithi186 ceph-mon[31505]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-21T09:01:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T09:01:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:44.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:43 smithi066 ceph-mon[27953]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-21T09:01:44.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T09:01:44.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:45.202 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:44 smithi083 ceph-mon[31413]: Deploying daemon osd.4 on smithi083 2024-04-21T09:01:45.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:44 smithi186 ceph-mon[31505]: Deploying daemon osd.4 on smithi083 2024-04-21T09:01:45.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:44 smithi066 ceph-mon[27953]: Deploying daemon osd.4 on smithi083 2024-04-21T09:01:46.034 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:45 smithi083 ceph-mon[31413]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:01:46.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:45 smithi186 ceph-mon[31505]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:01:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:45 smithi066 ceph-mon[27953]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:01:48.102 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:47 smithi083 ceph-mon[31413]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-21T09:01:48.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:47 smithi186 ceph-mon[31505]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-21T09:01:48.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:47 smithi066 ceph-mon[27953]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-21T09:01:50.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:50 smithi066 ceph-mon[27953]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-21T09:01:50.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:50.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:50.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:50.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:50 smithi186 ceph-mon[31505]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-21T09:01:50.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:50 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:50.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:50 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:50.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:50 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:50.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:50 smithi083 ceph-mon[31413]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-21T09:01:50.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:50 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:50.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:50 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:50.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:50 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:52.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:52 smithi083 ceph-mon[31413]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:52.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:52 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:52 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:52 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:52.324 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:52 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:52.324 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:52 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:52 smithi066 ceph-mon[27953]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:52.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:52 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:52 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:52 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:52.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:52 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:52.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:52 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:52 smithi186 ceph-mon[31505]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:52.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:52 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:52 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:52.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:52 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:52.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:52 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:52.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:52 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:53.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.065+0000 7fab14ff9700 1 -- 172.21.15.83:0/445121677 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fab18062e80 con 0x7fab000a3c40 2024-04-21T09:01:53.065 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 4 on host 'smithi083' 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fab000a3c40 msgr2=0x7fab000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fab000a3c40 0x7fab000a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009990 tx=0x7fab0c00b870 comp rx=0 tx=0).stop 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18102aa0 msgr2=0x7fab18198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18102aa0 0x7fab18198b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d980 tx=0x7fab0800dc90 comp rx=0 tx=0).stop 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 shutdown_connections 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fab000a3c40 0x7fab000a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab18102aa0 0x7fab18198b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab18104640 0x7fab181a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:53.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 --2- 172.21.15.83:0/445121677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab18103ca0 0x7fab181990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:53.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 >> 172.21.15.83:0/445121677 conn(0x7fab180fe040 msgr2=0x7fab18107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:01:53.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 shutdown_connections 2024-04-21T09:01:53.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:01:53.067+0000 7fab1df56700 1 -- 172.21.15.83:0/445121677 wait complete. 2024-04-21T09:01:53.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:53 smithi083 ceph-mon[31413]: from='osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T09:01:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:53 smithi083 ceph-mon[31413]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T09:01:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:53.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:53 smithi186 ceph-mon[31505]: from='osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T09:01:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:53 smithi186 ceph-mon[31505]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T09:01:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:53 smithi066 ceph-mon[27953]: from='osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T09:01:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:53 smithi066 ceph-mon[27953]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T09:01:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:01:53.698 DEBUG:teuthology.orchestra.run.smithi083:osd.4> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.4.service 2024-04-21T09:01:53.700 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2024-04-21T09:01:53.701 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T09:01:53.944 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:01:54.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-mon[31413]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:54.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-mon[31413]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T09:01:54.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-mon[31413]: from='osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:01:54.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-mon[31413]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T09:01:54.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:54.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-mon[31413]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:01:54.193 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4[42993]: 2024-04-21T09:01:54.139+0000 7f47f1fa9700 -1 osd.4 0 waiting for initial osdmap 2024-04-21T09:01:54.194 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:01:54 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4[42993]: 2024-04-21T09:01:54.144+0000 7f47edd9c700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:01:54.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:54 smithi186 ceph-mon[31505]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:54 smithi186 ceph-mon[31505]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T09:01:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:54 smithi186 ceph-mon[31505]: from='osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:01:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:54 smithi186 ceph-mon[31505]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T09:01:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:54 smithi186 ceph-mon[31505]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:01:54.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:54 smithi066 ceph-mon[27953]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:54.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:54 smithi066 ceph-mon[27953]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T09:01:54.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:54 smithi066 ceph-mon[27953]: from='osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:01:54.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:54 smithi066 ceph-mon[27953]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T09:01:54.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:54.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:54 smithi066 ceph-mon[27953]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:01:55.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:55 smithi083 ceph-mon[31413]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:01:55.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:55 smithi083 ceph-mon[31413]: osdmap e33: 5 total, 4 up, 5 in 2024-04-21T09:01:55.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:55 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:55.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:55 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:55.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:55 smithi083 ceph-mon[31413]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:55.398 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:55 smithi186 ceph-mon[31505]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:01:55.398 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:55 smithi186 ceph-mon[31505]: osdmap e33: 5 total, 4 up, 5 in 2024-04-21T09:01:55.398 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:55 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:55.398 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:55 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:55.398 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:55 smithi186 ceph-mon[31505]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:55.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:55 smithi066 ceph-mon[27953]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:01:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:55 smithi066 ceph-mon[27953]: osdmap e33: 5 total, 4 up, 5 in 2024-04-21T09:01:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:55 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:55 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:55 smithi066 ceph-mon[27953]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-21T09:01:55.972 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T09:01:55.972 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T09:01:55.972 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-21T09:01:55.972 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-21T09:01:55.972 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113945 s, 920 MB/s 2024-04-21T09:01:55.972 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-21T09:01:56.445 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:56 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:01:56.446 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:56 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:01:56.446 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:56 smithi083 ceph-mon[31413]: osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] boot 2024-04-21T09:01:56.446 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:56 smithi083 ceph-mon[31413]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T09:01:56.446 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:56 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:56 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:01:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:56 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:01:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:56 smithi186 ceph-mon[31505]: osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] boot 2024-04-21T09:01:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:56 smithi186 ceph-mon[31505]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T09:01:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:56 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:56 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:01:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:56 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:01:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:56 smithi066 ceph-mon[27953]: osd.4 [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] boot 2024-04-21T09:01:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:56 smithi066 ceph-mon[27953]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T09:01:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:56 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:01:56.663 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-04-21T09:01:56.898 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: osdmap e35: 5 total, 5 up, 5 in 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi083 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:57.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:57.321 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: osdmap e35: 5 total, 5 up, 5 in 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi083 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: osdmap e35: 5 total, 5 up, 5 in 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi083 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:01:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:01:58.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 -- 172.21.15.186:0/3771163273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc102ba0 msgr2=0x7f67cc103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 --2- 172.21.15.186:0/3771163273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc102ba0 0x7f67cc103000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009990 tx=0x7f67bc009ca0 comp rx=0 tx=0).stop 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 -- 172.21.15.186:0/3771163273 shutdown_connections 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 --2- 172.21.15.186:0/3771163273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67cc1019a0 0x7f67cc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 --2- 172.21.15.186:0/3771163273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67cc103540 0x7f67cc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 --2- 172.21.15.186:0/3771163273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc102ba0 0x7f67cc103000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 -- 172.21.15.186:0/3771163273 >> 172.21.15.186:0/3771163273 conn(0x7f67cc0fcf60 msgr2=0x7f67cc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 -- 172.21.15.186:0/3771163273 shutdown_connections 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.491+0000 7f67d1426700 1 -- 172.21.15.186:0/3771163273 wait complete. 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 Processor -- start 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 -- start start 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67cc1019a0 0x7f67cc105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:58.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67cc102ba0 0x7f67cc103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc103540 0x7f67cc1044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc10d4b0 con 0x7f67cc102ba0 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc10d330 con 0x7f67cc1019a0 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67d1426700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc10d630 con 0x7f67cc103540 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67cb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc103540 0x7f67cc1044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67cb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc103540 0x7f67cc1044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:60240/0 (socket says 172.21.15.186:60240) 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67cb7fe700 1 -- 172.21.15.186:0/3459942377 learned_addr learned my addr 172.21.15.186:0/3459942377 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67ca7fc700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67cc102ba0 0x7f67cc103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.492+0000 7f67caffd700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67cc1019a0 0x7f67cc105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67cb7fe700 1 -- 172.21.15.186:0/3459942377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67cc1019a0 msgr2=0x7f67cc105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67cb7fe700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67cc1019a0 0x7f67cc105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67cb7fe700 1 -- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67cc102ba0 msgr2=0x7f67cc103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:01:58.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67cb7fe700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67cc102ba0 0x7f67cc103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:01:58.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67cb7fe700 1 -- 172.21.15.186:0/3459942377 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc104d70 con 0x7f67cc103540 2024-04-21T09:01:58.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67cb7fe700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc103540 0x7f67cc1044c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67c4007ee0 tx=0x7f67c400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:01:58.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67b3fff700 1 -- 172.21.15.186:0/3459942377 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400ddd0 con 0x7f67cc103540 2024-04-21T09:01:58.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc1a2e00 con 0x7f67cc103540 2024-04-21T09:01:58.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67b3fff700 1 -- 172.21.15.186:0/3459942377 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c4004020 con 0x7f67cc103540 2024-04-21T09:01:58.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67b3fff700 1 -- 172.21.15.186:0/3459942377 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4015660 con 0x7f67cc103540 2024-04-21T09:01:58.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.493+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc1a3260 con 0x7f67cc103540 2024-04-21T09:01:58.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.494+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67cc069730 con 0x7f67cc103540 2024-04-21T09:01:58.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.495+0000 7f67b3fff700 1 -- 172.21.15.186:0/3459942377 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67c40041d0 con 0x7f67cc103540 2024-04-21T09:01:58.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.495+0000 7f67b3fff700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f67b40a3c40 0x7f67b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:01:58.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.496+0000 7f67b3fff700 1 -- 172.21.15.186:0/3459942377 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f67c40cee30 con 0x7f67cc103540 2024-04-21T09:01:58.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.496+0000 7f67caffd700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f67b40a3c40 0x7f67b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:01:58.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.496+0000 7f67caffd700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f67b40a3c40 0x7f67b40a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67c0006fd0 tx=0x7f67c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:01:58.499 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.500+0000 7f67b3fff700 1 -- 172.21.15.186:0/3459942377 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67c4010070 con 0x7f67cc103540 2024-04-21T09:01:58.708 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:01:58.709+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f67cc0ff170 con 0x7f67b40a3c40 2024-04-21T09:01:59.551 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:59 smithi186 ceph-mon[31505]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:01:59.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:59 smithi186 ceph-mon[31505]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:59.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:59 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:59.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:59 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:59.552 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:01:59 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:59.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:59 smithi083 ceph-mon[31413]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:01:59.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:59 smithi083 ceph-mon[31413]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:59.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:59 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:59.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:59 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:59.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:01:59 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:01:59.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:59 smithi066 ceph-mon[27953]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:01:59.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:59 smithi066 ceph-mon[27953]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:01:59.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:59 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:01:59.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:59 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:01:59.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:01:59 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:01.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:01 smithi186 ceph-mon[31505]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:01.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:01 smithi083 ceph-mon[31413]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:01 smithi066 ceph-mon[27953]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:02.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/3977343869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]: dispatch 2024-04-21T09:02:02.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:02 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]: dispatch 2024-04-21T09:02:02.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:02 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]': finished 2024-04-21T09:02:02.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:02 smithi186 ceph-mon[31505]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T09:02:02.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:02 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:02.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/1142494340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/3977343869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]: dispatch 2024-04-21T09:02:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:02 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]: dispatch 2024-04-21T09:02:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:02 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]': finished 2024-04-21T09:02:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:02 smithi083 ceph-mon[31413]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T09:02:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:02 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:02.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/1142494340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:02.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/3977343869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]: dispatch 2024-04-21T09:02:02.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:02 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]: dispatch 2024-04-21T09:02:02.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:02 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c32caead-4fab-4b5e-9bc5-7d1bb3877747"}]': finished 2024-04-21T09:02:02.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:02 smithi066 ceph-mon[27953]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T09:02:02.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:02 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:02.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/1142494340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:03.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:03 smithi186 ceph-mon[31505]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:03 smithi083 ceph-mon[31413]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:03.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:03 smithi066 ceph-mon[27953]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:05.803 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:05 smithi186 ceph-mon[31505]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:05 smithi083 ceph-mon[31413]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:05.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:05 smithi066 ceph-mon[27953]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:07.793 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:07 smithi186 ceph-mon[31505]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:07.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:07 smithi083 ceph-mon[31413]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:07.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:07 smithi066 ceph-mon[27953]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:09.795 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:09 smithi186 ceph-mon[31505]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:09.795 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:09 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T09:02:09.795 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:09 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:09.795 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:09 smithi186 ceph-mon[31505]: Deploying daemon osd.5 on smithi186 2024-04-21T09:02:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:09 smithi083 ceph-mon[31413]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:09 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T09:02:09.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:09 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:09.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:09 smithi083 ceph-mon[31413]: Deploying daemon osd.5 on smithi186 2024-04-21T09:02:09.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:09 smithi066 ceph-mon[27953]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:09.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:09 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T09:02:09.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:09 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:09.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:09 smithi066 ceph-mon[27953]: Deploying daemon osd.5 on smithi186 2024-04-21T09:02:11.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:11 smithi186 ceph-mon[31505]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:11.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:11 smithi083 ceph-mon[31413]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:11.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:11 smithi066 ceph-mon[27953]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:13.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:13 smithi186 ceph-mon[31505]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:13.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:13 smithi083 ceph-mon[31413]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:13.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:13 smithi066 ceph-mon[27953]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:15 smithi083 ceph-mon[31413]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:15.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:15.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:15 smithi066 ceph-mon[27953]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:15.907 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:15 smithi186 ceph-mon[31505]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:15.907 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:15.907 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:15.907 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:17 smithi083 ceph-mon[31413]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:17 smithi066 ceph-mon[27953]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:17.877 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:17 smithi186 ceph-mon[31505]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:17.926 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.924+0000 7f67b3fff700 1 -- 172.21.15.186:0/3459942377 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67cc0ff170 con 0x7f67b40a3c40 2024-04-21T09:02:17.926 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f67b40a3c40 msgr2=0x7f67b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f67b40a3c40 0x7f67b40a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67c0006fd0 tx=0x7f67c0008040 comp rx=0 tx=0).stop 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc103540 msgr2=0x7f67cc1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc103540 0x7f67cc1044c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67c4007ee0 tx=0x7f67c400d3b0 comp rx=0 tx=0).stop 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 shutdown_connections 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f67b40a3c40 0x7f67b40a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67cc1019a0 0x7f67cc105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:17.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67cc103540 0x7f67cc1044c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:17.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 --2- 172.21.15.186:0/3459942377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67cc102ba0 0x7f67cc103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:17.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 >> 172.21.15.186:0/3459942377 conn(0x7f67cc0fcf60 msgr2=0x7f67cc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:02:17.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 shutdown_connections 2024-04-21T09:02:17.929 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:02:17.927+0000 7f67d1426700 1 -- 172.21.15.186:0/3459942377 wait complete. 2024-04-21T09:02:18.675 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.5.service 2024-04-21T09:02:18.677 INFO:tasks.cephadm:Deploying osd.6 on smithi066 with /dev/vg_nvme/lv_2... 2024-04-21T09:02:18.677 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T09:02:18.934 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:02:19.056 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.056 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.056 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.056 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.056 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:18 smithi066 ceph-mon[27953]: from='osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T09:02:19.056 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:18 smithi066 ceph-mon[27953]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T09:02:19.328 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:18 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:18 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:18 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:18 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:18 smithi186 ceph-mon[31505]: from='osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T09:02:19.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:18 smithi186 ceph-mon[31505]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T09:02:19.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:18 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:18 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:18 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:18 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:19.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:18 smithi083 ceph-mon[31413]: from='osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T09:02:19.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:18 smithi083 ceph-mon[31413]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T09:02:20.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:19 smithi186 ceph-mon[31505]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:20.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:19 smithi186 ceph-mon[31505]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T09:02:20.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:19 smithi186 ceph-mon[31505]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T09:02:20.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:20.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:19 smithi186 ceph-mon[31505]: from='osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:02:20.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:19 smithi186 ceph-mon[31505]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:02:20.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:19 smithi066 ceph-mon[27953]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:20.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:19 smithi066 ceph-mon[27953]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T09:02:20.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:19 smithi066 ceph-mon[27953]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T09:02:20.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:20.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:19 smithi066 ceph-mon[27953]: from='osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:02:20.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:19 smithi066 ceph-mon[27953]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:02:20.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:19 smithi083 ceph-mon[31413]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:20.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:19 smithi083 ceph-mon[31413]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T09:02:20.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:19 smithi083 ceph-mon[31413]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T09:02:20.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:20.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:19 smithi083 ceph-mon[31413]: from='osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:02:20.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:19 smithi083 ceph-mon[31413]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:02:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T09:02:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T09:02:20.939 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-21T09:02:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-21T09:02:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113232 s, 926 MB/s 2024-04-21T09:02:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-21T09:02:20.978 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:02:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:02:20.696+0000 7f0176b96700 -1 osd.5 0 waiting for initial osdmap 2024-04-21T09:02:20.978 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:02:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:02:20.704+0000 7f016e981700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:02:21.227 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:21 smithi186 ceph-mon[31505]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:02:21.228 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:21 smithi186 ceph-mon[31505]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T09:02:21.228 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:21.228 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:21.554 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:21 smithi066 ceph-mon[27953]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:02:21.554 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:21 smithi066 ceph-mon[27953]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T09:02:21.554 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:21.554 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:21.554 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:21 smithi066 ceph-mon[27953]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:21.555 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:21 smithi066 ceph-mon[27953]: from='osd.5 ' entity='osd.5' 2024-04-21T09:02:21.560 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:21 smithi083 ceph-mon[31413]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:02:21.560 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:21 smithi083 ceph-mon[31413]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T09:02:21.560 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:21.560 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:21.560 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:21 smithi083 ceph-mon[31413]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:21.561 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:21 smithi083 ceph-mon[31413]: from='osd.5 ' entity='osd.5' 2024-04-21T09:02:21.583 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2024-04-21T09:02:21.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:21 smithi186 ceph-mon[31505]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-21T09:02:21.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:21 smithi186 ceph-mon[31505]: from='osd.5 ' entity='osd.5' 2024-04-21T09:02:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:02:22.424 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] boot 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi186 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:22.426 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:22.426 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.426 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:22.426 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:22.426 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:22.426 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] boot 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi186 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:22.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: osd.5 [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] boot 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi186 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:22.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:22.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:22.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:22.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:22.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:22.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:23.337 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:23 smithi066 ceph-mon[27953]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T09:02:23.337 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:23 smithi066 ceph-mon[27953]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:23.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 -- 172.21.15.66:0/1426181099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8104640 msgr2=0x7fb5b810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:23.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 --2- 172.21.15.66:0/1426181099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8104640 0x7fb5b810aef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000b0d0 tx=0x7fb5b000b3e0 comp rx=0 tx=0).stop 2024-04-21T09:02:23.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 -- 172.21.15.66:0/1426181099 shutdown_connections 2024-04-21T09:02:23.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 --2- 172.21.15.66:0/1426181099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8104640 0x7fb5b810aef0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 --2- 172.21.15.66:0/1426181099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5b8102aa0 0x7fb5b8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 --2- 172.21.15.66:0/1426181099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b8103ca0 0x7fb5b8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 -- 172.21.15.66:0/1426181099 >> 172.21.15.66:0/1426181099 conn(0x7fb5b80fe040 msgr2=0x7fb5b8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 -- 172.21.15.66:0/1426181099 shutdown_connections 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.557+0000 7fb5bc943700 1 -- 172.21.15.66:0/1426181099 wait complete. 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 Processor -- start 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 -- start start 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8102aa0 0x7fb5b8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b8103ca0 0x7fb5b81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5b8104640 0x7fb5b81a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b810e540 con 0x7fb5b8102aa0 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b810e3c0 con 0x7fb5b8103ca0 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5bc943700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b810e6c0 con 0x7fb5b8104640 2024-04-21T09:02:23.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8102aa0 0x7fb5b8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8102aa0 0x7fb5b8198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33468/0 (socket says 172.21.15.66:33468) 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 -- 172.21.15.66:0/2137803638 learned_addr learned my addr 172.21.15.66:0/2137803638 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b5d9b700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b8103ca0 0x7fb5b81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b6d9d700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5b8104640 0x7fb5b81a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 -- 172.21.15.66:0/2137803638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5b8104640 msgr2=0x7fb5b81a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5b8104640 0x7fb5b81a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 -- 172.21.15.66:0/2137803638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b8103ca0 msgr2=0x7fb5b81990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b8103ca0 0x7fb5b81990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 -- 172.21.15.66:0/2137803638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b81a07e0 con 0x7fb5b8102aa0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.558+0000 7fb5b659c700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8102aa0 0x7fb5b8198b80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800d9b0 tx=0x7fb5a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.559+0000 7fb5a77fe700 1 -- 172.21.15.66:0/2137803638 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8017070 con 0x7fb5b8102aa0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.559+0000 7fb5a77fe700 1 -- 172.21.15.66:0/2137803638 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a8009de0 con 0x7fb5b8102aa0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.559+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b81a0ad0 con 0x7fb5b8102aa0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.559+0000 7fb5a77fe700 1 -- 172.21.15.66:0/2137803638 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a800f3c0 con 0x7fb5b8102aa0 2024-04-21T09:02:23.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.559+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b81a0fc0 con 0x7fb5b8102aa0 2024-04-21T09:02:23.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.560+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b8069730 con 0x7fb5b8102aa0 2024-04-21T09:02:23.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.561+0000 7fb5a77fe700 1 -- 172.21.15.66:0/2137803638 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5a8009930 con 0x7fb5b8102aa0 2024-04-21T09:02:23.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.562+0000 7fb5a77fe700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb5a00a3c40 0x7fb5a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:23.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.562+0000 7fb5b5d9b700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb5a00a3c40 0x7fb5a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:23.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.562+0000 7fb5a77fe700 1 -- 172.21.15.66:0/2137803638 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7fb5a80cf030 con 0x7fb5b8102aa0 2024-04-21T09:02:23.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.562+0000 7fb5b5d9b700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb5a00a3c40 0x7fb5a00a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac005950 tx=0x7fb5ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:02:23.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.567+0000 7fb5a77fe700 1 -- 172.21.15.66:0/2137803638 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5a809c8b0 con 0x7fb5b8102aa0 2024-04-21T09:02:23.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:23 smithi186 ceph-mon[31505]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T09:02:23.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:23 smithi186 ceph-mon[31505]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:23 smithi083 ceph-mon[31413]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T09:02:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:23 smithi083 ceph-mon[31413]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:23.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:23.786+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb5b8062e80 con 0x7fb5a00a3c40 2024-04-21T09:02:24.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:24 smithi066 ceph-mon[27953]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T09:02:24.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:24 smithi066 ceph-mon[27953]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:02:24.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:02:24.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:02:24.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:24 smithi186 ceph-mon[31505]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T09:02:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:24 smithi186 ceph-mon[31505]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:02:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:02:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:02:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:24 smithi083 ceph-mon[31413]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T09:02:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:24 smithi083 ceph-mon[31413]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:02:24.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:02:24.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:02:24.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:25 smithi186 ceph-mon[31505]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:25 smithi186 ceph-mon[31505]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T09:02:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:25 smithi083 ceph-mon[31413]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:25 smithi083 ceph-mon[31413]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T09:02:25.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:25 smithi066 ceph-mon[27953]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:25.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:25 smithi066 ceph-mon[27953]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T09:02:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:26 smithi186 ceph-mon[31505]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T09:02:26.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:26 smithi083 ceph-mon[31413]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T09:02:26.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:26 smithi066 ceph-mon[27953]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T09:02:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:27 smithi186 ceph-mon[31505]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:27 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/238196145' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ffb4890-5238-4ba0-9197-82b350fbdcd2"}]: dispatch 2024-04-21T09:02:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:27 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/238196145' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ffb4890-5238-4ba0-9197-82b350fbdcd2"}]': finished 2024-04-21T09:02:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:27 smithi186 ceph-mon[31505]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T09:02:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:27.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:27 smithi083 ceph-mon[31413]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:27.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:27 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/238196145' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ffb4890-5238-4ba0-9197-82b350fbdcd2"}]: dispatch 2024-04-21T09:02:27.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:27 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/238196145' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ffb4890-5238-4ba0-9197-82b350fbdcd2"}]': finished 2024-04-21T09:02:27.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:27 smithi083 ceph-mon[31413]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T09:02:27.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:27 smithi066 ceph-mon[27953]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:27 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/238196145' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ffb4890-5238-4ba0-9197-82b350fbdcd2"}]: dispatch 2024-04-21T09:02:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:27 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/238196145' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ffb4890-5238-4ba0-9197-82b350fbdcd2"}]': finished 2024-04-21T09:02:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:27 smithi066 ceph-mon[27953]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T09:02:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:28 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1876944652' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:28.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:28 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1876944652' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:28.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:28 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1876944652' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:29.520 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:29 smithi066 ceph-mon[27953]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-21T09:02:29.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:29 smithi186 ceph-mon[31505]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-21T09:02:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:29 smithi083 ceph-mon[31413]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-21T09:02:31.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:31 smithi083 ceph-mon[31413]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:02:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:31 smithi186 ceph-mon[31505]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:02:31.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:31 smithi066 ceph-mon[27953]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:02:33.368 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:33 smithi066 ceph-mon[27953]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T09:02:33.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:33 smithi186 ceph-mon[31505]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T09:02:33.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:33 smithi083 ceph-mon[31413]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T09:02:35.457 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:35 smithi066 ceph-mon[27953]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T09:02:35.458 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T09:02:35.458 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:35.458 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:35 smithi066 ceph-mon[27953]: Deploying daemon osd.6 on smithi066 2024-04-21T09:02:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:35 smithi186 ceph-mon[31505]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T09:02:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T09:02:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:35 smithi186 ceph-mon[31505]: Deploying daemon osd.6 on smithi066 2024-04-21T09:02:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:35 smithi083 ceph-mon[31413]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T09:02:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T09:02:35.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:35.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:35 smithi083 ceph-mon[31413]: Deploying daemon osd.6 on smithi066 2024-04-21T09:02:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:37 smithi186 ceph-mon[31505]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:02:37.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:37 smithi083 ceph-mon[31413]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:02:37.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:37 smithi066 ceph-mon[27953]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:02:39.548 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:39 smithi066 ceph-mon[27953]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:39.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:39 smithi186 ceph-mon[31505]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:39 smithi083 ceph-mon[31413]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:41 smithi066 ceph-mon[27953]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:41.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:41 smithi186 ceph-mon[31505]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:41.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:41 smithi083 ceph-mon[31413]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:43.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:43 smithi186 ceph-mon[31505]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:43.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:43 smithi083 ceph-mon[31413]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:43.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:43 smithi066 ceph-mon[27953]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:44.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.294+0000 7fb5a77fe700 1 -- 172.21.15.66:0/2137803638 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb5b8062e80 con 0x7fb5a00a3c40 2024-04-21T09:02:44.294 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 6 on host 'smithi066' 2024-04-21T09:02:44.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb5a00a3c40 msgr2=0x7fb5a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:44.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb5a00a3c40 0x7fb5a00a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac005950 tx=0x7fb5ac00b410 comp rx=0 tx=0).stop 2024-04-21T09:02:44.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8102aa0 msgr2=0x7fb5b8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:44.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8102aa0 0x7fb5b8198b80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800d9b0 tx=0x7fb5a800dcc0 comp rx=0 tx=0).stop 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 shutdown_connections 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb5a00a3c40 0x7fb5a00a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5b8104640 0x7fb5b81a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5b8102aa0 0x7fb5b8198b80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 --2- 172.21.15.66:0/2137803638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b8103ca0 0x7fb5b81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 >> 172.21.15.66:0/2137803638 conn(0x7fb5b80fe040 msgr2=0x7fb5b8107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 shutdown_connections 2024-04-21T09:02:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:02:44.297+0000 7fb5bc943700 1 -- 172.21.15.66:0/2137803638 wait complete. 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:44.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:44 smithi066 ceph-mon[27953]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:44 smithi186 ceph-mon[31505]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:02:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:44 smithi083 ceph-mon[31413]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T09:02:45.003 DEBUG:teuthology.orchestra.run.smithi066:osd.6> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.6.service 2024-04-21T09:02:45.005 INFO:tasks.cephadm:Deploying osd.7 on smithi083 with /dev/vg_nvme/lv_2... 2024-04-21T09:02:45.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T09:02:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:02:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:45 smithi066 ceph-mon[27953]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:45 smithi066 ceph-mon[27953]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T09:02:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:45 smithi066 ceph-mon[27953]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T09:02:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:45 smithi066 ceph-mon[27953]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:02:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:45.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:45 smithi083 ceph-mon[31413]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:45.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:45 smithi083 ceph-mon[31413]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T09:02:45.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:45 smithi083 ceph-mon[31413]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T09:02:45.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:45 smithi083 ceph-mon[31413]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:02:45.723 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:45.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:45 smithi186 ceph-mon[31505]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:45.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:45 smithi186 ceph-mon[31505]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T09:02:45.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:45 smithi186 ceph-mon[31505]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T09:02:45.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:45 smithi186 ceph-mon[31505]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:02:45.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:46.611 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:02:46 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:02:46.288+0000 7f3b7d838700 -1 osd.6 0 waiting for initial osdmap 2024-04-21T09:02:46.611 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:02:46 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:02:46.297+0000 7f3b7962b700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:02:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T09:02:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T09:02:47.368 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-21T09:02:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-21T09:02:47.368 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130543 s, 803 MB/s 2024-04-21T09:02:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-21T09:02:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:47 smithi186 ceph-mon[31505]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:47 smithi186 ceph-mon[31505]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:02:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:47 smithi186 ceph-mon[31505]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T09:02:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:47 smithi083 ceph-mon[31413]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:47 smithi083 ceph-mon[31413]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:02:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:47 smithi083 ceph-mon[31413]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T09:02:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:47.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:47 smithi066 ceph-mon[27953]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-21T09:02:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:47 smithi066 ceph-mon[27953]: from='osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:02:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:47 smithi066 ceph-mon[27953]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T09:02:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:48.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-04-21T09:02:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:02:48.580 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:48 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:02:48.580 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:48 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:02:48.580 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:48 smithi083 ceph-mon[31413]: osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] boot 2024-04-21T09:02:48.580 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:48 smithi083 ceph-mon[31413]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T09:02:48.580 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:48 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:48.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:48 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:02:48.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:48 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:02:48.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:48 smithi186 ceph-mon[31505]: osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] boot 2024-04-21T09:02:48.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:48 smithi186 ceph-mon[31505]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T09:02:48.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:48 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:48.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:48 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:02:48.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:48 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:02:48.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:48 smithi066 ceph-mon[27953]: osd.6 [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] boot 2024-04-21T09:02:48.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:48 smithi066 ceph-mon[27953]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T09:02:48.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:48 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi066 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi066 to 5696M 2024-04-21T09:02:49.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:49.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:49.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:49 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi066 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi066 to 5696M 2024-04-21T09:02:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:49.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:49.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:49 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi066 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi066 to 5696M 2024-04-21T09:02:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:49.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:49.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:02:49.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:49 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:02:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.000+0000 7f5415f9e700 1 -- 172.21.15.83:0/633571207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410102bc0 msgr2=0x7f5410103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.000+0000 7f5415f9e700 1 --2- 172.21.15.83:0/633571207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410102bc0 0x7f5410103020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54040099c0 tx=0x7f5404009cd0 comp rx=0 tx=0).stop 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.000+0000 7f5415f9e700 1 -- 172.21.15.83:0/633571207 shutdown_connections 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.000+0000 7f5415f9e700 1 --2- 172.21.15.83:0/633571207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5410103560 0x7f5410109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.000+0000 7f5415f9e700 1 --2- 172.21.15.83:0/633571207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54101019c0 0x7f5410101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.000+0000 7f5415f9e700 1 --2- 172.21.15.83:0/633571207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410102bc0 0x7f5410103020 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.000+0000 7f5415f9e700 1 -- 172.21.15.83:0/633571207 >> 172.21.15.83:0/633571207 conn(0x7f54100fcf60 msgr2=0x7f54100ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 -- 172.21.15.83:0/633571207 shutdown_connections 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 -- 172.21.15.83:0/633571207 wait complete. 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 Processor -- start 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 -- start start 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101019c0 0x7f5410105950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5410102bc0 0x7f5410103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5410103560 0x7f54101044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f541010d4b0 con 0x7f5410102bc0 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f541010d330 con 0x7f54101019c0 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f5415f9e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f541010d630 con 0x7f5410103560 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.001+0000 7f540f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101019c0 0x7f5410105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101019c0 0x7f5410105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57826/0 (socket says 172.21.15.83:57826) 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 -- 172.21.15.83:0/854851846 learned_addr learned my addr 172.21.15.83:0/854851846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540effd700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5410102bc0 0x7f5410103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540ffff700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5410103560 0x7f54101044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 -- 172.21.15.83:0/854851846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5410103560 msgr2=0x7f54101044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5410103560 0x7f54101044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 -- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5410102bc0 msgr2=0x7f5410103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5410102bc0 0x7f5410103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 -- 172.21.15.83:0/854851846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5410104d90 con 0x7f54101019c0 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540ffff700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5410103560 0x7f54101044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540effd700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5410102bc0 0x7f5410103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540f7fe700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101019c0 0x7f5410105950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f540000c990 tx=0x7f540000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540cff9700 1 -- 172.21.15.83:0/854851846 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400007b00 con 0x7f54101019c0 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540cff9700 1 -- 172.21.15.83:0/854851846 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5400007ca0 con 0x7f54101019c0 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54101a2d10 con 0x7f54101019c0 2024-04-21T09:02:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f540cff9700 1 -- 172.21.15.83:0/854851846 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400018660 con 0x7f54101019c0 2024-04-21T09:02:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.002+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54101a31a0 con 0x7f54101019c0 2024-04-21T09:02:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.003+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54101a3430 con 0x7f54101019c0 2024-04-21T09:02:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.005+0000 7f540cff9700 1 -- 172.21.15.83:0/854851846 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5400004020 con 0x7f54101019c0 2024-04-21T09:02:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.005+0000 7f540cff9700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f53f80a3cc0 0x7f53f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:02:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.005+0000 7f540cff9700 1 -- 172.21.15.83:0/854851846 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f54000cf760 con 0x7f54101019c0 2024-04-21T09:02:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.005+0000 7f540effd700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f53f80a3cc0 0x7f53f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:02:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.006+0000 7f540effd700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f53f80a3cc0 0x7f53f80a6180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5404000c00 tx=0x7f5404005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:02:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.009+0000 7f540cff9700 1 -- 172.21.15.83:0/854851846 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5400014030 con 0x7f54101019c0 2024-04-21T09:02:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:02:50.214+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f54100ff0c0 con 0x7f53f80a3cc0 2024-04-21T09:02:50.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:50 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:02:50.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:50 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:02:50.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:50 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:50.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:50 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:02:50.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:50 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:02:50.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:50 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:02:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:02:50.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:50 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:02:51.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:51 smithi083 ceph-mon[31413]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:02:51.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:51 smithi083 ceph-mon[31413]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:51 smithi186 ceph-mon[31505]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:02:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:51 smithi186 ceph-mon[31505]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:51 smithi066 ceph-mon[27953]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:02:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:51 smithi066 ceph-mon[27953]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:53 smithi186 ceph-mon[31505]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:53 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/2015759250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]: dispatch 2024-04-21T09:02:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:53 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]: dispatch 2024-04-21T09:02:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:53 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]': finished 2024-04-21T09:02:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:53 smithi186 ceph-mon[31505]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T09:02:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:53 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:02:53.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:53 smithi083 ceph-mon[31413]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:53.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:53 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/2015759250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]: dispatch 2024-04-21T09:02:53.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:53 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]: dispatch 2024-04-21T09:02:53.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:53 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]': finished 2024-04-21T09:02:53.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:53 smithi083 ceph-mon[31413]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T09:02:53.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:53 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:02:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:53 smithi066 ceph-mon[27953]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:53 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/2015759250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]: dispatch 2024-04-21T09:02:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:53 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]: dispatch 2024-04-21T09:02:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:53 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a6d8f44-b895-4c32-a527-c129c044aa7d"}]': finished 2024-04-21T09:02:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:53 smithi066 ceph-mon[27953]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T09:02:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:53 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:02:54.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:54 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/3675109262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:54.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:54 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/3675109262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:54.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:54 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/3675109262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:02:55.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:55 smithi186 ceph-mon[31505]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:55 smithi083 ceph-mon[31413]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:55.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:55 smithi066 ceph-mon[27953]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:57 smithi186 ceph-mon[31505]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:57.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:57 smithi083 ceph-mon[31413]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:57 smithi066 ceph-mon[27953]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:59.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:02:59 smithi186 ceph-mon[31505]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:59.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:02:59 smithi083 ceph-mon[31413]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:02:59.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:02:59 smithi066 ceph-mon[27953]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:00 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T09:03:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:00 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:00 smithi186 ceph-mon[31505]: Deploying daemon osd.7 on smithi083 2024-04-21T09:03:00.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:00 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T09:03:00.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:00 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:00.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:00 smithi083 ceph-mon[31413]: Deploying daemon osd.7 on smithi083 2024-04-21T09:03:00.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:00 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T09:03:00.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:00 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:00.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:00 smithi066 ceph-mon[27953]: Deploying daemon osd.7 on smithi083 2024-04-21T09:03:01.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:01 smithi186 ceph-mon[31505]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:01.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:01 smithi083 ceph-mon[31413]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:01 smithi066 ceph-mon[27953]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:03 smithi186 ceph-mon[31505]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:03 smithi083 ceph-mon[31413]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:03 smithi066 ceph-mon[27953]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:05.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:05 smithi186 ceph-mon[31505]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:05 smithi083 ceph-mon[31413]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:05.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:05 smithi066 ceph-mon[27953]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:06.338 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:06.338 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:06.338 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:06.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:06.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:06.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:06.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:06.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:06.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:07.848 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:07 smithi083 ceph-mon[31413]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:08.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:07 smithi186 ceph-mon[31505]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:08.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:07 smithi066 ceph-mon[27953]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:09.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.080+0000 7f540cff9700 1 -- 172.21.15.83:0/854851846 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f54100ff0c0 con 0x7f53f80a3cc0 2024-04-21T09:03:09.081 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 7 on host 'smithi083' 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f53f80a3cc0 msgr2=0x7f53f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f53f80a3cc0 0x7f53f80a6180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5404000c00 tx=0x7f5404005fb0 comp rx=0 tx=0).stop 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101019c0 msgr2=0x7f5410105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101019c0 0x7f5410105950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f540000c990 tx=0x7f540000cd50 comp rx=0 tx=0).stop 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 shutdown_connections 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f53f80a3cc0 0x7f53f80a6180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101019c0 0x7f5410105950 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5410103560 0x7f54101044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 --2- 172.21.15.83:0/854851846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5410102bc0 0x7f5410103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.082+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 >> 172.21.15.83:0/854851846 conn(0x7f54100fcf60 msgr2=0x7f54100fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.083+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 shutdown_connections 2024-04-21T09:03:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:03:09.083+0000 7f5415f9e700 1 -- 172.21.15.83:0/854851846 wait complete. 2024-04-21T09:03:10.082 DEBUG:teuthology.orchestra.run.smithi083:osd.7> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.7.service 2024-04-21T09:03:10.085 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2024-04-21T09:03:10.085 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T09:03:10.334 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:03:10.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:10 smithi186 ceph-mon[31505]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:10.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:10.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:10.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:10 smithi186 ceph-mon[31505]: from='osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T09:03:10.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:10 smithi186 ceph-mon[31505]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T09:03:10.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:10 smithi083 ceph-mon[31413]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:10.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:10.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:10 smithi083 ceph-mon[31413]: from='osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T09:03:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:10 smithi083 ceph-mon[31413]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T09:03:10.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:10 smithi066 ceph-mon[27953]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:10.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:10.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:10.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:10 smithi066 ceph-mon[27953]: from='osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T09:03:10.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:10 smithi066 ceph-mon[27953]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T09:03:11.121 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-mon[31413]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T09:03:11.121 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-mon[31413]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T09:03:11.121 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:11.121 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-mon[31413]: from='osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:03:11.121 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-mon[31413]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:03:11.121 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:11.121 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:11 smithi186 ceph-mon[31505]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T09:03:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:11 smithi186 ceph-mon[31505]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T09:03:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:11 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:11 smithi186 ceph-mon[31505]: from='osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:03:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:11 smithi186 ceph-mon[31505]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:03:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:11 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:11 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:11 smithi066 ceph-mon[27953]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T09:03:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:11 smithi066 ceph-mon[27953]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T09:03:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:11 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:11 smithi066 ceph-mon[27953]: from='osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:03:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:11 smithi066 ceph-mon[27953]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:03:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:11 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:11 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:11.860 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:03:11.519+0000 7f8a33909700 -1 osd.7 0 waiting for initial osdmap 2024-04-21T09:03:11.860 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:03:11 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:03:11.529+0000 7f8a2b6f4700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:03:12.211 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:12 smithi083 ceph-mon[31413]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:12.211 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:12 smithi083 ceph-mon[31413]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:03:12.211 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:12 smithi083 ceph-mon[31413]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T09:03:12.211 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:12 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:12.212 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:12 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:12.212 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:12 smithi083 ceph-mon[31413]: from='osd.7 ' entity='osd.7' 2024-04-21T09:03:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:12 smithi186 ceph-mon[31505]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:12 smithi186 ceph-mon[31505]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:03:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:12 smithi186 ceph-mon[31505]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T09:03:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:12 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:12 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:12.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:12 smithi186 ceph-mon[31505]: from='osd.7 ' entity='osd.7' 2024-04-21T09:03:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:12 smithi066 ceph-mon[27953]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:12 smithi066 ceph-mon[27953]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:03:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:12 smithi066 ceph-mon[27953]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T09:03:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:12 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:12 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:12 smithi066 ceph-mon[27953]: from='osd.7 ' entity='osd.7' 2024-04-21T09:03:12.418 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T09:03:12.419 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T09:03:12.419 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-21T09:03:12.419 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-21T09:03:12.419 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011466 s, 915 MB/s 2024-04-21T09:03:12.419 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-21T09:03:13.054 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-04-21T09:03:13.310 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:03:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:13 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:03:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:13 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:03:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:13 smithi186 ceph-mon[31505]: osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] boot 2024-04-21T09:03:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:13 smithi186 ceph-mon[31505]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T09:03:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:13.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:13 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:03:13.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:13 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:03:13.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:13.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:13 smithi083 ceph-mon[31413]: osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] boot 2024-04-21T09:03:13.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:13 smithi083 ceph-mon[31413]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T09:03:13.341 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:13 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:03:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:13 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:03:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:13 smithi066 ceph-mon[27953]: osd.7 [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] boot 2024-04-21T09:03:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:13 smithi066 ceph-mon[27953]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T09:03:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:03:14.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:14 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:14.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:14 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-21T09:03:14.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T09:03:14.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:14.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:14 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:15.012 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.998+0000 7ff3b693b700 1 -- 172.21.15.186:0/1707215048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 msgr2=0x7ff3b0104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:15.012 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.998+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1707215048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0104100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009990 tx=0x7ff3a0009ca0 comp rx=0 tx=0).stop 2024-04-21T09:03:15.012 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 -- 172.21.15.186:0/1707215048 shutdown_connections 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1707215048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b0102aa0 0x7ff3b0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1707215048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3b0104640 0x7ff3b010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1707215048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0104100 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 -- 172.21.15.186:0/1707215048 >> 172.21.15.186:0/1707215048 conn(0x7ff3b00fe040 msgr2=0x7ff3b0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 -- 172.21.15.186:0/1707215048 shutdown_connections 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 -- 172.21.15.186:0/1707215048 wait complete. 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 Processor -- start 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:14.999+0000 7ff3b693b700 1 -- start start 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3b0102aa0 0x7ff3b0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b0104640 0x7ff3b01a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010e530 con 0x7ff3b0102aa0 2024-04-21T09:03:15.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010e3b0 con 0x7ff3b0104640 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010e6b0 con 0x7ff3b0103ca0 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:35462/0 (socket says 172.21.15.186:35462) 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 -- 172.21.15.186:0/1812896880 learned_addr learned my addr 172.21.15.186:0/1812896880 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3affff700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3b0102aa0 0x7ff3b0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 -- 172.21.15.186:0/1812896880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b0104640 msgr2=0x7ff3b01a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b0104640 0x7ff3b01a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 -- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3b0102aa0 msgr2=0x7ff3b0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3b0102aa0 0x7ff3b0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 -- 172.21.15.186:0/1812896880 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b01a0870 con 0x7ff3b0103ca0 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3affff700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3b0102aa0 0x7ff3b0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3af7fe700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0199160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0006010 tx=0x7ff3a0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:03:15.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3ad7fa700 1 -- 172.21.15.186:0/1812896880 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0027070 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b01a0b00 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3ad7fa700 1 -- 172.21.15.186:0/1812896880 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a0022470 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3ad7fa700 1 -- 172.21.15.186:0/1812896880 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a000f6e0 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.000+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b01a0fb0 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.001+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b0069730 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.002+0000 7ff3ad7fa700 1 -- 172.21.15.186:0/1812896880 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff3a002a070 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.003+0000 7ff3ad7fa700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3980a3c40 0x7ff3980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.003+0000 7ff3ad7fa700 1 -- 172.21.15.186:0/1812896880 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7ff3a00d0830 con 0x7ff3b0103ca0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.003+0000 7ff3affff700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3980a3c40 0x7ff3980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.004+0000 7ff3affff700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3980a3c40 0x7ff3980a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4005fd0 tx=0x7ff3a4005f20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:03:15.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.007+0000 7ff3ad7fa700 1 -- 172.21.15.186:0/1812896880 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3a0013070 con 0x7ff3b0103ca0 2024-04-21T09:03:15.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:15.214+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff3b0062e80 con 0x7ff3980a3c40 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi083 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:15 smithi083 ceph-mon[31413]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi083 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:15.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:15 smithi066 ceph-mon[27953]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi083 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:15 smithi186 ceph-mon[31505]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T09:03:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:16 smithi083 ceph-mon[31413]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:03:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:03:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:16 smithi083 ceph-mon[31413]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T09:03:16.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:16 smithi066 ceph-mon[27953]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:16.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:03:16.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:03:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:16.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:16 smithi066 ceph-mon[27953]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T09:03:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:16 smithi186 ceph-mon[31505]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:03:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:03:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:16 smithi186 ceph-mon[31505]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T09:03:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:17 smithi186 ceph-mon[31505]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:03:17.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:17 smithi083 ceph-mon[31413]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:03:17.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:17 smithi066 ceph-mon[27953]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:03:18.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:18 smithi186 ceph-mon[31505]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:18.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:18 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/500305882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]: dispatch 2024-04-21T09:03:18.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:18 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]: dispatch 2024-04-21T09:03:18.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:18 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]': finished 2024-04-21T09:03:18.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:18 smithi186 ceph-mon[31505]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T09:03:18.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:18 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:18.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:18 smithi083 ceph-mon[31413]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:18 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/500305882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]: dispatch 2024-04-21T09:03:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:18 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]: dispatch 2024-04-21T09:03:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:18 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]': finished 2024-04-21T09:03:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:18 smithi083 ceph-mon[31413]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T09:03:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:18 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:18.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:18 smithi066 ceph-mon[27953]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:18.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:18 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/500305882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]: dispatch 2024-04-21T09:03:18.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:18 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]: dispatch 2024-04-21T09:03:18.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:18 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2c7530c-00b0-49d6-9157-9c3c86f9b4a6"}]': finished 2024-04-21T09:03:18.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:18 smithi066 ceph-mon[27953]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T09:03:18.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:18 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:19.511 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:19 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/2707020148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:03:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:19 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/2707020148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:03:19.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:19 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/2707020148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:03:20.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:20 smithi186 ceph-mon[31505]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:20.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:20 smithi083 ceph-mon[31413]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:20.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:20 smithi066 ceph-mon[27953]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:22.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:22 smithi186 ceph-mon[31505]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:22.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:22 smithi083 ceph-mon[31413]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:22.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:22 smithi066 ceph-mon[27953]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:24.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:24 smithi186 ceph-mon[31505]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:24 smithi083 ceph-mon[31413]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:24.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:24 smithi066 ceph-mon[27953]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:25 smithi186 ceph-mon[31505]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:25 smithi083 ceph-mon[31413]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:25.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:25 smithi066 ceph-mon[27953]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:26.330 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:26 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T09:03:26.330 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:26 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:26.330 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:26 smithi186 ceph-mon[31505]: Deploying daemon osd.8 on smithi186 2024-04-21T09:03:26.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:26 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T09:03:26.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:26 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:26.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:26 smithi083 ceph-mon[31413]: Deploying daemon osd.8 on smithi186 2024-04-21T09:03:26.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:26 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T09:03:26.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:26 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:26.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:26 smithi066 ceph-mon[27953]: Deploying daemon osd.8 on smithi186 2024-04-21T09:03:27.284 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:27 smithi186 ceph-mon[31505]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:27.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:27 smithi083 ceph-mon[31413]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:27 smithi066 ceph-mon[27953]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:29.438 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:29 smithi186 ceph-mon[31505]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:29 smithi083 ceph-mon[31413]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:29.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:29 smithi066 ceph-mon[27953]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:32.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:32 smithi186 ceph-mon[31505]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:32.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:32 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:32.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:32 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:32.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:32 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:32.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:32 smithi083 ceph-mon[31413]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:32.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:32 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:32.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:32 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:32.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:32 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:32.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:32 smithi066 ceph-mon[27953]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:32.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:32 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:32.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:32 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:32.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:32 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:34.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:34 smithi186 ceph-mon[31505]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:34.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:34 smithi083 ceph-mon[31413]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:34.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:34 smithi066 ceph-mon[27953]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:34.786 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.786+0000 7ff3ad7fa700 1 -- 172.21.15.186:0/1812896880 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff3b0062e80 con 0x7ff3980a3c40 2024-04-21T09:03:34.786 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3980a3c40 msgr2=0x7ff3980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3980a3c40 0x7ff3980a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4005fd0 tx=0x7ff3a4005f20 comp rx=0 tx=0).stop 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 msgr2=0x7ff3b0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0199160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0006010 tx=0x7ff3a0009f90 comp rx=0 tx=0).stop 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 shutdown_connections 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff3980a3c40 0x7ff3980a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b0104640 0x7ff3b01a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3b0102aa0 0x7ff3b0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 --2- 172.21.15.186:0/1812896880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0103ca0 0x7ff3b0199160 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 >> 172.21.15.186:0/1812896880 conn(0x7ff3b00fe040 msgr2=0x7ff3b01070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 shutdown_connections 2024-04-21T09:03:34.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:03:34.789+0000 7ff3b693b700 1 -- 172.21.15.186:0/1812896880 wait complete. 2024-04-21T09:03:35.438 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.8.service 2024-04-21T09:03:35.441 INFO:tasks.cephadm:Deploying osd.9 on smithi066 with /dev/vg_nvme/lv_1... 2024-04-21T09:03:35.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T09:03:35.705 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:03:35.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:35 smithi066 ceph-mon[27953]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:35.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:35.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:35.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:35 smithi066 ceph-mon[27953]: from='osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T09:03:35.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:35 smithi066 ceph-mon[27953]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T09:03:35.983 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:35.983 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:35 smithi186 ceph-mon[31505]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:35 smithi186 ceph-mon[31505]: from='osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T09:03:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:35 smithi186 ceph-mon[31505]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T09:03:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:35 smithi083 ceph-mon[31413]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:35 smithi083 ceph-mon[31413]: from='osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T09:03:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:35 smithi083 ceph-mon[31413]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T09:03:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:37.014 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:36 smithi066 ceph-mon[27953]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T09:03:37.014 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:36 smithi066 ceph-mon[27953]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T09:03:37.014 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:37.014 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:36 smithi066 ceph-mon[27953]: from='osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:03:37.014 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:36 smithi066 ceph-mon[27953]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:03:37.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:36 smithi186 ceph-mon[31505]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T09:03:37.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:36 smithi186 ceph-mon[31505]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T09:03:37.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:36 smithi186 ceph-mon[31505]: from='osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:03:37.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:36 smithi186 ceph-mon[31505]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:03:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:36 smithi083 ceph-mon[31413]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T09:03:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:36 smithi083 ceph-mon[31413]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T09:03:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:36 smithi083 ceph-mon[31413]: from='osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:03:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:36 smithi083 ceph-mon[31413]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:03:37.428 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:03:37.202+0000 7fc7e8be8700 -1 osd.8 0 waiting for initial osdmap 2024-04-21T09:03:37.428 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:03:37.211+0000 7fc7e01d2700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:03:37.950 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:37 smithi066 ceph-mon[27953]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:37.951 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:37 smithi066 ceph-mon[27953]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:03:37.951 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:37 smithi066 ceph-mon[27953]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T09:03:37.951 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:37.951 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:37 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:37.951 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:37 smithi066 ceph-mon[27953]: from='osd.8 ' entity='osd.8' 2024-04-21T09:03:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T09:03:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T09:03:37.951 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-21T09:03:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-21T09:03:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114522 s, 916 MB/s 2024-04-21T09:03:37.951 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 2024-04-21T09:03:37.952 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-21T09:03:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-mon[31505]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-mon[31505]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:03:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-mon[31505]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T09:03:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:37 smithi186 ceph-mon[31505]: from='osd.8 ' entity='osd.8' 2024-04-21T09:03:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:37 smithi083 ceph-mon[31413]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:37 smithi083 ceph-mon[31413]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:03:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:37 smithi083 ceph-mon[31413]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T09:03:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:37 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:37 smithi083 ceph-mon[31413]: from='osd.8 ' entity='osd.8' 2024-04-21T09:03:38.571 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2024-04-21T09:03:38.817 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:03:38.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:38 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:03:38.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:38 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:03:38.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:38.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:38 smithi066 ceph-mon[27953]: osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] boot 2024-04-21T09:03:38.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:38 smithi066 ceph-mon[27953]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T09:03:38.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:38 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:39.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:38 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:03:39.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:38 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:03:39.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:38 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:39.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:38 smithi186 ceph-mon[31505]: osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] boot 2024-04-21T09:03:39.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:38 smithi186 ceph-mon[31505]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T09:03:39.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:38 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:39.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:38 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:03:39.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:38 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:03:39.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:38 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:39.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:38 smithi083 ceph-mon[31413]: osd.8 [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] boot 2024-04-21T09:03:39.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:38 smithi083 ceph-mon[31413]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T09:03:39.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:38 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:03:40.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:40.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T09:03:40.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:40.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:40.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:40.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:40.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:03:40.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:40.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 -- 172.21.15.66:0/2213708774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098108790 msgr2=0x7f109810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:40.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 --2- 172.21.15.66:0/2213708774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098108790 0x7f109810ac00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1088009990 tx=0x7f1088009ca0 comp rx=0 tx=0).stop 2024-04-21T09:03:40.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 -- 172.21.15.66:0/2213708774 shutdown_connections 2024-04-21T09:03:40.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 --2- 172.21.15.66:0/2213708774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098108790 0x7f109810ac00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 --2- 172.21.15.66:0/2213708774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1098103530 0x7f1098105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 --2- 172.21.15.66:0/2213708774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098105e60 0x7f1098108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 -- 172.21.15.66:0/2213708774 >> 172.21.15.66:0/2213708774 conn(0x7f10980fcf40 msgr2=0x7f10980ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 -- 172.21.15.66:0/2213708774 shutdown_connections 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 -- 172.21.15.66:0/2213708774 wait complete. 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 Processor -- start 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.592+0000 7f10a05cf700 1 -- start start 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10a05cf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098103530 0x7f1098102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10a05cf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1098105e60 0x7f10981014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10a05cf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098108790 0x7f1098101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10a05cf700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f109810e520 con 0x7f1098103530 2024-04-21T09:03:40.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10a05cf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f109810e3a0 con 0x7f1098108790 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10a05cf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f109810e6a0 con 0x7f1098105e60 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098103530 0x7f1098102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098103530 0x7f1098102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43926/0 (socket says 172.21.15.66:43926) 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 -- 172.21.15.66:0/4011904842 learned_addr learned my addr 172.21.15.66:0/4011904842 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109db6a700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1098105e60 0x7f10981014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109eb6c700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098108790 0x7f1098101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 -- 172.21.15.66:0/4011904842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1098105e60 msgr2=0x7f10981014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1098105e60 0x7f10981014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 -- 172.21.15.66:0/4011904842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098108790 msgr2=0x7f1098101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098108790 0x7f1098101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 -- 172.21.15.66:0/4011904842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981022d0 con 0x7f1098103530 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f109e36b700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098103530 0x7f1098102e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1090007f10 tx=0x7f109000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10977fe700 1 -- 172.21.15.66:0/4011904842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090004bb0 con 0x7f1098103530 2024-04-21T09:03:40.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.593+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981a2d20 con 0x7f1098103530 2024-04-21T09:03:40.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.594+0000 7f10977fe700 1 -- 172.21.15.66:0/4011904842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1090004d50 con 0x7f1098103530 2024-04-21T09:03:40.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.594+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981a3180 con 0x7f1098103530 2024-04-21T09:03:40.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.594+0000 7f10977fe700 1 -- 172.21.15.66:0/4011904842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10900059b0 con 0x7f1098103530 2024-04-21T09:03:40.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.595+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109802df80 con 0x7f1098103530 2024-04-21T09:03:40.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.596+0000 7f10977fe700 1 -- 172.21.15.66:0/4011904842 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f109001a020 con 0x7f1098103530 2024-04-21T09:03:40.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.596+0000 7f10977fe700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f10800a3c40 0x7f10800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:03:40.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.597+0000 7f109db6a700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f10800a3c40 0x7f10800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:03:40.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.597+0000 7f10977fe700 1 -- 172.21.15.66:0/4011904842 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f10900cf1c0 con 0x7f1098103530 2024-04-21T09:03:40.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.597+0000 7f109db6a700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f10800a3c40 0x7f10800a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f108c009c60 tx=0x7f108c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:03:40.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.601+0000 7f10977fe700 1 -- 172.21.15.66:0/4011904842 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1090010070 con 0x7f1098103530 2024-04-21T09:03:40.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:03:40.823+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1098062e80 con 0x7f10800a3c40 2024-04-21T09:03:41.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:40 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi186 2024-04-21T09:03:41.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:40 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-21T09:03:41.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:40 smithi186 ceph-mon[31505]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T09:03:41.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:40 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi186 2024-04-21T09:03:41.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:40 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-21T09:03:41.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:40 smithi083 ceph-mon[31413]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T09:03:41.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:40 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi186 2024-04-21T09:03:41.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:40 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-21T09:03:41.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:40 smithi066 ceph-mon[27953]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T09:03:42.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:41 smithi186 ceph-mon[31505]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:42.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:41 smithi186 ceph-mon[31505]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T09:03:42.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:41 smithi186 ceph-mon[31505]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:03:42.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:03:42.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:03:42.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:42.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:41 smithi083 ceph-mon[31413]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:42.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:41 smithi083 ceph-mon[31413]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T09:03:42.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:41 smithi083 ceph-mon[31413]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:03:42.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:03:42.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:03:42.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:42.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:41 smithi066 ceph-mon[27953]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:42.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:41 smithi066 ceph-mon[27953]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T09:03:42.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:41 smithi066 ceph-mon[27953]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:03:42.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:03:42.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:03:42.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:43.006 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:42 smithi066 ceph-mon[27953]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T09:03:43.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:42 smithi186 ceph-mon[31505]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T09:03:43.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:42 smithi083 ceph-mon[31413]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T09:03:44.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:43 smithi186 ceph-mon[31505]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:44.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:43 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1016596632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b58b67c-416e-4de6-b41c-334ef88ea3ab"}]: dispatch 2024-04-21T09:03:44.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:43 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1016596632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b58b67c-416e-4de6-b41c-334ef88ea3ab"}]': finished 2024-04-21T09:03:44.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:43 smithi186 ceph-mon[31505]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T09:03:44.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:03:44.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:43 smithi083 ceph-mon[31413]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:44.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:43 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1016596632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b58b67c-416e-4de6-b41c-334ef88ea3ab"}]: dispatch 2024-04-21T09:03:44.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:43 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1016596632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b58b67c-416e-4de6-b41c-334ef88ea3ab"}]': finished 2024-04-21T09:03:44.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:43 smithi083 ceph-mon[31413]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T09:03:44.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:03:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:43 smithi066 ceph-mon[27953]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:43 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1016596632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b58b67c-416e-4de6-b41c-334ef88ea3ab"}]: dispatch 2024-04-21T09:03:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:43 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1016596632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b58b67c-416e-4de6-b41c-334ef88ea3ab"}]': finished 2024-04-21T09:03:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:43 smithi066 ceph-mon[27953]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T09:03:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:03:45.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:44 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4008706738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:03:45.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:44 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4008706738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:03:45.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:44 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4008706738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:03:46.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:45 smithi186 ceph-mon[31505]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-21T09:03:46.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:45 smithi083 ceph-mon[31413]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-21T09:03:46.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:45 smithi066 ceph-mon[27953]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-21T09:03:48.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:47 smithi186 ceph-mon[31505]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:03:48.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:47 smithi083 ceph-mon[31413]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:03:48.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:47 smithi066 ceph-mon[27953]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:03:50.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:49 smithi186 ceph-mon[31505]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-21T09:03:50.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:49 smithi083 ceph-mon[31413]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-21T09:03:50.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:49 smithi066 ceph-mon[27953]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-21T09:03:52.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:51 smithi186 ceph-mon[31505]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-21T09:03:52.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T09:03:52.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:52.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:51 smithi083 ceph-mon[31413]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-21T09:03:52.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T09:03:52.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:52.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:51 smithi066 ceph-mon[27953]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-21T09:03:52.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T09:03:52.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:03:52.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:52 smithi066 ceph-mon[27953]: Deploying daemon osd.9 on smithi066 2024-04-21T09:03:53.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:52 smithi186 ceph-mon[31505]: Deploying daemon osd.9 on smithi066 2024-04-21T09:03:53.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:52 smithi083 ceph-mon[31413]: Deploying daemon osd.9 on smithi066 2024-04-21T09:03:53.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:53 smithi066 ceph-mon[27953]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:03:54.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:53 smithi186 ceph-mon[31505]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:03:54.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:53 smithi083 ceph-mon[31413]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T09:03:55.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:55 smithi066 ceph-mon[27953]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:56.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:55 smithi186 ceph-mon[31505]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:56.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:55 smithi083 ceph-mon[31413]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:57 smithi186 ceph-mon[31505]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:58.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:58.089 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:57 smithi066 ceph-mon[27953]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:58.089 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:58.089 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:58.089 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:03:58.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:57 smithi083 ceph-mon[31413]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:03:58.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:03:58.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:00.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:03:59 smithi186 ceph-mon[31505]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:00.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:03:59 smithi083 ceph-mon[31413]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:00.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:03:59 smithi066 ceph-mon[27953]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:01.432 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 9 on host 'smithi066' 2024-04-21T09:04:01.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.431+0000 7f10977fe700 1 -- 172.21.15.66:0/4011904842 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1098062e80 con 0x7f10800a3c40 2024-04-21T09:04:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.433+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f10800a3c40 msgr2=0x7f10800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.433+0000 7f10a05cf700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f10800a3c40 0x7f10800a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f108c009c60 tx=0x7f108c009260 comp rx=0 tx=0).stop 2024-04-21T09:04:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.433+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098103530 msgr2=0x7f1098102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.433+0000 7f10a05cf700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098103530 0x7f1098102e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1090007f10 tx=0x7f109000d3b0 comp rx=0 tx=0).stop 2024-04-21T09:04:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 shutdown_connections 2024-04-21T09:04:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f10800a3c40 0x7f10800a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098108790 0x7f1098101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:01.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098103530 0x7f1098102e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:01.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 --2- 172.21.15.66:0/4011904842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1098105e60 0x7f10981014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:01.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 >> 172.21.15.66:0/4011904842 conn(0x7f10980fcf40 msgr2=0x7f1098103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:01.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 shutdown_connections 2024-04-21T09:04:01.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:01.434+0000 7f10a05cf700 1 -- 172.21.15.66:0/4011904842 wait complete. 2024-04-21T09:04:01.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:01 smithi066 ceph-mon[27953]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:01.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:01 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:01.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:01 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:01.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:01 smithi066 ceph-mon[27953]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T09:04:02.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:01 smithi186 ceph-mon[31505]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:02.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:01 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:02.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:01 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:02.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:01 smithi186 ceph-mon[31505]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T09:04:02.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:01 smithi083 ceph-mon[31413]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:02.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:01 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:02.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:01 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:02.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:01 smithi083 ceph-mon[31413]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T09:04:02.135 DEBUG:teuthology.orchestra.run.smithi066:osd.9> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.9.service 2024-04-21T09:04:02.138 INFO:tasks.cephadm:Deploying osd.10 on smithi083 with /dev/vg_nvme/lv_1... 2024-04-21T09:04:02.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T09:04:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:04:03.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:03 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:03.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:03 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:03.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:03 smithi083 ceph-mon[31413]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:03.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:03 smithi083 ceph-mon[31413]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T09:04:03.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:03 smithi083 ceph-mon[31413]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T09:04:03.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:03 smithi083 ceph-mon[31413]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:04:03.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:03 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:03.573 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:03.573 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:03.573 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-mon[27953]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:03.573 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-mon[27953]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T09:04:03.573 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-mon[27953]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T09:04:03.573 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-mon[27953]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:04:03.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:03.574 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:04:03.392+0000 7f031ae2d700 -1 osd.9 0 waiting for initial osdmap 2024-04-21T09:04:03.574 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:04:03 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:04:03.403+0000 7f0316c20700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:04:03.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:03 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:03 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:03 smithi186 ceph-mon[31505]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:03 smithi186 ceph-mon[31505]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T09:04:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:03 smithi186 ceph-mon[31505]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T09:04:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:03 smithi186 ceph-mon[31505]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-21T09:04:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:03 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T09:04:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T09:04:04.604 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-21T09:04:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-21T09:04:04.604 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115697 s, 906 MB/s 2024-04-21T09:04:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-21T09:04:04.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:04 smithi066 ceph-mon[27953]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:04:04.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:04 smithi066 ceph-mon[27953]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T09:04:04.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:04 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:04.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:04 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:04.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:04 smithi186 ceph-mon[31505]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:04:04.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:04 smithi186 ceph-mon[31505]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T09:04:04.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:04 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:04.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:04 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:04.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:04 smithi083 ceph-mon[31413]: from='osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-21T09:04:04.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:04 smithi083 ceph-mon[31413]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T09:04:04.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:04 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:04.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:04 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:05.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-04-21T09:04:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:05 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:04:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:05 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:04:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:05 smithi083 ceph-mon[31413]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:05 smithi083 ceph-mon[31413]: osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] boot 2024-04-21T09:04:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:05 smithi083 ceph-mon[31413]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T09:04:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:05 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:05 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:04:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:05 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:04:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:05 smithi066 ceph-mon[27953]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:05 smithi066 ceph-mon[27953]: osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] boot 2024-04-21T09:04:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:05 smithi066 ceph-mon[27953]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T09:04:05.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:05 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:05.653 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:04:05.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:05 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:04:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:05 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:04:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:05 smithi186 ceph-mon[31505]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T09:04:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:05 smithi186 ceph-mon[31505]: osd.9 [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] boot 2024-04-21T09:04:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:05 smithi186 ceph-mon[31505]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T09:04:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:05 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:04:06.824 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T09:04:06.824 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi066 2024-04-21T09:04:06.824 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi066 to 4272M 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:06.825 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T09:04:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi066 2024-04-21T09:04:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi066 to 4272M 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:06.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi066 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi066 to 4272M 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:06.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:06.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:06.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:06.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 -- 172.21.15.83:0/2024383373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c105e50 msgr2=0x7fe28c108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 --2- 172.21.15.83:0/2024383373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c105e50 0x7fe28c108240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe280009990 tx=0x7fe280009ca0 comp rx=0 tx=0).stop 2024-04-21T09:04:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 -- 172.21.15.83:0/2024383373 shutdown_connections 2024-04-21T09:04:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 --2- 172.21.15.83:0/2024383373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c108780 0x7fe28c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:07.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 --2- 172.21.15.83:0/2024383373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe28c103520 0x7fe28c105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:07.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 --2- 172.21.15.83:0/2024383373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c105e50 0x7fe28c108240 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:07.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 -- 172.21.15.83:0/2024383373 >> 172.21.15.83:0/2024383373 conn(0x7fe28c0fcf30 msgr2=0x7fe28c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:07.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 -- 172.21.15.83:0/2024383373 shutdown_connections 2024-04-21T09:04:07.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.320+0000 7fe290881700 1 -- 172.21.15.83:0/2024383373 wait complete. 2024-04-21T09:04:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 Processor -- start 2024-04-21T09:04:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 -- start start 2024-04-21T09:04:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c103520 0x7fe28c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe28c105e50 0x7fe28c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c108780 0x7fe28c1a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c10e590 con 0x7fe28c105e50 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c10e410 con 0x7fe28c103520 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe290881700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c10e710 con 0x7fe28c108780 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c103520 0x7fe28c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c103520 0x7fe28c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49442/0 (socket says 172.21.15.83:49442) 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 -- 172.21.15.83:0/318048068 learned_addr learned my addr 172.21.15.83:0/318048068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28ad9d700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c108780 0x7fe28c1a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe289d9b700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe28c105e50 0x7fe28c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 -- 172.21.15.83:0/318048068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c108780 msgr2=0x7fe28c1a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c108780 0x7fe28c1a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 -- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe28c105e50 msgr2=0x7fe28c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe28c105e50 0x7fe28c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 -- 172.21.15.83:0/318048068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c1a2ab0 con 0x7fe28c103520 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.321+0000 7fe28a59c700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c103520 0x7fe28c19ae50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe28400ba50 tx=0x7fe28400be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.322+0000 7fe27b7fe700 1 -- 172.21.15.83:0/318048068 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284019070 con 0x7fe28c103520 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.322+0000 7fe27b7fe700 1 -- 172.21.15.83:0/318048068 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe284014440 con 0x7fe28c103520 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.322+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c1a2da0 con 0x7fe28c103520 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.322+0000 7fe27b7fe700 1 -- 172.21.15.83:0/318048068 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840075d0 con 0x7fe28c103520 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.322+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c1a31f0 con 0x7fe28c103520 2024-04-21T09:04:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.323+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe28c069730 con 0x7fe28c103520 2024-04-21T09:04:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.324+0000 7fe27b7fe700 1 -- 172.21.15.83:0/318048068 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe28401c070 con 0x7fe28c103520 2024-04-21T09:04:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.324+0000 7fe27b7fe700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe2740a3c40 0x7fe2740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.325+0000 7fe27b7fe700 1 -- 172.21.15.83:0/318048068 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fe2840cf480 con 0x7fe28c103520 2024-04-21T09:04:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.325+0000 7fe289d9b700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe2740a3c40 0x7fe2740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.325+0000 7fe289d9b700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe2740a3c40 0x7fe2740a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe28000b5c0 tx=0x7fe280005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.328+0000 7fe27b7fe700 1 -- 172.21.15.83:0/318048068 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe28409c660 con 0x7fe28c103520 2024-04-21T09:04:07.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:07.536+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe28c062e80 con 0x7fe2740a3c40 2024-04-21T09:04:08.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:07 smithi066 ceph-mon[27953]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:08.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:04:08.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:04:08.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:08.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:07 smithi083 ceph-mon[31413]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:08.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:04:08.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:04:08.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:08.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:07 smithi186 ceph-mon[31505]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:08.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:04:08.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:04:08.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:09.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:08 smithi066 ceph-mon[27953]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:04:09.126 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:08 smithi083 ceph-mon[31413]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:04:09.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:08 smithi186 ceph-mon[31505]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:04:10.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:09 smithi186 ceph-mon[31505]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:09 smithi083 ceph-mon[31413]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:10.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:09 smithi066 ceph-mon[27953]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:11.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/3089100647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]: dispatch 2024-04-21T09:04:11.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:10 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]: dispatch 2024-04-21T09:04:11.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:10 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]': finished 2024-04-21T09:04:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:10 smithi083 ceph-mon[31413]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T09:04:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/3089100647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]: dispatch 2024-04-21T09:04:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:10 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]: dispatch 2024-04-21T09:04:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:10 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]': finished 2024-04-21T09:04:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:10 smithi186 ceph-mon[31505]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T09:04:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:11.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/3089100647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]: dispatch 2024-04-21T09:04:11.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:10 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]: dispatch 2024-04-21T09:04:11.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:10 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a1a9369-e557-41ed-99a8-942cc6a1bd91"}]': finished 2024-04-21T09:04:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:10 smithi066 ceph-mon[27953]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T09:04:11.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:12.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:11 smithi186 ceph-mon[31505]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.83:0/1662536327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:04:12.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:11 smithi083 ceph-mon[31413]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:12.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.83:0/1662536327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:04:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:11 smithi066 ceph-mon[27953]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.83:0/1662536327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:04:14.319 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:13 smithi083 ceph-mon[31413]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:14.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:13 smithi186 ceph-mon[31505]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:14.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:13 smithi066 ceph-mon[27953]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:16.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:15 smithi083 ceph-mon[31413]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:16.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:15 smithi186 ceph-mon[31505]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:16.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:15 smithi066 ceph-mon[27953]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:18.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:17 smithi186 ceph-mon[31505]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:18.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:17 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T09:04:18.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:17 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:18.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:17 smithi083 ceph-mon[31413]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:18.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:17 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T09:04:18.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:17 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:18.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:17 smithi066 ceph-mon[27953]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:18.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:17 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T09:04:18.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:17 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:19.239 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:18 smithi186 ceph-mon[31505]: Deploying daemon osd.10 on smithi083 2024-04-21T09:04:19.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:18 smithi083 ceph-mon[31413]: Deploying daemon osd.10 on smithi083 2024-04-21T09:04:19.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:18 smithi066 ceph-mon[27953]: Deploying daemon osd.10 on smithi083 2024-04-21T09:04:20.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:19 smithi083 ceph-mon[31413]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:20.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:19 smithi186 ceph-mon[31505]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:20.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:19 smithi066 ceph-mon[27953]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:22.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:21 smithi083 ceph-mon[31413]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:22.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:21 smithi186 ceph-mon[31505]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:22.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:21 smithi066 ceph-mon[27953]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:24.189 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:23 smithi083 ceph-mon[31413]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:24.189 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:24.189 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:24.189 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:24.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:23 smithi186 ceph-mon[31505]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:24.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:24.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:24.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:24.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:23 smithi066 ceph-mon[27953]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:24.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:24.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:24.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:26.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:25 smithi083 ceph-mon[31413]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:26.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:25 smithi186 ceph-mon[31505]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:26.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:25 smithi066 ceph-mon[27953]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:27.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.005+0000 7fe27b7fe700 1 -- 172.21.15.83:0/318048068 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe28c062e80 con 0x7fe2740a3c40 2024-04-21T09:04:27.006 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 10 on host 'smithi083' 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.007+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe2740a3c40 msgr2=0x7fe2740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.007+0000 7fe290881700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe2740a3c40 0x7fe2740a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe28000b5c0 tx=0x7fe280005fd0 comp rx=0 tx=0).stop 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.007+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c103520 msgr2=0x7fe28c19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.007+0000 7fe290881700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c103520 0x7fe28c19ae50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe28400ba50 tx=0x7fe28400be10 comp rx=0 tx=0).stop 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 shutdown_connections 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe2740a3c40 0x7fe2740a6100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe28c103520 0x7fe28c19ae50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c108780 0x7fe28c1a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 --2- 172.21.15.83:0/318048068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe28c105e50 0x7fe28c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 >> 172.21.15.83:0/318048068 conn(0x7fe28c0fcf30 msgr2=0x7fe28c102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 shutdown_connections 2024-04-21T09:04:27.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-21T09:04:27.008+0000 7fe290881700 1 -- 172.21.15.83:0/318048068 wait complete. 2024-04-21T09:04:28.047 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:27 smithi083 ceph-mon[31413]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:28.047 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:28.047 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:28.047 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:27 smithi083 ceph-mon[31413]: from='osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T09:04:28.047 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:27 smithi083 ceph-mon[31413]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T09:04:28.075 DEBUG:teuthology.orchestra.run.smithi083:osd.10> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.10.service 2024-04-21T09:04:28.078 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2024-04-21T09:04:28.078 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T09:04:28.329 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:04:28.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:27 smithi186 ceph-mon[31505]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:28.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:28.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:28.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:27 smithi186 ceph-mon[31505]: from='osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T09:04:28.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:27 smithi186 ceph-mon[31505]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T09:04:28.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:27 smithi066 ceph-mon[27953]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:28.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:28.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:28.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:27 smithi066 ceph-mon[27953]: from='osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T09:04:28.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:27 smithi066 ceph-mon[27953]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T09:04:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:28 smithi186 ceph-mon[31505]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T09:04:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:28 smithi186 ceph-mon[31505]: from='osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:04:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:28 smithi186 ceph-mon[31505]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T09:04:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:28 smithi186 ceph-mon[31505]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:04:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:28 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:28 smithi083 ceph-mon[31413]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T09:04:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:28 smithi083 ceph-mon[31413]: from='osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:04:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:28 smithi083 ceph-mon[31413]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T09:04:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:28 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:28 smithi083 ceph-mon[31413]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:04:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:28 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:28 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:29.271 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:28 smithi066 ceph-mon[27953]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T09:04:29.271 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:28 smithi066 ceph-mon[27953]: from='osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:04:29.271 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:28 smithi066 ceph-mon[27953]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T09:04:29.271 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:29.271 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:28 smithi066 ceph-mon[27953]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-21T09:04:29.272 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:29.272 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:28 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:29.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:04:29.430+0000 7ff5e66c4700 -1 osd.10 0 waiting for initial osdmap 2024-04-21T09:04:29.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:04:29.446+0000 7ff5de4af700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: from='osd.10 ' entity='osd.10' 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] boot 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T09:04:30.053 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: from='osd.10 ' entity='osd.10' 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] boot 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T09:04:30.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: from='osd.10 ' entity='osd.10' 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: osd.10 [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] boot 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T09:04:30.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:04:30.549 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T09:04:30.549 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T09:04:30.549 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-21T09:04:30.549 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-21T09:04:30.550 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117824 s, 890 MB/s 2024-04-21T09:04:30.550 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-21T09:04:31.289 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:31 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:04:31.289 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:31 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:04:31.289 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:31 smithi186 ceph-mon[31505]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T09:04:31.293 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:31 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:04:31.293 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:31 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:04:31.293 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:31 smithi083 ceph-mon[31413]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T09:04:31.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:31 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:04:31.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:31 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:04:31.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:31 smithi066 ceph-mon[27953]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T09:04:31.510 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2024-04-21T09:04:31.748 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:04:32.123 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:32 smithi186 ceph-mon[31505]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:32.123 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:32 smithi186 ceph-mon[31505]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T09:04:32.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:32 smithi083 ceph-mon[31413]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:32.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:32 smithi083 ceph-mon[31413]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T09:04:32.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:32 smithi066 ceph-mon[27953]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:32.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:32 smithi066 ceph-mon[27953]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T09:04:33.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi083 2024-04-21T09:04:33.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:33.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:33.314 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.558 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 -- 172.21.15.186:0/1941240133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c01035d0 msgr2=0x7f96c01059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:33.558 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 --2- 172.21.15.186:0/1941240133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c01035d0 0x7f96c01059c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009990 tx=0x7f96b0009ca0 comp rx=0 tx=0).stop 2024-04-21T09:04:33.558 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 -- 172.21.15.186:0/1941240133 shutdown_connections 2024-04-21T09:04:33.558 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 --2- 172.21.15.186:0/1941240133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96c0108830 0x7f96c010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:33.558 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 --2- 172.21.15.186:0/1941240133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c01035d0 0x7f96c01059c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:33.558 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 --2- 172.21.15.186:0/1941240133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96c0105f00 0x7f96c01082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:33.558 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 -- 172.21.15.186:0/1941240133 >> 172.21.15.186:0/1941240133 conn(0x7f96c00fcfa0 msgr2=0x7f96c00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 -- 172.21.15.186:0/1941240133 shutdown_connections 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.531+0000 7f96c766b700 1 -- 172.21.15.186:0/1941240133 wait complete. 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 Processor -- start 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 -- start start 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96c01035d0 0x7f96c0198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c0105f00 0x7f96c01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96c0108830 0x7f96c01a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96c010e600 con 0x7f96c01035d0 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96c010e480 con 0x7f96c0108830 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c766b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f96c010e780 con 0x7f96c0105f00 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c4c06700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c0105f00 0x7f96c01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:33.559 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c4c06700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c0105f00 0x7f96c01991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57772/0 (socket says 172.21.15.186:57772) 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.532+0000 7f96c4c06700 1 -- 172.21.15.186:0/1040175753 learned_addr learned my addr 172.21.15.186:0/1040175753 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c5407700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96c01035d0 0x7f96c0198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c5c08700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96c0108830 0x7f96c01a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c4c06700 1 -- 172.21.15.186:0/1040175753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96c0108830 msgr2=0x7f96c01a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c4c06700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96c0108830 0x7f96c01a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c4c06700 1 -- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96c01035d0 msgr2=0x7f96c0198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c4c06700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96c01035d0 0x7f96c0198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c4c06700 1 -- 172.21.15.186:0/1040175753 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c01a0930 con 0x7f96c0105f00 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c5c08700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96c0108830 0x7f96c01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c4c06700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c0105f00 0x7f96c01991f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b530 tx=0x7f96bc00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96b67fc700 1 -- 172.21.15.186:0/1040175753 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc015020 con 0x7f96c0105f00 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96b67fc700 1 -- 172.21.15.186:0/1040175753 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96bc011ce0 con 0x7f96c0105f00 2024-04-21T09:04:33.560 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c01a0c20 con 0x7f96c0105f00 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96b67fc700 1 -- 172.21.15.186:0/1040175753 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc010e40 con 0x7f96c0105f00 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.533+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a1160 con 0x7f96c0105f00 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.534+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c0069730 con 0x7f96c0105f00 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.535+0000 7f96b67fc700 1 -- 172.21.15.186:0/1040175753 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f96bc018020 con 0x7f96c0105f00 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.536+0000 7f96b67fc700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96ac0a3c40 0x7f96ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.536+0000 7f96b67fc700 1 -- 172.21.15.186:0/1040175753 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f96bc0cfb00 con 0x7f96c0105f00 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.536+0000 7f96c5407700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96ac0a3c40 0x7f96ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.536+0000 7f96c5407700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96ac0a3c40 0x7f96ac0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b5c0 tx=0x7f96b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:33.561 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.540+0000 7f96b67fc700 1 -- 172.21.15.186:0/1040175753 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96bc01f020 con 0x7f96c0105f00 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi083 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:33.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:33.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:33.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T09:04:33.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:33.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:33.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi083 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:33.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T09:04:33.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:33.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:33.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:33.749 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:33.749+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f96c0062e80 con 0x7f96ac0a3c40 2024-04-21T09:04:35.018 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:34 smithi186 ceph-mon[31505]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T09:04:35.018 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:34 smithi186 ceph-mon[31505]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:04:35.018 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:34 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:04:35.018 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:34 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:04:35.018 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:34 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:35.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:34 smithi083 ceph-mon[31413]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T09:04:35.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:34 smithi083 ceph-mon[31413]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:04:35.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:34 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:04:35.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:34 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:04:35.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:34 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:35.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:34 smithi066 ceph-mon[27953]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T09:04:35.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:34 smithi066 ceph-mon[27953]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:04:35.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:34 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T09:04:35.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:34 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T09:04:35.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:34 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:35.700 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:35 smithi186 ceph-mon[31505]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T09:04:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:35 smithi083 ceph-mon[31413]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T09:04:36.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:35 smithi066 ceph-mon[27953]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T09:04:36.979 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:36 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/260265800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]: dispatch 2024-04-21T09:04:36.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:36 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]: dispatch 2024-04-21T09:04:36.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:36 smithi186 ceph-mon[31505]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]': finished 2024-04-21T09:04:36.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:36 smithi186 ceph-mon[31505]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T09:04:36.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:37.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:36 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/260265800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]: dispatch 2024-04-21T09:04:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:36 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]: dispatch 2024-04-21T09:04:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:36 smithi083 ceph-mon[31413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]': finished 2024-04-21T09:04:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:36 smithi083 ceph-mon[31413]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T09:04:37.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:36 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/260265800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]: dispatch 2024-04-21T09:04:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:36 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]: dispatch 2024-04-21T09:04:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:36 smithi066 ceph-mon[27953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc"}]': finished 2024-04-21T09:04:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:36 smithi066 ceph-mon[27953]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T09:04:37.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:38.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:37 smithi186 ceph-mon[31505]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-21T09:04:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:37 smithi186 ceph-mon[31505]: from='client.? 172.21.15.186:0/495496977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:04:38.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:37 smithi083 ceph-mon[31413]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-21T09:04:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:37 smithi083 ceph-mon[31413]: from='client.? 172.21.15.186:0/495496977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:04:38.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:37 smithi066 ceph-mon[27953]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-21T09:04:38.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:37 smithi066 ceph-mon[27953]: from='client.? 172.21.15.186:0/495496977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T09:04:40.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:39 smithi066 ceph-mon[27953]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:04:40.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:39 smithi186 ceph-mon[31505]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:04:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:39 smithi083 ceph-mon[31413]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T09:04:42.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:41 smithi083 ceph-mon[31413]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-21T09:04:42.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:41 smithi066 ceph-mon[27953]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-21T09:04:42.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:41 smithi186 ceph-mon[31505]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-21T09:04:43.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:43 smithi186 ceph-mon[31505]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:44.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:43 smithi083 ceph-mon[31413]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:44.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:43 smithi066 ceph-mon[27953]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:44.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T09:04:44.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:44 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:45.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T09:04:45.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:44 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:45.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T09:04:45.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:44 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:46.037 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:45 smithi186 ceph-mon[31505]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:46.037 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:45 smithi186 ceph-mon[31505]: Deploying daemon osd.11 on smithi186 2024-04-21T09:04:46.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:45 smithi083 ceph-mon[31413]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:46.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:45 smithi083 ceph-mon[31413]: Deploying daemon osd.11 on smithi186 2024-04-21T09:04:46.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:45 smithi066 ceph-mon[27953]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:46.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:45 smithi066 ceph-mon[27953]: Deploying daemon osd.11 on smithi186 2024-04-21T09:04:47.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:47 smithi186 ceph-mon[31505]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:48.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:47 smithi083 ceph-mon[31413]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:48.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:47 smithi066 ceph-mon[27953]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:49.912 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:49 smithi083 ceph-mon[31413]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:49.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:49 smithi186 ceph-mon[31505]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:50.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:49 smithi066 ceph-mon[27953]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:51.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:51.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:51.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:51 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:51.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:51 smithi083 ceph-mon[31413]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:51.441 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:51.442 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:51.442 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:51 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:51.442 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:51 smithi186 ceph-mon[31505]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:51.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:51 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:51 smithi066 ceph-mon[27953]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:53.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:53 smithi186 ceph-mon[31505]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:53.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:53 smithi083 ceph-mon[31413]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:53 smithi066 ceph-mon[27953]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:54.222 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.221+0000 7f96b67fc700 1 -- 172.21.15.186:0/1040175753 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f96c0062e80 con 0x7f96ac0a3c40 2024-04-21T09:04:54.223 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96ac0a3c40 msgr2=0x7f96ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96ac0a3c40 0x7f96ac0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b5c0 tx=0x7f96b0005fb0 comp rx=0 tx=0).stop 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c0105f00 msgr2=0x7f96c01991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c0105f00 0x7f96c01991f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b530 tx=0x7f96bc00b840 comp rx=0 tx=0).stop 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 shutdown_connections 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96ac0a3c40 0x7f96ac0a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96c0108830 0x7f96c01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96c01035d0 0x7f96c0198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 --2- 172.21.15.186:0/1040175753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96c0105f00 0x7f96c01991f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 >> 172.21.15.186:0/1040175753 conn(0x7f96c00fcfa0 msgr2=0x7f96c0103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 shutdown_connections 2024-04-21T09:04:54.225 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:04:54.224+0000 7f96c766b700 1 -- 172.21.15.186:0/1040175753 wait complete. 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:54 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:54 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:54.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:54 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:04:54.969 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.11.service 2024-04-21T09:04:54.972 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-21T09:04:54.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-21T09:04:55.232 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:04:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-mon[31505]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-mon[31505]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T09:04:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-mon[31505]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T09:04:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-mon[31505]: from='osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:04:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-mon[31505]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:04:55.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:04:55.403+0000 7f949f69c700 -1 osd.11 0 waiting for initial osdmap 2024-04-21T09:04:55.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:04:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:04:55.416+0000 7f949b48f700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T09:04:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:55 smithi066 ceph-mon[27953]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:55 smithi066 ceph-mon[27953]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T09:04:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:55 smithi066 ceph-mon[27953]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T09:04:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:55 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:55 smithi066 ceph-mon[27953]: from='osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:04:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:55 smithi066 ceph-mon[27953]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:04:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:55 smithi083 ceph-mon[31413]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:55 smithi083 ceph-mon[31413]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T09:04:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:55 smithi083 ceph-mon[31413]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T09:04:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:55 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:55 smithi083 ceph-mon[31413]: from='osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:04:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:55 smithi083 ceph-mon[31413]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-21T09:04:56.463 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:56 smithi186 ceph-mon[31505]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:04:56.464 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:56 smithi186 ceph-mon[31505]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T09:04:56.464 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:56 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:56.464 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:56 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:56.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:56 smithi066 ceph-mon[27953]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:04:56.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:56 smithi066 ceph-mon[27953]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T09:04:56.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:56 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:56.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:56 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:56.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:56 smithi083 ceph-mon[31413]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-21T09:04:56.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:56 smithi083 ceph-mon[31413]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T09:04:56.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:56 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:56.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:56 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:56.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 -- 172.21.15.66:0/2378645084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe81019a0 msgr2=0x7f7fe8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:56.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 --2- 172.21.15.66:0/2378645084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe81019a0 0x7f7fe8101dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80099c0 tx=0x7f7fd8009cd0 comp rx=0 tx=0).stop 2024-04-21T09:04:56.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 -- 172.21.15.66:0/2378645084 shutdown_connections 2024-04-21T09:04:56.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 --2- 172.21.15.66:0/2378645084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe8103540 0x7f7fe8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:56.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 --2- 172.21.15.66:0/2378645084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe81019a0 0x7f7fe8101dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 --2- 172.21.15.66:0/2378645084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe8102ba0 0x7f7fe8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 -- 172.21.15.66:0/2378645084 >> 172.21.15.66:0/2378645084 conn(0x7f7fe80fcf40 msgr2=0x7f7fe80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 -- 172.21.15.66:0/2378645084 shutdown_connections 2024-04-21T09:04:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.924+0000 7f7fee309700 1 -- 172.21.15.66:0/2378645084 wait complete. 2024-04-21T09:04:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 Processor -- start 2024-04-21T09:04:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 -- start start 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe81019a0 0x7f7fe8198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe8102ba0 0x7f7fe8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe8103540 0x7f7fe81a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810d4b0 con 0x7f7fe8102ba0 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810d330 con 0x7f7fe8103540 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fee309700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810d630 con 0x7f7fe81019a0 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fe77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe8102ba0 0x7f7fe8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fe77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe8102ba0 0x7f7fe8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50294/0 (socket says 172.21.15.66:50294) 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.925+0000 7f7fe77fe700 1 -- 172.21.15.66:0/1084430639 learned_addr learned my addr 172.21.15.66:0/1084430639 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:04:56.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe7fff700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe81019a0 0x7f7fe8198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fec8a6700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe8103540 0x7f7fe81a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe77fe700 1 -- 172.21.15.66:0/1084430639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe81019a0 msgr2=0x7f7fe8198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe77fe700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe81019a0 0x7f7fe8198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe77fe700 1 -- 172.21.15.66:0/1084430639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe8103540 msgr2=0x7f7fe81a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe77fe700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe8103540 0x7f7fe81a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe77fe700 1 -- 172.21.15.66:0/1084430639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe81a08c0 con 0x7f7fe8102ba0 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe77fe700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe8102ba0 0x7f7fe8199200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d980 tx=0x7f7fd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe57fa700 1 -- 172.21.15.66:0/1084430639 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0017070 con 0x7f7fe8102ba0 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe8103e20 con 0x7f7fe8102ba0 2024-04-21T09:04:56.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe57fa700 1 -- 172.21.15.66:0/1084430639 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd0005020 con 0x7f7fe8102ba0 2024-04-21T09:04:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fe57fa700 1 -- 172.21.15.66:0/1084430639 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0009b50 con 0x7f7fe8102ba0 2024-04-21T09:04:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.926+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe81042e0 con 0x7f7fe8102ba0 2024-04-21T09:04:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.927+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe8069730 con 0x7f7fe8102ba0 2024-04-21T09:04:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.928+0000 7f7fe57fa700 1 -- 172.21.15.66:0/1084430639 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7fd00051d0 con 0x7f7fe8102ba0 2024-04-21T09:04:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.929+0000 7f7fe57fa700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7fd40a3c40 0x7f7fd40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.929+0000 7f7fe7fff700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7fd40a3c40 0x7f7fd40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.929+0000 7f7fe57fa700 1 -- 172.21.15.66:0/1084430639 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7fd00d0260 con 0x7f7fe8102ba0 2024-04-21T09:04:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.929+0000 7f7fe7fff700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7fd40a3c40 0x7f7fd40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006010 tx=0x7f7fd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:56.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:56.933+0000 7f7fe57fa700 1 -- 172.21.15.66:0/1084430639 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fd0020020 con 0x7f7fe8102ba0 2024-04-21T09:04:57.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.143+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7fe8104810 con 0x7f7fe8102ba0 2024-04-21T09:04:57.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.143+0000 7f7fe57fa700 1 -- 172.21.15.66:0/1084430639 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f7fd009d0e0 con 0x7f7fe8102ba0 2024-04-21T09:04:57.143 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:04:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.145+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7fd40a3c40 msgr2=0x7f7fd40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.145+0000 7f7fee309700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7fd40a3c40 0x7f7fd40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006010 tx=0x7f7fd8000bc0 comp rx=0 tx=0).stop 2024-04-21T09:04:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.145+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe8102ba0 msgr2=0x7f7fe8199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.145+0000 7f7fee309700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe8102ba0 0x7f7fe8199200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d980 tx=0x7f7fd000dc90 comp rx=0 tx=0).stop 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 shutdown_connections 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7fd40a3c40 0x7f7fd40a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe8103540 0x7f7fe81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe81019a0 0x7f7fe8198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 --2- 172.21.15.66:0/1084430639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe8102ba0 0x7f7fe8199200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 >> 172.21.15.66:0/1084430639 conn(0x7f7fe80fcf40 msgr2=0x7f7fe80fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 shutdown_connections 2024-04-21T09:04:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:57.146+0000 7f7fee309700 1 -- 172.21.15.66:0/1084430639 wait complete. 2024-04-21T09:04:57.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: purged_snaps scrub starts 2024-04-21T09:04:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: purged_snaps scrub ok 2024-04-21T09:04:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] boot 2024-04-21T09:04:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T09:04:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:57 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1084430639' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T09:04:57.766 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: purged_snaps scrub starts 2024-04-21T09:04:57.766 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: purged_snaps scrub ok 2024-04-21T09:04:57.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:57.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:57.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] boot 2024-04-21T09:04:57.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T09:04:57.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:57.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:57 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1084430639' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T09:04:57.794 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713690296,"num_in_osds":12,"osd_in_since":1713690276,"num_remapped_pgs":0} 2024-04-21T09:04:57.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T09:04:57.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: purged_snaps scrub starts 2024-04-21T09:04:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: purged_snaps scrub ok 2024-04-21T09:04:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-21T09:04:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: osd.11 [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] boot 2024-04-21T09:04:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T09:04:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:04:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:57 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1084430639' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T09:04:58.053 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:04:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T09:04:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: Detected new or changed devices on smithi186 2024-04-21T09:04:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:58.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:04:58 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: Detected new or changed devices on smithi186 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:58.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:04:58 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: Detected new or changed devices on smithi186 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:04:58.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:04:58 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 -- 172.21.15.66:0/3371406082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 msgr2=0x7fe4d410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 --2- 172.21.15.66:0/3371406082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d410f900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009990 tx=0x7fe4cc009ca0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 -- 172.21.15.66:0/3371406082 shutdown_connections 2024-04-21T09:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 --2- 172.21.15.66:0/3371406082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d410f900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 --2- 172.21.15.66:0/3371406082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4d4108a30 0x7fe4d4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 --2- 172.21.15.66:0/3371406082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d406a750 0x7fe4d406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 -- 172.21.15.66:0/3371406082 >> 172.21.15.66:0/3371406082 conn(0x7fe4d40788f0 msgr2=0x7fe4d4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 -- 172.21.15.66:0/3371406082 shutdown_connections 2024-04-21T09:04:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.735+0000 7fe4db480700 1 -- 172.21.15.66:0/3371406082 wait complete. 2024-04-21T09:04:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 Processor -- start 2024-04-21T09:04:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 -- start start 2024-04-21T09:04:59.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d406a750 0x7fe4d4199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:59.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:59.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4d4108a30 0x7fe4d419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:59.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4112c20 con 0x7fe4d406b0f0 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4112aa0 con 0x7fe4d406a750 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4db480700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4112da0 con 0x7fe4d4108a30 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50306/0 (socket says 172.21.15.66:50306) 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 -- 172.21.15.66:0/1230698508 learned_addr learned my addr 172.21.15.66:0/1230698508 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d921c700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d406a750 0x7fe4d4199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d9a1d700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4d4108a30 0x7fe4d419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 -- 172.21.15.66:0/1230698508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4d4108a30 msgr2=0x7fe4d419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4d4108a30 0x7fe4d419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 -- 172.21.15.66:0/1230698508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d406a750 msgr2=0x7fe4d4199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d406a750 0x7fe4d4199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.736+0000 7fe4d8a1b700 1 -- 172.21.15.66:0/1230698508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d419e080 con 0x7fe4d406b0f0 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.737+0000 7fe4d8a1b700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d41995b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000c9b0 tx=0x7fe4d000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:59.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.737+0000 7fe4c67fc700 1 -- 172.21.15.66:0/1230698508 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0007b00 con 0x7fe4d406b0f0 2024-04-21T09:04:59.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.737+0000 7fe4c67fc700 1 -- 172.21.15.66:0/1230698508 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d000ce50 con 0x7fe4d406b0f0 2024-04-21T09:04:59.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.737+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d419e370 con 0x7fe4d406b0f0 2024-04-21T09:04:59.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.737+0000 7fe4c67fc700 1 -- 172.21.15.66:0/1230698508 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0018660 con 0x7fe4d406b0f0 2024-04-21T09:04:59.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.737+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d41a5bf0 con 0x7fe4d406b0f0 2024-04-21T09:04:59.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.738+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d4069730 con 0x7fe4d406b0f0 2024-04-21T09:04:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.739+0000 7fe4c67fc700 1 -- 172.21.15.66:0/1230698508 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe4d000f450 con 0x7fe4d406b0f0 2024-04-21T09:04:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.740+0000 7fe4c67fc700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c00a3c40 0x7fe4c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:04:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.740+0000 7fe4d921c700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c00a3c40 0x7fe4c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:04:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.740+0000 7fe4c67fc700 1 -- 172.21.15.66:0/1230698508 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe4d00cfe70 con 0x7fe4d406b0f0 2024-04-21T09:04:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.740+0000 7fe4d921c700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c00a3c40 0x7fe4c00a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009550 tx=0x7fe4c80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:04:59.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.745+0000 7fe4c67fc700 1 -- 172.21.15.66:0/1230698508 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4d0014030 con 0x7fe4d406b0f0 2024-04-21T09:04:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.954+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4d402df80 con 0x7fe4d406b0f0 2024-04-21T09:04:59.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.955+0000 7fe4c67fc700 1 -- 172.21.15.66:0/1230698508 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18418 (secure 0 0 0) 0x7fe4d009ccf0 con 0x7fe4d406b0f0 2024-04-21T09:04:59.955 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:04:59.956 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:04:57.408258+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c00a3c40 msgr2=0x7fe4c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c00a3c40 0x7fe4c00a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009550 tx=0x7fe4c80091b0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 msgr2=0x7fe4d41995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d41995b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000c9b0 tx=0x7fe4d000ccc0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 shutdown_connections 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c00a3c40 0x7fe4c00a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d406a750 0x7fe4d4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4d4108a30 0x7fe4d419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 --2- 172.21.15.66:0/1230698508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d406b0f0 0x7fe4d41995b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:04:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 >> 172.21.15.66:0/1230698508 conn(0x7fe4d40788f0 msgr2=0x7fe4d40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:04:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 shutdown_connections 2024-04-21T09:04:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:04:59.957+0000 7fe4db480700 1 -- 172.21.15.66:0/1230698508 wait complete. 2024-04-21T09:05:00.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:00 smithi066 ceph-mon[27953]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:00.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:00 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1230698508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:00.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:00 smithi083 ceph-mon[31413]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:00.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:00 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1230698508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:00 smithi186 ceph-mon[31505]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:00 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1230698508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:00.657 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T09:05:00.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-21T09:05:00.903 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:05:02.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:02 smithi066 ceph-mon[27953]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:02.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:02 smithi186 ceph-mon[31505]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:02.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:02 smithi083 ceph-mon[31413]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 -- 172.21.15.66:0/3448154130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c103480 msgr2=0x7efc4c10f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 --2- 172.21.15.66:0/3448154130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c103480 0x7efc4c10f8d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b0a0 tx=0x7efc4400b3b0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 -- 172.21.15.66:0/3448154130 shutdown_connections 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 --2- 172.21.15.66:0/3448154130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c108a70 0x7efc4c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 --2- 172.21.15.66:0/3448154130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c103480 0x7efc4c10f8d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 --2- 172.21.15.66:0/3448154130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc4c102ac0 0x7efc4c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 -- 172.21.15.66:0/3448154130 >> 172.21.15.66:0/3448154130 conn(0x7efc4c079bb0 msgr2=0x7efc4c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 -- 172.21.15.66:0/3448154130 shutdown_connections 2024-04-21T09:05:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.630+0000 7efc53ff2700 1 -- 172.21.15.66:0/3448154130 wait complete. 2024-04-21T09:05:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 Processor -- start 2024-04-21T09:05:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 -- start start 2024-04-21T09:05:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc4c102ac0 0x7efc4c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c103480 0x7efc4c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c108a70 0x7efc4c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c112ca0 con 0x7efc4c108a70 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c112b20 con 0x7efc4c103480 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.631+0000 7efc53ff2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c112e20 con 0x7efc4c102ac0 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c108a70 0x7efc4c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c108a70 0x7efc4c0757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50320/0 (socket says 172.21.15.66:50320) 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 -- 172.21.15.66:0/1609160859 learned_addr learned my addr 172.21.15.66:0/1609160859 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5158d700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c103480 0x7efc4c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc51d8e700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc4c102ac0 0x7efc4c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 -- 172.21.15.66:0/1609160859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc4c102ac0 msgr2=0x7efc4c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc4c102ac0 0x7efc4c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 -- 172.21.15.66:0/1609160859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c103480 msgr2=0x7efc4c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c103480 0x7efc4c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 -- 172.21.15.66:0/1609160859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c1a7640 con 0x7efc4c108a70 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc5258f700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c108a70 0x7efc4c0757c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc440062a0 tx=0x7efc44004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc42ffd700 1 -- 172.21.15.66:0/1609160859 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44021070 con 0x7efc4c108a70 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c1a78d0 con 0x7efc4c108a70 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc42ffd700 1 -- 172.21.15.66:0/1609160859 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc44003c00 con 0x7efc4c108a70 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc42ffd700 1 -- 172.21.15.66:0/1609160859 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44004830 con 0x7efc4c108a70 2024-04-21T09:05:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.632+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1a7e60 con 0x7efc4c108a70 2024-04-21T09:05:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.633+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c069730 con 0x7efc4c108a70 2024-04-21T09:05:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.635+0000 7efc42ffd700 1 -- 172.21.15.66:0/1609160859 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efc44007a00 con 0x7efc4c108a70 2024-04-21T09:05:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.635+0000 7efc42ffd700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efc380a3c40 0x7efc380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.635+0000 7efc51d8e700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efc380a3c40 0x7efc380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.635+0000 7efc42ffd700 1 -- 172.21.15.66:0/1609160859 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efc4400f020 con 0x7efc4c108a70 2024-04-21T09:05:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.636+0000 7efc51d8e700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efc380a3c40 0x7efc380a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc48005950 tx=0x7efc4800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.640+0000 7efc42ffd700 1 -- 172.21.15.66:0/1609160859 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc4402a020 con 0x7efc4c108a70 2024-04-21T09:05:02.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.847+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc4c02df80 con 0x7efc4c108a70 2024-04-21T09:05:02.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.848+0000 7efc42ffd700 1 -- 172.21.15.66:0/1609160859 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7efc4409e400 con 0x7efc4c108a70 2024-04-21T09:05:02.848 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efc380a3c40 msgr2=0x7efc380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efc380a3c40 0x7efc380a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc48005950 tx=0x7efc4800a300 comp rx=0 tx=0).stop 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c108a70 msgr2=0x7efc4c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c108a70 0x7efc4c0757c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc440062a0 tx=0x7efc44004e20 comp rx=0 tx=0).stop 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 shutdown_connections 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efc380a3c40 0x7efc380a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4c108a70 0x7efc4c0757c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc4c102ac0 0x7efc4c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 --2- 172.21.15.66:0/1609160859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c103480 0x7efc4c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:02.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 >> 172.21.15.66:0/1609160859 conn(0x7efc4c079bb0 msgr2=0x7efc4c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:02.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 shutdown_connections 2024-04-21T09:05:02.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:02.850+0000 7efc53ff2700 1 -- 172.21.15.66:0/1609160859 wait complete. 2024-04-21T09:05:03.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:03 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1609160859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:03.568 INFO:tasks.cephadm:Adding mds.a on smithi066 2024-04-21T09:05:03.568 INFO:tasks.cephadm:Adding mds.d on smithi066 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.g on smithi066 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.j on smithi066 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.b on smithi083 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.e on smithi083 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.h on smithi083 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.k on smithi083 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.c on smithi186 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.f on smithi186 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.i on smithi186 2024-04-21T09:05:03.569 INFO:tasks.cephadm:Adding mds.l on smithi186 2024-04-21T09:05:03.570 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2024-04-21T09:05:03.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:03 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1609160859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:03 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1609160859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:03.819 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:05:04.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:04 smithi186 ceph-mon[31505]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:04.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:04 smithi066 ceph-mon[27953]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:04.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:04 smithi083 ceph-mon[31413]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:05.501 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 -- 172.21.15.186:0/4204355421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180102330 msgr2=0x7f9180102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 --2- 172.21.15.186:0/4204355421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180102330 0x7f9180102790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9170009990 tx=0x7f9170009ca0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 -- 172.21.15.186:0/4204355421 shutdown_connections 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 --2- 172.21.15.186:0/4204355421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180101130 0x7f9180101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 --2- 172.21.15.186:0/4204355421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9180102cd0 0x7f9180109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 --2- 172.21.15.186:0/4204355421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180102330 0x7f9180102790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 -- 172.21.15.186:0/4204355421 >> 172.21.15.186:0/4204355421 conn(0x7f91800fc6b0 msgr2=0x7f91800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 -- 172.21.15.186:0/4204355421 shutdown_connections 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.498+0000 7f91869c4700 1 -- 172.21.15.186:0/4204355421 wait complete. 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 Processor -- start 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 -- start start 2024-04-21T09:05:05.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180101130 0x7f9180197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9180102330 0x7f9180198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180102cd0 0x7f918019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d4c0 con 0x7f9180102330 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d340 con 0x7f9180102cd0 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f91869c4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d640 con 0x7f9180101130 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f917ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180101130 0x7f9180197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f917ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180101130 0x7f9180197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:42130/0 (socket says 172.21.15.186:42130) 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f917ffff700 1 -- 172.21.15.186:0/2487301971 learned_addr learned my addr 172.21.15.186:0/2487301971 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f917f7fe700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9180102330 0x7f9180198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f9184f61700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180102cd0 0x7f918019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f917ffff700 1 -- 172.21.15.186:0/2487301971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180102cd0 msgr2=0x7f918019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.499+0000 7f917ffff700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180102cd0 0x7f918019f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.503 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f917ffff700 1 -- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9180102330 msgr2=0x7f9180198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f917ffff700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9180102330 0x7f9180198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f917ffff700 1 -- 172.21.15.186:0/2487301971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918010a090 con 0x7f9180101130 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f917ffff700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180101130 0x7f9180197ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f917400c970 tx=0x7f917400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f917d7fa700 1 -- 172.21.15.186:0/2487301971 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174004220 con 0x7f9180101130 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918010a380 con 0x7f9180101130 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f917d7fa700 1 -- 172.21.15.186:0/2487301971 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9174007d50 con 0x7f9180101130 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f917d7fa700 1 -- 172.21.15.186:0/2487301971 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174003d00 con 0x7f9180101130 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.500+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918010a8c0 con 0x7f9180101130 2024-04-21T09:05:05.504 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.501+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9180069730 con 0x7f9180101130 2024-04-21T09:05:05.505 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.502+0000 7f917d7fa700 1 -- 172.21.15.186:0/2487301971 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f91740078a0 con 0x7f9180101130 2024-04-21T09:05:05.505 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.503+0000 7f917d7fa700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91680a3c40 0x7f91680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:05.505 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.503+0000 7f917d7fa700 1 -- 172.21.15.186:0/2487301971 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f91740cfb10 con 0x7f9180101130 2024-04-21T09:05:05.505 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.503+0000 7f917f7fe700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91680a3c40 0x7f91680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:05.505 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.504+0000 7f917f7fe700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91680a3c40 0x7f91680a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f917000b5c0 tx=0x7f9170005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:05.508 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.507+0000 7f917d7fa700 1 -- 172.21.15.186:0/2487301971 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9174014030 con 0x7f9180101130 2024-04-21T09:05:05.716 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.715+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) -- 0x7f918002e2f0 con 0x7f91680a3c40 2024-04-21T09:05:05.724 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.724+0000 7f917d7fa700 1 -- 172.21.15.186:0/2487301971 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f918002e2f0 con 0x7f91680a3c40 2024-04-21T09:05:05.724 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91680a3c40 msgr2=0x7f91680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91680a3c40 0x7f91680a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f917000b5c0 tx=0x7f9170005fb0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180101130 msgr2=0x7f9180197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180101130 0x7f9180197ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f917400c970 tx=0x7f917400cc80 comp rx=0 tx=0).stop 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 shutdown_connections 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91680a3c40 0x7f91680a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180102cd0 0x7f918019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9180101130 0x7f9180197ae0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 --2- 172.21.15.186:0/2487301971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9180102330 0x7f9180198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:05.727 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 >> 172.21.15.186:0/2487301971 conn(0x7f91800fc6b0 msgr2=0x7f91801056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:05.728 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 shutdown_connections 2024-04-21T09:05:05.728 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-21T09:05:05.726+0000 7f91869c4700 1 -- 172.21.15.186:0/2487301971 wait complete. 2024-04-21T09:05:06.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:06 smithi066 ceph-mon[27953]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:06.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:06.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:06 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:06.425 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:06 smithi186 ceph-mon[31505]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:06.425 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:06.425 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:06 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:06.451 DEBUG:teuthology.orchestra.run.smithi066:mds.a> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.a.service 2024-04-21T09:05:06.454 DEBUG:teuthology.orchestra.run.smithi066:mds.d> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.d.service 2024-04-21T09:05:06.497 DEBUG:teuthology.orchestra.run.smithi066:mds.g> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.g.service 2024-04-21T09:05:06.541 DEBUG:teuthology.orchestra.run.smithi066:mds.j> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.j.service 2024-04-21T09:05:06.584 DEBUG:teuthology.orchestra.run.smithi083:mds.b> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.b.service 2024-04-21T09:05:06.587 DEBUG:teuthology.orchestra.run.smithi083:mds.e> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.e.service 2024-04-21T09:05:06.606 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:06 smithi083 ceph-mon[31413]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:06.606 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:06.606 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:06 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:06.607 DEBUG:teuthology.orchestra.run.smithi083:mds.h> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.h.service 2024-04-21T09:05:06.649 DEBUG:teuthology.orchestra.run.smithi083:mds.k> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.k.service 2024-04-21T09:05:06.692 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.c.service 2024-04-21T09:05:06.695 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.f.service 2024-04-21T09:05:06.737 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.i.service 2024-04-21T09:05:06.780 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.l.service 2024-04-21T09:05:06.823 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-21T09:05:06.823 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:05:06.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.935+0000 7f82feec1640 1 -- 172.21.15.66:0/1540157169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 msgr2=0x7f82f8104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:06.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.935+0000 7f82feec1640 1 --2- 172.21.15.66:0/1540157169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8104e90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009900 tx=0x7f82ec02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- 172.21.15.66:0/1540157169 shutdown_connections 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 --2- 172.21.15.66:0/1540157169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82f8107d00 0x7f82f810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 --2- 172.21.15.66:0/1540157169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82f81053d0 0x7f82f81077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 --2- 172.21.15.66:0/1540157169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8104e90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- 172.21.15.66:0/1540157169 >> 172.21.15.66:0/1540157169 conn(0x7f82f80fc470 msgr2=0x7f82f80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- 172.21.15.66:0/1540157169 shutdown_connections 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- 172.21.15.66:0/1540157169 wait complete. 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 Processor -- start 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- start start 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82f81053d0 0x7f82f8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:06.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82f8107d00 0x7f82f819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810db00 con 0x7f82f8102aa0 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810d980 con 0x7f82f81053d0 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82feec1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810dc80 con 0x7f82f8107d00 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.936+0000 7f82fcc5d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54020/0 (socket says 172.21.15.66:54020) 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 -- 172.21.15.66:0/2341543273 learned_addr learned my addr 172.21.15.66:0/2341543273 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fd45e640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82f8107d00 0x7f82f819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82f7fff640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82f81053d0 0x7f82f8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 -- 172.21.15.66:0/2341543273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82f8107d00 msgr2=0x7f82f819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82f8107d00 0x7f82f819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 -- 172.21.15.66:0/2341543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82f81053d0 msgr2=0x7f82f8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82f81053d0 0x7f82f8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:06.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 -- 172.21.15.66:0/2341543273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f819fe20 con 0x7f82f8102aa0 2024-04-21T09:05:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82f7fff640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82f81053d0 0x7f82f8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82fcc5d640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8198200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82ec006010 tx=0x7f82ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82f5ffb640 1 -- 172.21.15.66:0/2341543273 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec043020 con 0x7f82f8102aa0 2024-04-21T09:05:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82f5ffb640 1 -- 172.21.15.66:0/2341543273 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ec004440 con 0x7f82f8102aa0 2024-04-21T09:05:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f81a00b0 con 0x7f82f8102aa0 2024-04-21T09:05:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82f5ffb640 1 -- 172.21.15.66:0/2341543273 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec003bf0 con 0x7f82f8102aa0 2024-04-21T09:05:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.937+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f81a0600 con 0x7f82f8102aa0 2024-04-21T09:05:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.938+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f80fdf60 con 0x7f82f8102aa0 2024-04-21T09:05:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.940+0000 7f82f5ffb640 1 -- 172.21.15.66:0/2341543273 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f82ec046020 con 0x7f82f8102aa0 2024-04-21T09:05:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.940+0000 7f82f5ffb640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f82e80a3bb0 0x7f82e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.940+0000 7f82f7fff640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f82e80a3bb0 0x7f82e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.940+0000 7f82f5ffb640 1 -- 172.21.15.66:0/2341543273 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f82ec0f2320 con 0x7f82f8102aa0 2024-04-21T09:05:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.941+0000 7f82f7fff640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f82e80a3bb0 0x7f82e80a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82f8199710 tx=0x7f82e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:06.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:06.945+0000 7f82f5ffb640 1 -- 172.21.15.66:0/2341543273 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82ec04d020 con 0x7f82f8102aa0 2024-04-21T09:05:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.136+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82f8062b50 con 0x7f82f8102aa0 2024-04-21T09:05:07.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.137+0000 7f82f5ffb640 1 -- 172.21.15.66:0/2341543273 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18418 (secure 0 0 0) 0x7f82ec0bf1a0 con 0x7f82f8102aa0 2024-04-21T09:05:07.137 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:07.137 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:04:57.408258+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:05:07.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f82e80a3bb0 msgr2=0x7f82e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f82e80a3bb0 0x7f82e80a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82f8199710 tx=0x7f82e4005f90 comp rx=0 tx=0).stop 2024-04-21T09:05:07.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 msgr2=0x7f82f8198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8198200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82ec006010 tx=0x7f82ec004040 comp rx=0 tx=0).stop 2024-04-21T09:05:07.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 shutdown_connections 2024-04-21T09:05:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f82e80a3bb0 0x7f82e80a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82f8107d00 0x7f82f819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82f81053d0 0x7f82f8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 --2- 172.21.15.66:0/2341543273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82f8102aa0 0x7f82f8198200 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.139+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 >> 172.21.15.66:0/2341543273 conn(0x7f82f80fc470 msgr2=0x7f82f81034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.140+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 shutdown_connections 2024-04-21T09:05:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.140+0000 7f82feec1640 1 -- 172.21.15.66:0/2341543273 wait complete. 2024-04-21T09:05:07.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T09:05:07.150 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:05:07.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 -- 172.21.15.66:0/3181625447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c102a30 msgr2=0x7f5d4c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 --2- 172.21.15.66:0/3181625447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c102a30 0x7f5d4c104e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009900 tx=0x7f5d3c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:07.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 -- 172.21.15.66:0/3181625447 shutdown_connections 2024-04-21T09:05:07.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 --2- 172.21.15.66:0/3181625447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4c107c90 0x7f5d4c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 --2- 172.21.15.66:0/3181625447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d4c105360 0x7f5d4c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 --2- 172.21.15.66:0/3181625447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c102a30 0x7f5d4c104e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 -- 172.21.15.66:0/3181625447 >> 172.21.15.66:0/3181625447 conn(0x7f5d4c0fc440 msgr2=0x7f5d4c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 -- 172.21.15.66:0/3181625447 shutdown_connections 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.256+0000 7f5d54606640 1 -- 172.21.15.66:0/3181625447 wait complete. 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 Processor -- start 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 -- start start 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4c102a30 0x7f5d4c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d4c105360 0x7f5d4c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c107c90 0x7f5d4c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10daf0 con 0x7f5d4c107c90 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10d970 con 0x7f5d4c105360 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d54606640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10dc70 con 0x7f5d4c102a30 2024-04-21T09:05:07.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c107c90 0x7f5d4c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c107c90 0x7f5d4c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54038/0 (socket says 172.21.15.66:54038) 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 -- 172.21.15.66:0/1444597379 learned_addr learned my addr 172.21.15.66:0/1444597379 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d523a2640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4c102a30 0x7f5d4c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d51ba1640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d4c105360 0x7f5d4c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 -- 172.21.15.66:0/1444597379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4c102a30 msgr2=0x7f5d4c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4c102a30 0x7f5d4c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 -- 172.21.15.66:0/1444597379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d4c105360 msgr2=0x7f5d4c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d4c105360 0x7f5d4c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.257+0000 7f5d52ba3640 1 -- 172.21.15.66:0/1444597379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c19fe40 con 0x7f5d4c107c90 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.258+0000 7f5d52ba3640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c107c90 0x7f5d4c19f7d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400eaa0 tx=0x7f5d4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:07.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.258+0000 7f5d437fe640 1 -- 172.21.15.66:0/1444597379 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400cdb0 con 0x7f5d4c107c90 2024-04-21T09:05:07.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.258+0000 7f5d437fe640 1 -- 172.21.15.66:0/1444597379 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d44005020 con 0x7f5d4c107c90 2024-04-21T09:05:07.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.258+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c1a0130 con 0x7f5d4c107c90 2024-04-21T09:05:07.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.258+0000 7f5d437fe640 1 -- 172.21.15.66:0/1444597379 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d440116d0 con 0x7f5d4c107c90 2024-04-21T09:05:07.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.258+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1a0680 con 0x7f5d4c107c90 2024-04-21T09:05:07.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.259+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c0fdf30 con 0x7f5d4c107c90 2024-04-21T09:05:07.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.260+0000 7f5d437fe640 1 -- 172.21.15.66:0/1444597379 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d440051d0 con 0x7f5d4c107c90 2024-04-21T09:05:07.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.260+0000 7f5d437fe640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d380a3bb0 0x7f5d380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.260+0000 7f5d523a2640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d380a3bb0 0x7f5d380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.261+0000 7f5d437fe640 1 -- 172.21.15.66:0/1444597379 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5d440d0090 con 0x7f5d4c107c90 2024-04-21T09:05:07.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.261+0000 7f5d523a2640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d380a3bb0 0x7f5d380a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009230 tx=0x7f5d3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:07.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.265+0000 7f5d437fe640 1 -- 172.21.15.66:0/1444597379 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d44016030 con 0x7f5d4c107c90 2024-04-21T09:05:07.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.449+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d4c062b50 con 0x7f5d4c107c90 2024-04-21T09:05:07.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.449+0000 7f5d437fe640 1 -- 172.21.15.66:0/1444597379 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f5d4409cf10 con 0x7f5d4c107c90 2024-04-21T09:05:07.449 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d380a3bb0 msgr2=0x7f5d380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d380a3bb0 0x7f5d380a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009230 tx=0x7f5d3c005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c107c90 msgr2=0x7f5d4c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c107c90 0x7f5d4c19f7d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400eaa0 tx=0x7f5d4400ef60 comp rx=0 tx=0).stop 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 shutdown_connections 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f5d380a3bb0 0x7f5d380a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4c107c90 0x7f5d4c19f7d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d4c105360 0x7f5d4c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 --2- 172.21.15.66:0/1444597379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d4c102a30 0x7f5d4c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 >> 172.21.15.66:0/1444597379 conn(0x7f5d4c0fc440 msgr2=0x7f5d4c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:07.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 shutdown_connections 2024-04-21T09:05:07.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.451+0000 7f5d54606640 1 -- 172.21.15.66:0/1444597379 wait complete. 2024-04-21T09:05:07.458 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:05:07.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-21T09:05:07.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:07.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:07.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:07.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:07.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:07.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:07 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:07.461 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:07 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:07.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- 172.21.15.66:0/1280398813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8100e70 msgr2=0x7ff7f8101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/1280398813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8100e70 0x7ff7f8101290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009900 tx=0x7ff7f402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- 172.21.15.66:0/1280398813 shutdown_connections 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/1280398813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f8102980 0x7ff7f81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/1280398813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f8101fc0 0x7ff7f8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/1280398813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8100e70 0x7ff7f8101290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- 172.21.15.66:0/1280398813 >> 172.21.15.66:0/1280398813 conn(0x7ff7f80fc3e0 msgr2=0x7ff7f80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- 172.21.15.66:0/1280398813 shutdown_connections 2024-04-21T09:05:07.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- 172.21.15.66:0/1280398813 wait complete. 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 Processor -- start 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- start start 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f8100e70 0x7ff7f8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8101fc0 0x7ff7f8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f8102980 0x7ff7f819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f810c9c0 con 0x7ff7f8101fc0 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f810c840 con 0x7ff7f8100e70 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.565+0000 7ff7ff94a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f810cb40 con 0x7ff7f8102980 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8101fc0 0x7ff7f8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8101fc0 0x7ff7f8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54060/0 (socket says 172.21.15.66:54060) 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 -- 172.21.15.66:0/2317027226 learned_addr learned my addr 172.21.15.66:0/2317027226 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fdee7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f8102980 0x7ff7f819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fd6e6640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f8100e70 0x7ff7f8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 -- 172.21.15.66:0/2317027226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f8102980 msgr2=0x7ff7f819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f8102980 0x7ff7f819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 -- 172.21.15.66:0/2317027226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f8100e70 msgr2=0x7ff7f8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f8100e70 0x7ff7f8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 -- 172.21.15.66:0/2317027226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f819fe30 con 0x7ff7f8101fc0 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fd6e6640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f8100e70 0x7ff7f8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7fcee5640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8101fc0 0x7ff7f8198750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9c0 tx=0x7ff7ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7ea7fc640 1 -- 172.21.15.66:0/2317027226 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00cd80 con 0x7ff7f8101fc0 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7ea7fc640 1 -- 172.21.15.66:0/2317027226 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ec0035f0 con 0x7ff7f8101fc0 2024-04-21T09:05:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7ea7fc640 1 -- 172.21.15.66:0/2317027226 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec010690 con 0x7ff7f8101fc0 2024-04-21T09:05:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81a0120 con 0x7ff7f8101fc0 2024-04-21T09:05:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.566+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81a0670 con 0x7ff7f8101fc0 2024-04-21T09:05:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.567+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f81a0930 con 0x7ff7f8101fc0 2024-04-21T09:05:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.568+0000 7ff7ea7fc640 1 -- 172.21.15.66:0/2317027226 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff7ec004000 con 0x7ff7f8101fc0 2024-04-21T09:05:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.569+0000 7ff7ea7fc640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff7e40a3c30 0x7ff7e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.569+0000 7ff7fd6e6640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff7e40a3c30 0x7ff7e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.570+0000 7ff7ea7fc640 1 -- 172.21.15.66:0/2317027226 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff7ec0d0640 con 0x7ff7f8101fc0 2024-04-21T09:05:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.570+0000 7ff7fd6e6640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff7e40a3c30 0x7ff7e40a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40096c0 tx=0x7ff7f4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:07.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.573+0000 7ff7ea7fc640 1 -- 172.21.15.66:0/2317027226 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7ec09d4c0 con 0x7ff7f8101fc0 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:07 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:07.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.797+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f81a0b00 con 0x7ff7f8101fc0 2024-04-21T09:05:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.797+0000 7ff7ea7fc640 1 -- 172.21.15.66:0/2317027226 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7ff7ec09cdc0 con 0x7ff7f8101fc0 2024-04-21T09:05:07.797 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:07.797 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-21T09:05:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 1 2024-04-21T09:05:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.799+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff7e40a3c30 msgr2=0x7ff7e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.799+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff7e40a3c30 0x7ff7e40a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40096c0 tx=0x7ff7f4000bc0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.799+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8101fc0 msgr2=0x7ff7f8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.799+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8101fc0 0x7ff7f8198750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9c0 tx=0x7ff7ec00ee80 comp rx=0 tx=0).stop 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 shutdown_connections 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff7e40a3c30 0x7ff7e40a60f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f8102980 0x7ff7f819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f8101fc0 0x7ff7f8198750 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 --2- 172.21.15.66:0/2317027226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f8100e70 0x7ff7f8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 >> 172.21.15.66:0/2317027226 conn(0x7ff7f80fc3e0 msgr2=0x7ff7f80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 shutdown_connections 2024-04-21T09:05:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:07.800+0000 7ff7ff94a640 1 -- 172.21.15.66:0/2317027226 wait complete. 2024-04-21T09:05:08.424 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:08 smithi186 ceph-mon[31505]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:08.424 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:08 smithi186 ceph-mon[31505]: Deploying daemon mds.l on smithi186 2024-04-21T09:05:08.424 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:08 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2341543273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:08.424 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:08 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1444597379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:08.424 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:08 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2317027226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:08.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:08 smithi083 ceph-mon[31413]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:08.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:08 smithi083 ceph-mon[31413]: Deploying daemon mds.l on smithi186 2024-04-21T09:05:08.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:08 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2341543273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:08.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:08 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1444597379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:08.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:08 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2317027226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:08.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:08 smithi066 ceph-mon[27953]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:08.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:08 smithi066 ceph-mon[27953]: Deploying daemon mds.l on smithi186 2024-04-21T09:05:08.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:08 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2341543273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:08.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:08 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1444597379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:08.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:08 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2317027226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:09.088 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:05:09 smithi186 podman[62153]: 2024-04-21 09:05:09.036963526 +0000 UTC m=+0.137932713 container create 1bfc3d587b3c79e1fd1668441ca808f10464584b1070f5c9e4a495d0f5f6194f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:05:09.588 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:05:09 smithi186 podman[62153]: 2024-04-21 09:05:09.281153798 +0000 UTC m=+0.382122985 container init 1bfc3d587b3c79e1fd1668441ca808f10464584b1070f5c9e4a495d0f5f6194f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-21T09:05:09.589 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:05:09 smithi186 podman[62153]: 2024-04-21 09:05:09.283925728 +0000 UTC m=+0.384894924 container start 1bfc3d587b3c79e1fd1668441ca808f10464584b1070f5c9e4a495d0f5f6194f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-21T09:05:09.589 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:05:09 smithi186 ceph-mds[62172]: -- 172.21.15.186:0/3811496780 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625dfaafa40 con 0x5625dfa82800 2024-04-21T09:05:09.589 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:05:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l[62168]: starting mds.l at 2024-04-21T09:05:09.589 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:05:09 smithi186 bash[62153]: 1bfc3d587b3c79e1fd1668441ca808f10464584b1070f5c9e4a495d0f5f6194f 2024-04-21T09:05:09.589 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:05:09 smithi186 systemd[1]: Started Ceph mds.l for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:09.811 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 -- 172.21.15.66:0/3078263712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102a10 msgr2=0x7f5508109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 --2- 172.21.15.66:0/3078263712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102a10 0x7f5508109330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009900 tx=0x7f54f802bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 -- 172.21.15.66:0/3078263712 shutdown_connections 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 --2- 172.21.15.66:0/3078263712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102a10 0x7f5508109330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 --2- 172.21.15.66:0/3078263712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102050 0x7f55081024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 --2- 172.21.15.66:0/3078263712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508100e70 0x7f5508101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 -- 172.21.15.66:0/3078263712 >> 172.21.15.66:0/3078263712 conn(0x7f55080fc470 msgr2=0x7f55080fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 -- 172.21.15.66:0/3078263712 shutdown_connections 2024-04-21T09:05:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.925+0000 7f550cb8f640 1 -- 172.21.15.66:0/3078263712 wait complete. 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 Processor -- start 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 -- start start 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508100e70 0x7f550806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102050 0x7f550819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102a10 0x7f550819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f550810ca20 con 0x7f5508102050 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f550810c8a0 con 0x7f5508102a10 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550cb8f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f550810cba0 con 0x7f5508100e70 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f5505d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102050 0x7f550819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f5506d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102a10 0x7f550819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f5505d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102050 0x7f550819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54070/0 (socket says 172.21.15.66:54070) 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f5505d9b640 1 -- 172.21.15.66:0/298497305 learned_addr learned my addr 172.21.15.66:0/298497305 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f5506d9d640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102a10 0x7f550819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:56692/0 (socket says 172.21.15.66:56692) 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.926+0000 7f550659c640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508100e70 0x7f550806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f5506d9d640 1 -- 172.21.15.66:0/298497305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508100e70 msgr2=0x7f550806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f5506d9d640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508100e70 0x7f550806b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f5506d9d640 1 -- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102050 msgr2=0x7f550819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f5506d9d640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102050 0x7f550819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f5506d9d640 1 -- 172.21.15.66:0/298497305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550819fd70 con 0x7f5508102a10 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f5505d9b640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102050 0x7f550819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f5506d9d640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102a10 0x7f550819f4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009230 tx=0x7f54f8005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f54f77fe640 1 -- 172.21.15.66:0/298497305 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8043020 con 0x7f5508102a10 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f54f77fe640 1 -- 172.21.15.66:0/298497305 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54f80043d0 con 0x7f5508102a10 2024-04-21T09:05:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55081a0000 con 0x7f5508102a10 2024-04-21T09:05:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f54f77fe640 1 -- 172.21.15.66:0/298497305 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8003aa0 con 0x7f5508102a10 2024-04-21T09:05:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.927+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081a0550 con 0x7f5508102a10 2024-04-21T09:05:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.928+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5508108210 con 0x7f5508102a10 2024-04-21T09:05:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.929+0000 7f54f77fe640 1 -- 172.21.15.66:0/298497305 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f54f8046020 con 0x7f5508102a10 2024-04-21T09:05:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.930+0000 7f54f77fe640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54f00a3bb0 0x7f54f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.930+0000 7f550659c640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54f00a3bb0 0x7f54f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.930+0000 7f54f77fe640 1 -- 172.21.15.66:0/298497305 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f54f80f2810 con 0x7f5508102a10 2024-04-21T09:05:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.930+0000 7f550659c640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54f00a3bb0 0x7f54f00a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5508101bc0 tx=0x7f5500009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:09.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:09.933+0000 7f54f77fe640 1 -- 172.21.15.66:0/298497305 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54f804d020 con 0x7f5508102a10 2024-04-21T09:05:10.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.161+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f550802d6e0 con 0x7f5508102a10 2024-04-21T09:05:10.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.162+0000 7f54f77fe640 1 -- 172.21.15.66:0/298497305 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f54f80bf690 con 0x7f5508102a10 2024-04-21T09:05:10.161 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:10.162 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T09:05:10.162 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2024-04-21T09:05:10.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54f00a3bb0 msgr2=0x7f54f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:10.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54f00a3bb0 0x7f54f00a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5508101bc0 tx=0x7f5500009500 comp rx=0 tx=0).stop 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102a10 msgr2=0x7f550819f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102a10 0x7f550819f4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009230 tx=0x7f54f8005e40 comp rx=0 tx=0).stop 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 shutdown_connections 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f54f00a3bb0 0x7f54f00a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5508102a10 0x7f550819f4b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5508102050 0x7f550819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 --2- 172.21.15.66:0/298497305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508100e70 0x7f550806b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 >> 172.21.15.66:0/298497305 conn(0x7f55080fc470 msgr2=0x7f55081053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 shutdown_connections 2024-04-21T09:05:10.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:10.164+0000 7f550cb8f640 1 -- 172.21.15.66:0/298497305 wait complete. 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6832/446664633,v1:172.21.15.186:6833/446664633] up:boot 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: fsmap 1 up:standby 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:10 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6832/446664633,v1:172.21.15.186:6833/446664633] up:boot 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: fsmap 1 up:standby 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:10.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:10 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6832/446664633,v1:172.21.15.186:6833/446664633] up:boot 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: fsmap 1 up:standby 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:10.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:10 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:11.388 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:11 smithi083 ceph-mon[31413]: Deploying daemon mds.k on smithi083 2024-04-21T09:05:11.388 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/298497305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:11.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:11 smithi186 ceph-mon[31505]: Deploying daemon mds.k on smithi083 2024-04-21T09:05:11.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/298497305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:11.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:11 smithi066 ceph-mon[27953]: Deploying daemon mds.k on smithi083 2024-04-21T09:05:11.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/298497305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:11.741 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:11 smithi083 systemd[1]: Starting Ceph mds.k for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:12.077 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:11 smithi083 podman[61505]: 2024-04-21 09:05:11.862126582 +0000 UTC m=+0.017868795 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:12.077 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:11 smithi083 podman[61505]: 2024-04-21 09:05:11.96552165 +0000 UTC m=+0.121263899 container create 7c2f6bee9ced4395a5e13b9039b90f6ad587d2dbb52ba6cff5022bcbd45488d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-21T09:05:12.175 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2488449887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44100830 msgr2=0x7fef44100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2488449887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44100830 0x7fef44100c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef40009900 tx=0x7fef4002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2488449887 shutdown_connections 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2488449887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef44101b30 0x7fef44101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2488449887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44101190 0x7fef441015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2488449887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44100830 0x7fef44100c50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2488449887 >> 172.21.15.66:0/2488449887 conn(0x7fef440fc470 msgr2=0x7fef440fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:12.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2488449887 shutdown_connections 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2488449887 wait complete. 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 Processor -- start 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 -- start start 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.290+0000 7fef4bbe6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44100830 0x7fef4419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4bbe6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44101190 0x7fef4419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4bbe6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef44101b30 0x7fef441a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4bbe6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410fd10 con 0x7fef44101190 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4bbe6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410fb90 con 0x7fef44100830 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4bbe6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410fe90 con 0x7fef44101b30 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44101190 0x7fef4419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:12.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44101190 0x7fef4419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54086/0 (socket says 172.21.15.66:54086) 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 -- 172.21.15.66:0/2160892557 learned_addr learned my addr 172.21.15.66:0/2160892557 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49982640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44100830 0x7fef4419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4a183640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef44101b30 0x7fef441a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 -- 172.21.15.66:0/2160892557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef44101b30 msgr2=0x7fef441a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef44101b30 0x7fef441a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 -- 172.21.15.66:0/2160892557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44100830 msgr2=0x7fef4419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44100830 0x7fef4419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 -- 172.21.15.66:0/2160892557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef441a1fa0 con 0x7fef44101190 2024-04-21T09:05:12.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef49181640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44101190 0x7fef4419a8c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9f0 tx=0x7fef3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:12.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef3affd640 1 -- 172.21.15.66:0/2160892557 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3400cd80 con 0x7fef44101190 2024-04-21T09:05:12.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef3affd640 1 -- 172.21.15.66:0/2160892557 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef340108e0 con 0x7fef44101190 2024-04-21T09:05:12.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef3affd640 1 -- 172.21.15.66:0/2160892557 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34018980 con 0x7fef44101190 2024-04-21T09:05:12.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef441a2290 con 0x7fef44101190 2024-04-21T09:05:12.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.291+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef441a2830 con 0x7fef44101190 2024-04-21T09:05:12.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.293+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4410b700 con 0x7fef44101190 2024-04-21T09:05:12.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.294+0000 7fef3affd640 1 -- 172.21.15.66:0/2160892557 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef34010430 con 0x7fef44101190 2024-04-21T09:05:12.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.294+0000 7fef3affd640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fef300a3c30 0x7fef300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:12.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.294+0000 7fef49982640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fef300a3c30 0x7fef300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:12.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.294+0000 7fef3affd640 1 -- 172.21.15.66:0/2160892557 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fef340d02d0 con 0x7fef44101190 2024-04-21T09:05:12.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.295+0000 7fef49982640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fef300a3c30 0x7fef300a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef40009230 tx=0x7fef40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:12.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.299+0000 7fef3affd640 1 -- 172.21.15.66:0/2160892557 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef3409d150 con 0x7fef44101190 2024-04-21T09:05:12.342 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:12 smithi083 ceph-mon[31413]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:12.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.527+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef4402d810 con 0x7fef44101190 2024-04-21T09:05:12.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.528+0000 7fef3affd640 1 -- 172.21.15.66:0/2160892557 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fef3409ca50 con 0x7fef44101190 2024-04-21T09:05:12.529 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:12.529 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T09:05:12.530 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2024-04-21T09:05:12.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fef300a3c30 msgr2=0x7fef300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:12.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fef300a3c30 0x7fef300a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef40009230 tx=0x7fef40005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44101190 msgr2=0x7fef4419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44101190 0x7fef4419a8c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9f0 tx=0x7fef3400eeb0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 shutdown_connections 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fef300a3c30 0x7fef300a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef44101b30 0x7fef441a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef44101190 0x7fef4419a8c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 --2- 172.21.15.66:0/2160892557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44100830 0x7fef4419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 >> 172.21.15.66:0/2160892557 conn(0x7fef440fc470 msgr2=0x7fef440fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 shutdown_connections 2024-04-21T09:05:12.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:12.530+0000 7fef4bbe6640 1 -- 172.21.15.66:0/2160892557 wait complete. 2024-04-21T09:05:12.543 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:12 smithi066 ceph-mon[27953]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:12.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:12 smithi186 ceph-mon[31505]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:12.611 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:12 smithi083 podman[61505]: 2024-04-21 09:05:12.343563604 +0000 UTC m=+0.499305807 container init 7c2f6bee9ced4395a5e13b9039b90f6ad587d2dbb52ba6cff5022bcbd45488d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-21T09:05:12.611 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:12 smithi083 podman[61505]: 2024-04-21 09:05:12.346298416 +0000 UTC m=+0.502040620 container start 7c2f6bee9ced4395a5e13b9039b90f6ad587d2dbb52ba6cff5022bcbd45488d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-21T09:05:12.611 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:12 smithi083 ceph-mds[61524]: -- 172.21.15.83:0/424297105 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b05c929a40 con 0x55b05c8fd000 2024-04-21T09:05:12.611 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:12 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k[61520]: starting mds.k at 2024-04-21T09:05:12.612 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:12 smithi083 bash[61505]: 7c2f6bee9ced4395a5e13b9039b90f6ad587d2dbb52ba6cff5022bcbd45488d4 2024-04-21T09:05:12.612 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:05:12 smithi083 systemd[1]: Started Ceph mds.k for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2160892557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: Deploying daemon mds.j on smithi066 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6832/325280073,v1:172.21.15.83:6833/325280073] up:boot 2024-04-21T09:05:13.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: fsmap 2 up:standby 2024-04-21T09:05:13.764 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:13 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2160892557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: Deploying daemon mds.j on smithi066 2024-04-21T09:05:13.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6832/325280073,v1:172.21.15.83:6833/325280073] up:boot 2024-04-21T09:05:13.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: fsmap 2 up:standby 2024-04-21T09:05:13.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:13 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2160892557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: Deploying daemon mds.j on smithi066 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6832/325280073,v1:172.21.15.83:6833/325280073] up:boot 2024-04-21T09:05:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: fsmap 2 up:standby 2024-04-21T09:05:13.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:13 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T09:05:14.545 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:14.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 -- 172.21.15.66:0/782128541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 msgr2=0x7f55981092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:14.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 --2- 172.21.15.66:0/782128541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f55981092a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5590009ca0 tx=0x7f559002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:14.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 -- 172.21.15.66:0/782128541 shutdown_connections 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 --2- 172.21.15.66:0/782128541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f55981092a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 --2- 172.21.15.66:0/782128541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5598101fc0 0x7f5598102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 --2- 172.21.15.66:0/782128541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5598100e70 0x7f5598101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 -- 172.21.15.66:0/782128541 >> 172.21.15.66:0/782128541 conn(0x7f55980fc3e0 msgr2=0x7f55980fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 -- 172.21.15.66:0/782128541 shutdown_connections 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.791+0000 7f559f511640 1 -- 172.21.15.66:0/782128541 wait complete. 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 Processor -- start 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 -- start start 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5598100e70 0x7f5598198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5598101fc0 0x7f55981986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:14.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f559819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f559810c9c0 con 0x7f5598102980 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f559810c840 con 0x7f5598101fc0 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559f511640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f559810cb40 con 0x7f5598100e70 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f559819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f559819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43102/0 (socket says 172.21.15.66:43102) 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 -- 172.21.15.66:0/1474471559 learned_addr learned my addr 172.21.15.66:0/1474471559 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559caac640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5598101fc0 0x7f55981986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559d2ad640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5598100e70 0x7f5598198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 -- 172.21.15.66:0/1474471559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5598100e70 msgr2=0x7f5598198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5598100e70 0x7f5598198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 -- 172.21.15.66:0/1474471559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5598101fc0 msgr2=0x7f55981986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5598101fc0 0x7f55981986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:14.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 -- 172.21.15.66:0/1474471559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559819fd80 con 0x7f5598102980 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559d2ad640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5598100e70 0x7f5598198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.792+0000 7f559daae640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f559819f710 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f559002c700 tx=0x7f5590034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.793+0000 7f558e7fc640 1 -- 172.21.15.66:0/1474471559 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590039040 con 0x7f5598102980 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.793+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55981a0010 con 0x7f5598102980 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.793+0000 7f558e7fc640 1 -- 172.21.15.66:0/1474471559 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590044070 con 0x7f5598102980 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.793+0000 7f558e7fc640 1 -- 172.21.15.66:0/1474471559 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900042a0 con 0x7f5598102980 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.793+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a0500 con 0x7f5598102980 2024-04-21T09:05:14.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.794+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55981a0760 con 0x7f5598102980 2024-04-21T09:05:14.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.795+0000 7f558e7fc640 1 -- 172.21.15.66:0/1474471559 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5590003970 con 0x7f5598102980 2024-04-21T09:05:14.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.796+0000 7f558e7fc640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f55840a3bb0 0x7f55840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:14.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.796+0000 7f559d2ad640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f55840a3bb0 0x7f55840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:14.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.796+0000 7f558e7fc640 1 -- 172.21.15.66:0/1474471559 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f55900f2340 con 0x7f5598102980 2024-04-21T09:05:14.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.796+0000 7f559d2ad640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f55840a3bb0 0x7f55840a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55940058c0 tx=0x7f5594005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:14.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:14.800+0000 7f558e7fc640 1 -- 172.21.15.66:0/1474471559 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55900bf1c0 con 0x7f5598102980 2024-04-21T09:05:15.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.034+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55981a0a50 con 0x7f5598102980 2024-04-21T09:05:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.034+0000 7f558e7fc640 1 -- 172.21.15.66:0/1474471559 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f55900beac0 con 0x7f5598102980 2024-04-21T09:05:15.035 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:15.035 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T09:05:15.036 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 3 2024-04-21T09:05:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f55840a3bb0 msgr2=0x7f55840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f55840a3bb0 0x7f55840a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55940058c0 tx=0x7f5594005850 comp rx=0 tx=0).stop 2024-04-21T09:05:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 msgr2=0x7f559819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f559819f710 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f559002c700 tx=0x7f5590034760 comp rx=0 tx=0).stop 2024-04-21T09:05:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 shutdown_connections 2024-04-21T09:05:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f55840a3bb0 0x7f55840a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5598102980 0x7f559819f710 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5598101fc0 0x7f55981986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.036+0000 7f559f511640 1 --2- 172.21.15.66:0/1474471559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5598100e70 0x7f5598198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.037+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 >> 172.21.15.66:0/1474471559 conn(0x7f55980fc3e0 msgr2=0x7f55980fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.037+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 shutdown_connections 2024-04-21T09:05:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:15.037+0000 7f559f511640 1 -- 172.21.15.66:0/1474471559 wait complete. 2024-04-21T09:05:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:15 smithi186 ceph-mon[31505]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:15 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1474471559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:15.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:15 smithi083 ceph-mon[31413]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:15.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:15 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1474471559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:15 smithi066 ceph-mon[27953]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:15 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1474471559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.66:6834/3512926718,v1:172.21.15.66:6835/3512926718] up:boot 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: fsmap 3 up:standby 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:16 smithi186 ceph-mon[31505]: Deploying daemon mds.d on smithi066 2024-04-21T09:05:16.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.66:6834/3512926718,v1:172.21.15.66:6835/3512926718] up:boot 2024-04-21T09:05:16.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: fsmap 3 up:standby 2024-04-21T09:05:16.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T09:05:16.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:16.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:16.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:16.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:16 smithi083 ceph-mon[31413]: Deploying daemon mds.d on smithi066 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.66:6834/3512926718,v1:172.21.15.66:6835/3512926718] up:boot 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: fsmap 3 up:standby 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:16 smithi066 ceph-mon[27953]: Deploying daemon mds.d on smithi066 2024-04-21T09:05:17.050 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:17.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 -- 172.21.15.66:0/2580966374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c102a00 msgr2=0x7fcd3c104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/2580966374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c102a00 0x7fcd3c104df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009900 tx=0x7fcd2c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 -- 172.21.15.66:0/2580966374 shutdown_connections 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/2580966374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c107c60 0x7fcd3c10a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/2580966374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c105330 0x7fcd3c107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/2580966374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c102a00 0x7fcd3c104df0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 -- 172.21.15.66:0/2580966374 >> 172.21.15.66:0/2580966374 conn(0x7fcd3c0fc410 msgr2=0x7fcd3c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.158+0000 7fcd40fb2640 1 -- 172.21.15.66:0/2580966374 shutdown_connections 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 -- 172.21.15.66:0/2580966374 wait complete. 2024-04-21T09:05:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 Processor -- start 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 -- start start 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c102a00 0x7fcd3c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c105330 0x7fcd3c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c107c60 0x7fcd3c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c10dab0 con 0x7fcd3c105330 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c10d930 con 0x7fcd3c107c60 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd40fb2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c10dc30 con 0x7fcd3c102a00 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.159+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c105330 0x7fcd3c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c105330 0x7fcd3c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43120/0 (socket says 172.21.15.66:43120) 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 -- 172.21.15.66:0/620962262 learned_addr learned my addr 172.21.15.66:0/620962262 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd3a59c640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c102a00 0x7fcd3c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd3ad9d640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c107c60 0x7fcd3c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 -- 172.21.15.66:0/620962262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c102a00 msgr2=0x7fcd3c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c102a00 0x7fcd3c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 -- 172.21.15.66:0/620962262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c107c60 msgr2=0x7fcd3c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c107c60 0x7fcd3c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 -- 172.21.15.66:0/620962262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c19fdd0 con 0x7fcd3c105330 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd33fff640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c105330 0x7fcd3c1986f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400d770 tx=0x7fcd2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd337fe640 1 -- 172.21.15.66:0/620962262 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24017070 con 0x7fcd3c105330 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1a00c0 con 0x7fcd3c105330 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd337fe640 1 -- 172.21.15.66:0/620962262 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd24009c30 con 0x7fcd3c105330 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd337fe640 1 -- 172.21.15.66:0/620962262 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400f620 con 0x7fcd3c105330 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.160+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a0660 con 0x7fcd3c105330 2024-04-21T09:05:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.161+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c0fdf00 con 0x7fcd3c105330 2024-04-21T09:05:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.162+0000 7fcd337fe640 1 -- 172.21.15.66:0/620962262 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcd2401a070 con 0x7fcd3c105330 2024-04-21T09:05:17.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.163+0000 7fcd337fe640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcd280a3bb0 0x7fcd280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.163+0000 7fcd3a59c640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcd280a3bb0 0x7fcd280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.163+0000 7fcd337fe640 1 -- 172.21.15.66:0/620962262 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcd240cfaa0 con 0x7fcd3c105330 2024-04-21T09:05:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.163+0000 7fcd3a59c640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcd280a3bb0 0x7fcd280a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009230 tx=0x7fcd2c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:17.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.167+0000 7fcd337fe640 1 -- 172.21.15.66:0/620962262 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd24009de0 con 0x7fcd3c105330 2024-04-21T09:05:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.398+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3c02d770 con 0x7fcd3c105330 2024-04-21T09:05:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.399+0000 7fcd337fe640 1 -- 172.21.15.66:0/620962262 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7fcd2409c920 con 0x7fcd3c105330 2024-04-21T09:05:17.400 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:17.400 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T09:05:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 4 2024-04-21T09:05:17.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcd280a3bb0 msgr2=0x7fcd280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcd280a3bb0 0x7fcd280a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009230 tx=0x7fcd2c005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c105330 msgr2=0x7fcd3c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c105330 0x7fcd3c1986f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400d770 tx=0x7fcd2400dc30 comp rx=0 tx=0).stop 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 shutdown_connections 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcd280a3bb0 0x7fcd280a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c107c60 0x7fcd3c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd3c105330 0x7fcd3c1986f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 --2- 172.21.15.66:0/620962262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c102a00 0x7fcd3c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 >> 172.21.15.66:0/620962262 conn(0x7fcd3c0fc410 msgr2=0x7fcd3c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 shutdown_connections 2024-04-21T09:05:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:17.402+0000 7fcd40fb2640 1 -- 172.21.15.66:0/620962262 wait complete. 2024-04-21T09:05:17.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:17 smithi186 ceph-mon[31505]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/620962262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:17 smithi083 ceph-mon[31413]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/620962262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:17 smithi066 ceph-mon[27953]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:17.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/620962262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:17.870 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:17 smithi066 systemd[1]: Starting Ceph mds.d for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:18.252 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:17 smithi066 podman[66235]: 2024-04-21 09:05:17.895219831 +0000 UTC m=+0.018176210 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:18.252 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:18 smithi066 podman[66235]: 2024-04-21 09:05:18.00878247 +0000 UTC m=+0.131738853 container create 5e59e9398166770e91e8f8177eb2828e522c633cf6408077cdc49f1d8bcace32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-21T09:05:18.252 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:18 smithi066 podman[66235]: 2024-04-21 09:05:18.253306481 +0000 UTC m=+0.376262859 container init 5e59e9398166770e91e8f8177eb2828e522c633cf6408077cdc49f1d8bcace32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-21T09:05:18.620 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:18 smithi066 podman[66235]: 2024-04-21 09:05:18.256011005 +0000 UTC m=+0.378967384 container start 5e59e9398166770e91e8f8177eb2828e522c633cf6408077cdc49f1d8bcace32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T09:05:18.620 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:18 smithi066 ceph-mds[66255]: -- 172.21.15.66:0/3633125052 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56126c011a40 con 0x56126bfe4c00 2024-04-21T09:05:18.620 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:18 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d[66251]: starting mds.d at 2024-04-21T09:05:18.620 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:18 smithi066 bash[66235]: 5e59e9398166770e91e8f8177eb2828e522c633cf6408077cdc49f1d8bcace32 2024-04-21T09:05:18.620 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:05:18 smithi066 systemd[1]: Started Ceph mds.d for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:19.417 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:19.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 -- 172.21.15.66:0/3911443397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84101ff0 msgr2=0x7fda84102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:19.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/3911443397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84101ff0 0x7fda84102470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda7402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:19.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 -- 172.21.15.66:0/3911443397 shutdown_connections 2024-04-21T09:05:19.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/3911443397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda841029b0 0x7fda841092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/3911443397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84101ff0 0x7fda84102470 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/3911443397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda84100ea0 0x7fda841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 -- 172.21.15.66:0/3911443397 >> 172.21.15.66:0/3911443397 conn(0x7fda840fc410 msgr2=0x7fda840fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 -- 172.21.15.66:0/3911443397 shutdown_connections 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 -- 172.21.15.66:0/3911443397 wait complete. 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 Processor -- start 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.534+0000 7fda8a9b6640 1 -- start start 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda8a9b6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84100ea0 0x7fda8410b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda8a9b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda84101ff0 0x7fda84109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda8a9b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda841029b0 0x7fda8410a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda8a9b6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda8410db10 con 0x7fda84100ea0 2024-04-21T09:05:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda8a9b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda8410d990 con 0x7fda841029b0 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda8a9b6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fda8410dc90 con 0x7fda84101ff0 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84100ea0 0x7fda8410b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84100ea0 0x7fda8410b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43164/0 (socket says 172.21.15.66:43164) 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 -- 172.21.15.66:0/2085215862 learned_addr learned my addr 172.21.15.66:0/2085215862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda837fe640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda84101ff0 0x7fda84109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 -- 172.21.15.66:0/2085215862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda84101ff0 msgr2=0x7fda84109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda84101ff0 0x7fda84109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda88f53640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda841029b0 0x7fda8410a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 -- 172.21.15.66:0/2085215862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda841029b0 msgr2=0x7fda8410a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda841029b0 0x7fda8410a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 -- 172.21.15.66:0/2085215862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8410aaa0 con 0x7fda84100ea0 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda88f53640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda841029b0 0x7fda8410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.535+0000 7fda83fff640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84100ea0 0x7fda8410b690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e9f0 tx=0x7fda7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:19.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.536+0000 7fda817fa640 1 -- 172.21.15.66:0/2085215862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800cd80 con 0x7fda84100ea0 2024-04-21T09:05:19.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.536+0000 7fda817fa640 1 -- 172.21.15.66:0/2085215862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda780108e0 con 0x7fda84100ea0 2024-04-21T09:05:19.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.536+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda841a22c0 con 0x7fda84100ea0 2024-04-21T09:05:19.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.536+0000 7fda817fa640 1 -- 172.21.15.66:0/2085215862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78018980 con 0x7fda84100ea0 2024-04-21T09:05:19.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.536+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda841a2830 con 0x7fda84100ea0 2024-04-21T09:05:19.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.537+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda841081b0 con 0x7fda84100ea0 2024-04-21T09:05:19.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.538+0000 7fda817fa640 1 -- 172.21.15.66:0/2085215862 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fda78010430 con 0x7fda84100ea0 2024-04-21T09:05:19.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.538+0000 7fda817fa640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:19.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.539+0000 7fda837fe640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:19.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.539+0000 7fda817fa640 1 -- 172.21.15.66:0/2085215862 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fda780d0330 con 0x7fda84100ea0 2024-04-21T09:05:19.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.539+0000 7fda837fe640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda74006010 tx=0x7fda74005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:19.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.543+0000 7fda817fa640 1 -- 172.21.15.66:0/2085215862 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda7809d1b0 con 0x7fda84100ea0 2024-04-21T09:05:19.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:19.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:19.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:19.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:19.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: Deploying daemon mds.e on smithi083 2024-04-21T09:05:19.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.66:6836/2533275448,v1:172.21.15.66:6837/2533275448] up:boot 2024-04-21T09:05:19.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: fsmap 4 up:standby 2024-04-21T09:05:19.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:19 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T09:05:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.774+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda8402d5c0 con 0x7fda84100ea0 2024-04-21T09:05:19.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.774+0000 7fda817fa640 1 -- 172.21.15.66:0/2085215862 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fda7809cab0 con 0x7fda84100ea0 2024-04-21T09:05:19.775 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:19.775 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T09:05:19.775 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 5 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fda6c0a3c30 msgr2=0x7fda6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda74006010 tx=0x7fda74005be0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84100ea0 msgr2=0x7fda8410b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84100ea0 0x7fda8410b690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e9f0 tx=0x7fda7800eeb0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 shutdown_connections 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda841029b0 0x7fda8410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda84101ff0 0x7fda84109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 --2- 172.21.15.66:0/2085215862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda84100ea0 0x7fda8410b690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 >> 172.21.15.66:0/2085215862 conn(0x7fda840fc410 msgr2=0x7fda84105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:19.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 shutdown_connections 2024-04-21T09:05:19.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:19.776+0000 7fda8a9b6640 1 -- 172.21.15.66:0/2085215862 wait complete. 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: Deploying daemon mds.e on smithi083 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.66:6836/2533275448,v1:172.21.15.66:6837/2533275448] up:boot 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: fsmap 4 up:standby 2024-04-21T09:05:19.787 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:19 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T09:05:19.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:19.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:19.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:19.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:19.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:19.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: Deploying daemon mds.e on smithi083 2024-04-21T09:05:19.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.66:6836/2533275448,v1:172.21.15.66:6837/2533275448] up:boot 2024-04-21T09:05:19.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: fsmap 4 up:standby 2024-04-21T09:05:19.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:19 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T09:05:20.760 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:20 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2085215862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:20.760 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:20 smithi083 systemd[1]: Starting Ceph mds.e for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:20.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:20 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2085215862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:20.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:20 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2085215862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:21.024 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:20 smithi083 podman[61821]: 2024-04-21 09:05:20.663724667 +0000 UTC m=+0.018005313 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:21.024 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:20 smithi083 podman[61821]: 2024-04-21 09:05:20.780541355 +0000 UTC m=+0.134822029 container create e42708ebc0bf8876a777daf907232ee4ed118244915ec2b44391652f296e58e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-21T09:05:21.360 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:21 smithi083 podman[61821]: 2024-04-21 09:05:21.025113901 +0000 UTC m=+0.379394534 container init e42708ebc0bf8876a777daf907232ee4ed118244915ec2b44391652f296e58e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T09:05:21.360 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:21 smithi083 podman[61821]: 2024-04-21 09:05:21.02763494 +0000 UTC m=+0.381915573 container start e42708ebc0bf8876a777daf907232ee4ed118244915ec2b44391652f296e58e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-21T09:05:21.360 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mds[61840]: -- 172.21.15.83:0/3438601921 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562332665a40 con 0x562332638800 2024-04-21T09:05:21.360 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e[61836]: starting mds.e at 2024-04-21T09:05:21.360 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:21 smithi083 bash[61821]: e42708ebc0bf8876a777daf907232ee4ed118244915ec2b44391652f296e58e0 2024-04-21T09:05:21.360 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:05:21 smithi083 systemd[1]: Started Ceph mds.e for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:21.789 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:21.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:21 smithi066 ceph-mon[27953]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:21.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:21.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:21.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:21 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:21.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:21 smithi186 ceph-mon[31505]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:21 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mon[31413]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:21 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 -- 172.21.15.66:0/1067215734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103190 msgr2=0x7f11a4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 --2- 172.21.15.66:0/1067215734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103190 0x7f11a4103610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009ca0 tx=0x7f11a002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 -- 172.21.15.66:0/1067215734 shutdown_connections 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 --2- 172.21.15.66:0/1067215734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11a4103b50 0x7f11a410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 --2- 172.21.15.66:0/1067215734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103190 0x7f11a4103610 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 --2- 172.21.15.66:0/1067215734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a4101fa0 0x7f11a41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 -- 172.21.15.66:0/1067215734 >> 172.21.15.66:0/1067215734 conn(0x7f11a40fd530 msgr2=0x7f11a40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 -- 172.21.15.66:0/1067215734 shutdown_connections 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.904+0000 7f11ac50d640 1 -- 172.21.15.66:0/1067215734 wait complete. 2024-04-21T09:05:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 Processor -- start 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 -- start start 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11a4101fa0 0x7f11a4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a4103190 0x7f11a41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103b50 0x7f11a419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11a410dae0 con 0x7f11a4103b50 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11a410d960 con 0x7f11a4103190 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11ac50d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f11a410dc60 con 0x7f11a4101fa0 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11aaaaa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103b50 0x7f11a419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11aaaaa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103b50 0x7f11a419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43192/0 (socket says 172.21.15.66:43192) 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11aaaaa640 1 -- 172.21.15.66:0/559303447 learned_addr learned my addr 172.21.15.66:0/559303447 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11a9aa8640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a4103190 0x7f11a41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.905+0000 7f11aa2a9640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11a4101fa0 0x7f11a4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11aaaaa640 1 -- 172.21.15.66:0/559303447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11a4101fa0 msgr2=0x7f11a4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11aaaaa640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11a4101fa0 0x7f11a4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11aaaaa640 1 -- 172.21.15.66:0/559303447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a4103190 msgr2=0x7f11a41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11aaaaa640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a4103190 0x7f11a41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11aaaaa640 1 -- 172.21.15.66:0/559303447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a419fd90 con 0x7f11a4103b50 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11aaaaa640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103b50 0x7f11a419f720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f119c00d750 tx=0x7f119c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11977fe640 1 -- 172.21.15.66:0/559303447 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c017070 con 0x7f11a4103b50 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11977fe640 1 -- 172.21.15.66:0/559303447 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f119c0054d0 con 0x7f11a4103b50 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11977fe640 1 -- 172.21.15.66:0/559303447 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c004c60 con 0x7f11a4103b50 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41a0080 con 0x7f11a4103b50 2024-04-21T09:05:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.906+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a41a05f0 con 0x7f11a4103b50 2024-04-21T09:05:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.907+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a41092c0 con 0x7f11a4103b50 2024-04-21T09:05:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.908+0000 7f11977fe640 1 -- 172.21.15.66:0/559303447 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f119c009bd0 con 0x7f11a4103b50 2024-04-21T09:05:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.909+0000 7f11977fe640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f11900a3c30 0x7f11900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.909+0000 7f11aa2a9640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f11900a3c30 0x7f11900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.909+0000 7f11977fe640 1 -- 172.21.15.66:0/559303447 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f119c0cf7c0 con 0x7f11a4103b50 2024-04-21T09:05:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.909+0000 7f11aa2a9640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f11900a3c30 0x7f11900a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f11a4102ff0 tx=0x7f119800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:21.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:21.914+0000 7f11977fe640 1 -- 172.21.15.66:0/559303447 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f119c09bff0 con 0x7f11a4103b50 2024-04-21T09:05:22.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.141+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11a402d5c0 con 0x7f11a4103b50 2024-04-21T09:05:22.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.142+0000 7f11977fe640 1 -- 172.21.15.66:0/559303447 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f119c0a02f0 con 0x7f11a4103b50 2024-04-21T09:05:22.143 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:22.143 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-21T09:05:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 6 2024-04-21T09:05:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f11900a3c30 msgr2=0x7f11900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f11900a3c30 0x7f11900a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f11a4102ff0 tx=0x7f119800b390 comp rx=0 tx=0).stop 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103b50 msgr2=0x7f11a419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103b50 0x7f11a419f720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f119c00d750 tx=0x7f119c00dc10 comp rx=0 tx=0).stop 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 shutdown_connections 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f11900a3c30 0x7f11900a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a4103b50 0x7f11a419f720 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a4103190 0x7f11a41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 --2- 172.21.15.66:0/559303447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11a4101fa0 0x7f11a4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 >> 172.21.15.66:0/559303447 conn(0x7f11a40fd530 msgr2=0x7f11a4106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 shutdown_connections 2024-04-21T09:05:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:22.144+0000 7f11ac50d640 1 -- 172.21.15.66:0/559303447 wait complete. 2024-04-21T09:05:22.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:22 smithi186 ceph-mon[31505]: Deploying daemon mds.f on smithi186 2024-04-21T09:05:22.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:22 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6834/2851815427,v1:172.21.15.83:6835/2851815427] up:boot 2024-04-21T09:05:22.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:22 smithi186 ceph-mon[31505]: fsmap 5 up:standby 2024-04-21T09:05:22.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:22 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:05:22.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:22 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/559303447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:22.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:22 smithi083 ceph-mon[31413]: Deploying daemon mds.f on smithi186 2024-04-21T09:05:22.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:22 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6834/2851815427,v1:172.21.15.83:6835/2851815427] up:boot 2024-04-21T09:05:22.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:22 smithi083 ceph-mon[31413]: fsmap 5 up:standby 2024-04-21T09:05:22.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:22 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:05:22.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:22 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/559303447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:22.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:22 smithi066 ceph-mon[27953]: Deploying daemon mds.f on smithi186 2024-04-21T09:05:22.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:22 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6834/2851815427,v1:172.21.15.83:6835/2851815427] up:boot 2024-04-21T09:05:22.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:22 smithi066 ceph-mon[27953]: fsmap 5 up:standby 2024-04-21T09:05:22.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:22 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:05:22.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/559303447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:23.432 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 systemd[1]: Starting Ceph mds.f for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:23.692 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:23 smithi186 ceph-mon[31505]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:23.692 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 podman[62477]: 2024-04-21 09:05:23.409601651 +0000 UTC m=+0.017936774 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:23.692 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 podman[62477]: 2024-04-21 09:05:23.558474237 +0000 UTC m=+0.166809393 container create deac478869e68e406000b3123513b7c751ea415493b662310c55894e92eb0599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:05:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:23 smithi083 ceph-mon[31413]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:23.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:23 smithi066 ceph-mon[27953]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:24.087 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 podman[62477]: 2024-04-21 09:05:23.811031093 +0000 UTC m=+0.419366206 container init deac478869e68e406000b3123513b7c751ea415493b662310c55894e92eb0599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:05:24.088 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 podman[62477]: 2024-04-21 09:05:23.813554174 +0000 UTC m=+0.421889287 container start deac478869e68e406000b3123513b7c751ea415493b662310c55894e92eb0599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:05:24.088 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 ceph-mds[62496]: -- 172.21.15.186:0/433695319 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603dd6c1a40 con 0x5603dd694c00 2024-04-21T09:05:24.088 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f[62492]: starting mds.f at 2024-04-21T09:05:24.088 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 bash[62477]: deac478869e68e406000b3123513b7c751ea415493b662310c55894e92eb0599 2024-04-21T09:05:24.088 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:05:23 smithi186 systemd[1]: Started Ceph mds.f for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:24.157 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:24.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 -- 172.21.15.66:0/601667677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c102050 msgr2=0x7f4b7c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:24.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 --2- 172.21.15.66:0/601667677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c102050 0x7f4b7c1024d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009900 tx=0x7f4b7002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:24.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 -- 172.21.15.66:0/601667677 shutdown_connections 2024-04-21T09:05:24.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 --2- 172.21.15.66:0/601667677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b7c102a10 0x7f4b7c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 --2- 172.21.15.66:0/601667677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c102050 0x7f4b7c1024d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 --2- 172.21.15.66:0/601667677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b7c100e70 0x7f4b7c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 -- 172.21.15.66:0/601667677 >> 172.21.15.66:0/601667677 conn(0x7f4b7c0fc470 msgr2=0x7f4b7c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:24.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 -- 172.21.15.66:0/601667677 shutdown_connections 2024-04-21T09:05:24.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.271+0000 7f4b82532640 1 -- 172.21.15.66:0/601667677 wait complete. 2024-04-21T09:05:24.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 Processor -- start 2024-04-21T09:05:24.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 -- start start 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c100e70 0x7f4b7c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b7c102050 0x7f4b7c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b7c102a10 0x7f4b7c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10ca00 con 0x7f4b7c100e70 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10c880 con 0x7f4b7c102a10 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b82532640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10cb80 con 0x7f4b7c102050 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c100e70 0x7f4b7c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:24.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c100e70 0x7f4b7c1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43212/0 (socket says 172.21.15.66:43212) 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b7bfff640 1 -- 172.21.15.66:0/859616768 learned_addr learned my addr 172.21.15.66:0/859616768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b7b7fe640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b7c102050 0x7f4b7c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.272+0000 7f4b80acf640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b7c102a10 0x7f4b7c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b7bfff640 1 -- 172.21.15.66:0/859616768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b7c102050 msgr2=0x7f4b7c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b7bfff640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b7c102050 0x7f4b7c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b7bfff640 1 -- 172.21.15.66:0/859616768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b7c102a10 msgr2=0x7f4b7c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b7bfff640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b7c102a10 0x7f4b7c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b7bfff640 1 -- 172.21.15.66:0/859616768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c19ecc0 con 0x7f4b7c100e70 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b80acf640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b7c102a10 0x7f4b7c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b7b7fe640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b7c102050 0x7f4b7c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b7bfff640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c100e70 0x7f4b7c1970a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00e9c0 tx=0x7f4b6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b797fa640 1 -- 172.21.15.66:0/859616768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00cd80 con 0x7f4b7c100e70 2024-04-21T09:05:24.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b797fa640 1 -- 172.21.15.66:0/859616768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b6c0035f0 con 0x7f4b7c100e70 2024-04-21T09:05:24.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7c19efb0 con 0x7f4b7c100e70 2024-04-21T09:05:24.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b797fa640 1 -- 172.21.15.66:0/859616768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c010690 con 0x7f4b7c100e70 2024-04-21T09:05:24.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.273+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7c19f410 con 0x7f4b7c100e70 2024-04-21T09:05:24.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.274+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b7c108210 con 0x7f4b7c100e70 2024-04-21T09:05:24.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.275+0000 7f4b797fa640 1 -- 172.21.15.66:0/859616768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4b6c0037a0 con 0x7f4b7c100e70 2024-04-21T09:05:24.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.276+0000 7f4b797fa640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4b640a3bb0 0x7f4b640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:24.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.276+0000 7f4b7b7fe640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4b640a3bb0 0x7f4b640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:24.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.276+0000 7f4b797fa640 1 -- 172.21.15.66:0/859616768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4b6c0d0370 con 0x7f4b7c100e70 2024-04-21T09:05:24.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.277+0000 7f4b7b7fe640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4b640a3bb0 0x7f4b640a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009230 tx=0x7f4b70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:24.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.280+0000 7f4b797fa640 1 -- 172.21.15.66:0/859616768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b6c09d1f0 con 0x7f4b7c100e70 2024-04-21T09:05:24.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.506+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b7c02d870 con 0x7f4b7c100e70 2024-04-21T09:05:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.506+0000 7f4b797fa640 1 -- 172.21.15.66:0/859616768 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f4b6c09de20 con 0x7f4b7c100e70 2024-04-21T09:05:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:24.507 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-21T09:05:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 7 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4b640a3bb0 msgr2=0x7f4b640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4b640a3bb0 0x7f4b640a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009230 tx=0x7f4b70005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c100e70 msgr2=0x7f4b7c1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c100e70 0x7f4b7c1970a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00e9c0 tx=0x7f4b6c00ee80 comp rx=0 tx=0).stop 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 shutdown_connections 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4b640a3bb0 0x7f4b640a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b7c102a10 0x7f4b7c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b7c102050 0x7f4b7c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 --2- 172.21.15.66:0/859616768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b7c100e70 0x7f4b7c1970a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 >> 172.21.15.66:0/859616768 conn(0x7f4b7c0fc470 msgr2=0x7f4b7c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 shutdown_connections 2024-04-21T09:05:24.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:24.508+0000 7f4b82532640 1 -- 172.21.15.66:0/859616768 wait complete. 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: Deploying daemon mds.c on smithi186 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6834/1405629184,v1:172.21.15.186:6835/1405629184] up:boot 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: fsmap 6 up:standby 2024-04-21T09:05:25.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:05:25.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:24 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/859616768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: Deploying daemon mds.c on smithi186 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6834/1405629184,v1:172.21.15.186:6835/1405629184] up:boot 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: fsmap 6 up:standby 2024-04-21T09:05:25.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:05:25.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:24 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/859616768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: Deploying daemon mds.c on smithi186 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6834/1405629184,v1:172.21.15.186:6835/1405629184] up:boot 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: fsmap 6 up:standby 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:05:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:24 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/859616768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:26.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:26 smithi186 ceph-mon[31505]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:26.338 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 systemd[1]: Starting Ceph mds.c for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:26.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:26 smithi083 ceph-mon[31413]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:26.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:26 smithi066 ceph-mon[27953]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:26.521 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 -- 172.21.15.66:0/1401998570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84101fc0 msgr2=0x7feb84102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 --2- 172.21.15.66:0/1401998570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84101fc0 0x7feb84102440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009900 tx=0x7feb6c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 -- 172.21.15.66:0/1401998570 shutdown_connections 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 --2- 172.21.15.66:0/1401998570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb84102980 0x7feb841092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 --2- 172.21.15.66:0/1401998570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84101fc0 0x7feb84102440 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 --2- 172.21.15.66:0/1401998570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb84100e70 0x7feb84101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 -- 172.21.15.66:0/1401998570 >> 172.21.15.66:0/1401998570 conn(0x7feb840fc3e0 msgr2=0x7feb840fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:26.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 -- 172.21.15.66:0/1401998570 shutdown_connections 2024-04-21T09:05:26.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 -- 172.21.15.66:0/1401998570 wait complete. 2024-04-21T09:05:26.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.635+0000 7feb8a152640 1 Processor -- start 2024-04-21T09:05:26.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb8a152640 1 -- start start 2024-04-21T09:05:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb8a152640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb84100e70 0x7feb84069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb8a152640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb84101fc0 0x7feb8419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb8a152640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84102980 0x7feb8419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb8a152640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410c9c0 con 0x7feb84102980 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb8a152640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410c840 con 0x7feb84101fc0 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb8a152640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410cb40 con 0x7feb84100e70 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84102980 0x7feb8419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84102980 0x7feb8419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57108/0 (socket says 172.21.15.66:57108) 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 -- 172.21.15.66:0/1338636799 learned_addr learned my addr 172.21.15.66:0/1338636799 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb82ffd640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb84101fc0 0x7feb8419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb837fe640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb84100e70 0x7feb84069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 -- 172.21.15.66:0/1338636799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb84100e70 msgr2=0x7feb84069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb84100e70 0x7feb84069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 -- 172.21.15.66:0/1338636799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb84101fc0 msgr2=0x7feb8419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:26.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb84101fc0 0x7feb8419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.636+0000 7feb83fff640 1 -- 172.21.15.66:0/1338636799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8419fe90 con 0x7feb84102980 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.637+0000 7feb83fff640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84102980 0x7feb8419f580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009fd0 tx=0x7feb7c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.637+0000 7feb80ff9640 1 -- 172.21.15.66:0/1338636799 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c017070 con 0x7feb84102980 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.637+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb841a0120 con 0x7feb84102980 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.637+0000 7feb80ff9640 1 -- 172.21.15.66:0/1338636799 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7c003e20 con 0x7feb84102980 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.637+0000 7feb80ff9640 1 -- 172.21.15.66:0/1338636799 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c010460 con 0x7feb84102980 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.637+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841a05f0 con 0x7feb84102980 2024-04-21T09:05:26.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.638+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb84108250 con 0x7feb84102980 2024-04-21T09:05:26.641 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 podman[62787]: 2024-04-21 09:05:26.263503886 +0000 UTC m=+0.017733180 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:26.641 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 podman[62787]: 2024-04-21 09:05:26.380122122 +0000 UTC m=+0.134351402 container create a7641693372a8e982e24d343985bb74f7c1080dfd577051b8aac366f15cf55fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5) 2024-04-21T09:05:26.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.639+0000 7feb80ff9640 1 -- 172.21.15.66:0/1338636799 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb7c003970 con 0x7feb84102980 2024-04-21T09:05:26.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.640+0000 7feb80ff9640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7feb700a3bb0 0x7feb700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:26.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.640+0000 7feb837fe640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7feb700a3bb0 0x7feb700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:26.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.640+0000 7feb80ff9640 1 -- 172.21.15.66:0/1338636799 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7feb7c0cfd20 con 0x7feb84102980 2024-04-21T09:05:26.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.640+0000 7feb837fe640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7feb700a3bb0 0x7feb700a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb74005fd0 tx=0x7feb74005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:26.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.645+0000 7feb80ff9640 1 -- 172.21.15.66:0/1338636799 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb7c09cba0 con 0x7feb84102980 2024-04-21T09:05:26.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.873+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb8402d810 con 0x7feb84102980 2024-04-21T09:05:26.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.873+0000 7feb80ff9640 1 -- 172.21.15.66:0/1338636799 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7feb7c09c4a0 con 0x7feb84102980 2024-04-21T09:05:26.874 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:26.874 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-21T09:05:26.874 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 7 2024-04-21T09:05:26.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7feb700a3bb0 msgr2=0x7feb700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:26.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7feb700a3bb0 0x7feb700a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb74005fd0 tx=0x7feb74005850 comp rx=0 tx=0).stop 2024-04-21T09:05:26.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84102980 msgr2=0x7feb8419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84102980 0x7feb8419f580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009fd0 tx=0x7feb7c00b260 comp rx=0 tx=0).stop 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 shutdown_connections 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7feb700a3bb0 0x7feb700a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb84102980 0x7feb8419f580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb84101fc0 0x7feb8419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 --2- 172.21.15.66:0/1338636799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb84100e70 0x7feb84069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 >> 172.21.15.66:0/1338636799 conn(0x7feb840fc3e0 msgr2=0x7feb84105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 shutdown_connections 2024-04-21T09:05:26.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:26.876+0000 7feb8a152640 1 -- 172.21.15.66:0/1338636799 wait complete. 2024-04-21T09:05:27.088 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 podman[62787]: 2024-04-21 09:05:26.641741001 +0000 UTC m=+0.395970282 container init a7641693372a8e982e24d343985bb74f7c1080dfd577051b8aac366f15cf55fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-21T09:05:27.088 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 podman[62787]: 2024-04-21 09:05:26.644731863 +0000 UTC m=+0.398961152 container start a7641693372a8e982e24d343985bb74f7c1080dfd577051b8aac366f15cf55fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-21T09:05:27.088 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 ceph-mds[62806]: -- 172.21.15.186:0/751389257 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a07f2afa40 con 0x55a07f283000 2024-04-21T09:05:27.088 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c[62802]: starting mds.c at 2024-04-21T09:05:27.088 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 bash[62787]: a7641693372a8e982e24d343985bb74f7c1080dfd577051b8aac366f15cf55fb 2024-04-21T09:05:27.088 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:05:26 smithi186 systemd[1]: Started Ceph mds.c for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: Deploying daemon mds.h on smithi083 2024-04-21T09:05:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1338636799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6836/1780100152,v1:172.21.15.186:6837/1780100152] up:boot 2024-04-21T09:05:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: fsmap 7 up:standby 2024-04-21T09:05:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:27 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:05:28.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:28.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: Deploying daemon mds.h on smithi083 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1338636799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6836/1780100152,v1:172.21.15.186:6837/1780100152] up:boot 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: fsmap 7 up:standby 2024-04-21T09:05:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:27 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:05:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: Deploying daemon mds.h on smithi083 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1338636799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6836/1780100152,v1:172.21.15.186:6837/1780100152] up:boot 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: fsmap 7 up:standby 2024-04-21T09:05:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:27 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:05:28.889 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 -- 172.21.15.66:0/857565268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c100620 msgr2=0x7f6e0c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 --2- 172.21.15.66:0/857565268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c100620 0x7f6e0c100a40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009900 tx=0x7f6e0402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 -- 172.21.15.66:0/857565268 shutdown_connections 2024-04-21T09:05:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 --2- 172.21.15.66:0/857565268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 --2- 172.21.15.66:0/857565268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e0c101810 0x7f6e0c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 --2- 172.21.15.66:0/857565268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c100620 0x7f6e0c100a40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 -- 172.21.15.66:0/857565268 >> 172.21.15.66:0/857565268 conn(0x7f6e0c0fbb90 msgr2=0x7f6e0c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 -- 172.21.15.66:0/857565268 shutdown_connections 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 -- 172.21.15.66:0/857565268 wait complete. 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.003+0000 7f6e1181f640 1 Processor -- start 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e1181f640 1 -- start start 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e1181f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e0c100620 0x7f6e0c1045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e1181f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c101810 0x7f6e0c102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e1181f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e1181f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c10b8e0 con 0x7f6e0c101810 2024-04-21T09:05:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e1181f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c10b760 con 0x7f6e0c1021d0 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e1181f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c10ba60 con 0x7f6e0c100620 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c101810 0x7f6e0c102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c101810 0x7f6e0c102bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57140/0 (socket says 172.21.15.66:57140) 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 -- 172.21.15.66:0/2831997239 learned_addr learned my addr 172.21.15.66:0/2831997239 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0b7fe640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0affd640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e0c100620 0x7f6e0c1045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 -- 172.21.15.66:0/2831997239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e0c100620 msgr2=0x7f6e0c1045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e0c100620 0x7f6e0c1045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 -- 172.21.15.66:0/2831997239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1021d0 msgr2=0x7f6e0c103130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c103130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.004+0000 7f6e0a7fc640 1 -- 172.21.15.66:0/2831997239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c1039c0 con 0x7f6e0c101810 2024-04-21T09:05:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.005+0000 7f6e0a7fc640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c101810 0x7f6e0c102bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000dbc0 tx=0x7f6e0000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.005+0000 7f6e1081d640 1 -- 172.21.15.66:0/2831997239 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00005020 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.005+0000 7f6e1081d640 1 -- 172.21.15.66:0/2831997239 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e000047a0 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.005+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c1a2320 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.005+0000 7f6e1081d640 1 -- 172.21.15.66:0/2831997239 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000e740 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.005+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c1a2820 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.006+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c1a2ae0 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.007+0000 7f6e1081d640 1 -- 172.21.15.66:0/2831997239 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e00004950 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.007+0000 7f6e1081d640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6df40a3bb0 0x7f6df40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.008+0000 7f6e0affd640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6df40a3bb0 0x7f6df40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.008+0000 7f6e1081d640 1 -- 172.21.15.66:0/2831997239 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6e000cff30 con 0x7f6e0c101810 2024-04-21T09:05:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.008+0000 7f6e0affd640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6df40a3bb0 0x7f6df40a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e04005ab0 tx=0x7f6e04005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.012+0000 7f6e1081d640 1 -- 172.21.15.66:0/2831997239 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e0009cdb0 con 0x7f6e0c101810 2024-04-21T09:05:29.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.242+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e0c1a2cb0 con 0x7f6e0c101810 2024-04-21T09:05:29.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.242+0000 7f6e1081d640 1 -- 172.21.15.66:0/2831997239 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f6e000870c0 con 0x7f6e0c101810 2024-04-21T09:05:29.243 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:29.243 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T09:05:29.244 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-04-21T09:05:29.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6df40a3bb0 msgr2=0x7f6df40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:29.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6df40a3bb0 0x7f6df40a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e04005ab0 tx=0x7f6e04005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c101810 msgr2=0x7f6e0c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c101810 0x7f6e0c102bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000dbc0 tx=0x7f6e0000a250 comp rx=0 tx=0).stop 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 shutdown_connections 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6df40a3bb0 0x7f6df40a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e0c101810 0x7f6e0c102bf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 --2- 172.21.15.66:0/2831997239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e0c100620 0x7f6e0c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 >> 172.21.15.66:0/2831997239 conn(0x7f6e0c0fbb90 msgr2=0x7f6e0c106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 shutdown_connections 2024-04-21T09:05:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:29.245+0000 7f6e1181f640 1 -- 172.21.15.66:0/2831997239 wait complete. 2024-04-21T09:05:29.283 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 systemd[1]: Starting Ceph mds.h for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:29.563 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 podman[62134]: 2024-04-21 09:05:29.186194066 +0000 UTC m=+0.017795865 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:29.564 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 podman[62134]: 2024-04-21 09:05:29.327916258 +0000 UTC m=+0.159518042 container create 66486971edd068516916cf9874aaf72b1cf8513377e7e90fc74b3e22b8a41514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-21T09:05:29.564 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 podman[62134]: 2024-04-21 09:05:29.564587321 +0000 UTC m=+0.396189104 container init 66486971edd068516916cf9874aaf72b1cf8513377e7e90fc74b3e22b8a41514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2831997239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:29.861 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 podman[62134]: 2024-04-21 09:05:29.567308347 +0000 UTC m=+0.398910138 container start 66486971edd068516916cf9874aaf72b1cf8513377e7e90fc74b3e22b8a41514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-21T09:05:29.861 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-mds[62153]: -- 172.21.15.83:0/2299537719 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559940f3ba40 con 0x559940f0e800 2024-04-21T09:05:29.861 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h[62149]: starting mds.h at 2024-04-21T09:05:29.861 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 bash[62134]: 66486971edd068516916cf9874aaf72b1cf8513377e7e90fc74b3e22b8a41514 2024-04-21T09:05:29.861 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:05:29 smithi083 systemd[1]: Started Ceph mds.h for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2831997239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:29 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:30.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:30.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2831997239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:30.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:30.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:30.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:30.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:30.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:30.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:29 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:31.031 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:30 smithi186 ceph-mon[31505]: Deploying daemon mds.i on smithi186 2024-04-21T09:05:31.031 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:30 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] up:boot 2024-04-21T09:05:31.031 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:30 smithi186 ceph-mon[31505]: fsmap 8 up:standby 2024-04-21T09:05:31.031 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:30 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:05:31.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:30 smithi083 ceph-mon[31413]: Deploying daemon mds.i on smithi186 2024-04-21T09:05:31.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:30 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] up:boot 2024-04-21T09:05:31.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:30 smithi083 ceph-mon[31413]: fsmap 8 up:standby 2024-04-21T09:05:31.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:30 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:05:31.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:30 smithi066 ceph-mon[27953]: Deploying daemon mds.i on smithi186 2024-04-21T09:05:31.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:30 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] up:boot 2024-04-21T09:05:31.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:30 smithi066 ceph-mon[27953]: fsmap 8 up:standby 2024-04-21T09:05:31.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:30 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:05:31.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:31.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 -- 172.21.15.66:0/413333402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954102a20 msgr2=0x7f7954104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:31.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 --2- 172.21.15.66:0/413333402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954102a20 0x7f7954104e10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7944009900 tx=0x7f794402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:31.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 -- 172.21.15.66:0/413333402 shutdown_connections 2024-04-21T09:05:31.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 --2- 172.21.15.66:0/413333402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954107c80 0x7f795410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 --2- 172.21.15.66:0/413333402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954105350 0x7f7954107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 --2- 172.21.15.66:0/413333402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954102a20 0x7f7954104e10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 -- 172.21.15.66:0/413333402 >> 172.21.15.66:0/413333402 conn(0x7f79540fc410 msgr2=0x7f79540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.372+0000 7f795a63f640 1 -- 172.21.15.66:0/413333402 shutdown_connections 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 -- 172.21.15.66:0/413333402 wait complete. 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 Processor -- start 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 -- start start 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954102a20 0x7f7954198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954105350 0x7f7954198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954107c80 0x7f795419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f795410dae0 con 0x7f7954107c80 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f795410d960 con 0x7f7954105350 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.373+0000 7f795a63f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f795410dc60 con 0x7f7954102a20 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954107c80 0x7f795419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:31.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954107c80 0x7f795419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57166/0 (socket says 172.21.15.66:57166) 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 -- 172.21.15.66:0/4259426912 learned_addr learned my addr 172.21.15.66:0/4259426912 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7953fff640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954102a20 0x7f7954198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f79537fe640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954105350 0x7f7954198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 -- 172.21.15.66:0/4259426912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954102a20 msgr2=0x7f7954198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954102a20 0x7f7954198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 -- 172.21.15.66:0/4259426912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954105350 msgr2=0x7f7954198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954105350 0x7f7954198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 -- 172.21.15.66:0/4259426912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795419fd50 con 0x7f7954107c80 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f7958bdc640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954107c80 0x7f795419f6e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f794c00eaa0 tx=0x7f794c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f79517fa640 1 -- 172.21.15.66:0/4259426912 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c004d20 con 0x7f7954107c80 2024-04-21T09:05:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f79517fa640 1 -- 172.21.15.66:0/4259426912 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794c005020 con 0x7f7954107c80 2024-04-21T09:05:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795419ffe0 con 0x7f7954107c80 2024-04-21T09:05:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f79517fa640 1 -- 172.21.15.66:0/4259426912 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c0116b0 con 0x7f7954107c80 2024-04-21T09:05:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.374+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79541a0550 con 0x7f7954107c80 2024-04-21T09:05:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.375+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79540fdf00 con 0x7f7954107c80 2024-04-21T09:05:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.376+0000 7f79517fa640 1 -- 172.21.15.66:0/4259426912 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f794c0051d0 con 0x7f7954107c80 2024-04-21T09:05:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.377+0000 7f79517fa640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f793c0a3c30 0x7f793c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.377+0000 7f7953fff640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f793c0a3c30 0x7f793c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.377+0000 7f79517fa640 1 -- 172.21.15.66:0/4259426912 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f794c0d0000 con 0x7f7954107c80 2024-04-21T09:05:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.378+0000 7f7953fff640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f793c0a3c30 0x7f793c0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7944009230 tx=0x7f7944005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.382+0000 7f79517fa640 1 -- 172.21.15.66:0/4259426912 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f794c016030 con 0x7f7954107c80 2024-04-21T09:05:31.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.608+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f795402d5c0 con 0x7f7954107c80 2024-04-21T09:05:31.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.609+0000 7f79517fa640 1 -- 172.21.15.66:0/4259426912 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f794c09ce80 con 0x7f7954107c80 2024-04-21T09:05:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:31.610 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T09:05:31.610 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-04-21T09:05:31.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.611+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f793c0a3c30 msgr2=0x7f793c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:31.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.611+0000 7f795a63f640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f793c0a3c30 0x7f793c0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7944009230 tx=0x7f7944005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.611+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954107c80 msgr2=0x7f795419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:31.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.611+0000 7f795a63f640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954107c80 0x7f795419f6e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f794c00eaa0 tx=0x7f794c00eea0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.611+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 shutdown_connections 2024-04-21T09:05:31.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.612+0000 7f795a63f640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f793c0a3c30 0x7f793c0a60f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.612+0000 7f795a63f640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7954107c80 0x7f795419f6e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.612+0000 7f795a63f640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954105350 0x7f7954198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.612+0000 7f795a63f640 1 --2- 172.21.15.66:0/4259426912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954102a20 0x7f7954198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:31.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.612+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 >> 172.21.15.66:0/4259426912 conn(0x7f79540fc410 msgr2=0x7f79541034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:31.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.612+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 shutdown_connections 2024-04-21T09:05:31.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:31.612+0000 7f795a63f640 1 -- 172.21.15.66:0/4259426912 wait complete. 2024-04-21T09:05:31.946 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:31 smithi186 ceph-mon[31505]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:31.946 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:31 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4259426912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:31.946 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:31 smithi186 systemd[1]: Starting Ceph mds.i for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:32.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:31 smithi083 ceph-mon[31413]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:32.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:31 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4259426912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:32.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:31 smithi066 ceph-mon[27953]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:32.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:31 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4259426912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:32.277 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 podman[63102]: 2024-04-21 09:05:31.920348194 +0000 UTC m=+0.021614626 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:32.278 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 podman[63102]: 2024-04-21 09:05:32.025539724 +0000 UTC m=+0.126806197 container create 51d59c97461494b91c2e93bad739f172fdabec7b48ad552ca0e61244e33fdc98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-21T09:05:32.278 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 podman[63102]: 2024-04-21 09:05:32.278155354 +0000 UTC m=+0.379421786 container init 51d59c97461494b91c2e93bad739f172fdabec7b48ad552ca0e61244e33fdc98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:05:32.587 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 podman[63102]: 2024-04-21 09:05:32.280783026 +0000 UTC m=+0.382049457 container start 51d59c97461494b91c2e93bad739f172fdabec7b48ad552ca0e61244e33fdc98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-21T09:05:32.588 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 ceph-mds[63122]: -- 172.21.15.186:0/3720519377 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595c8e09a40 con 0x5595c8ddc800 2024-04-21T09:05:32.588 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i[63118]: starting mds.i at 2024-04-21T09:05:32.588 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 bash[63102]: 51d59c97461494b91c2e93bad739f172fdabec7b48ad552ca0e61244e33fdc98 2024-04-21T09:05:32.588 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:05:32 smithi186 systemd[1]: Started Ceph mds.i for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:33.623 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:33.648 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: Deploying daemon mds.g on smithi066 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] up:boot 2024-04-21T09:05:33.649 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: fsmap 9 up:standby 2024-04-21T09:05:33.650 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:33 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:05:33.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 -- 172.21.15.66:0/165023999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c102a40 msgr2=0x7fd39c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:33.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/165023999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c102a40 0x7fd39c104e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009900 tx=0x7fd38c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:33.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 -- 172.21.15.66:0/165023999 shutdown_connections 2024-04-21T09:05:33.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/165023999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c107ca0 0x7fd39c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/165023999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd39c105370 0x7fd39c107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/165023999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c102a40 0x7fd39c104e30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 -- 172.21.15.66:0/165023999 >> 172.21.15.66:0/165023999 conn(0x7fd39c0fc470 msgr2=0x7fd39c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.729+0000 7fd3a3c77640 1 -- 172.21.15.66:0/165023999 shutdown_connections 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 -- 172.21.15.66:0/165023999 wait complete. 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 Processor -- start 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 -- start start 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd39c102a40 0x7fd39c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c105370 0x7fd39c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c107ca0 0x7fd39c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10db00 con 0x7fd39c105370 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10d980 con 0x7fd39c107ca0 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a3c77640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10dc80 con 0x7fd39c102a40 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a1212640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c105370 0x7fd39c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a1212640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c105370 0x7fd39c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57184/0 (socket says 172.21.15.66:57184) 2024-04-21T09:05:33.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a1212640 1 -- 172.21.15.66:0/812295622 learned_addr learned my addr 172.21.15.66:0/812295622 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.730+0000 7fd3a2214640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c107ca0 0x7fd39c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a1212640 1 -- 172.21.15.66:0/812295622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd39c102a40 msgr2=0x7fd39c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a1a13640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd39c102a40 0x7fd39c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a1212640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd39c102a40 0x7fd39c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a1212640 1 -- 172.21.15.66:0/812295622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c107ca0 msgr2=0x7fd39c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a1212640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c107ca0 0x7fd39c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a1212640 1 -- 172.21.15.66:0/812295622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c19fe00 con 0x7fd39c105370 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a1212640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c105370 0x7fd39c198720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd39800d740 tx=0x7fd39800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd392ffd640 1 -- 172.21.15.66:0/812295622 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398017070 con 0x7fd39c105370 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd392ffd640 1 -- 172.21.15.66:0/812295622 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd39800de50 con 0x7fd39c105370 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c1a00f0 con 0x7fd39c105370 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd392ffd640 1 -- 172.21.15.66:0/812295622 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398004ce0 con 0x7fd39c105370 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.731+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c1a0640 con 0x7fd39c105370 2024-04-21T09:05:33.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.732+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39c0fdf60 con 0x7fd39c105370 2024-04-21T09:05:33.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.733+0000 7fd392ffd640 1 -- 172.21.15.66:0/812295622 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd398004e80 con 0x7fd39c105370 2024-04-21T09:05:33.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.734+0000 7fd392ffd640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:33.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.734+0000 7fd3a1a13640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:33.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.734+0000 7fd392ffd640 1 -- 172.21.15.66:0/812295622 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd3980cf930 con 0x7fd39c105370 2024-04-21T09:05:33.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.734+0000 7fd3a1a13640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd3880a3c30 0x7fd3880a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0096c0 tx=0x7fd38c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.739+0000 7fd392ffd640 1 -- 172.21.15.66:0/812295622 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd39809c7b0 con 0x7fd39c105370 2024-04-21T09:05:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: Deploying daemon mds.g on smithi066 2024-04-21T09:05:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] up:boot 2024-04-21T09:05:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: fsmap 9 up:standby 2024-04-21T09:05:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:33 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:05:33.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:33.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:33.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:33.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:33.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:33.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: Deploying daemon mds.g on smithi066 2024-04-21T09:05:33.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] up:boot 2024-04-21T09:05:33.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: fsmap 9 up:standby 2024-04-21T09:05:33.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:33 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:05:33.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.966+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd39c02d810 con 0x7fd39c105370 2024-04-21T09:05:33.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.966+0000 7fd392ffd640 1 -- 172.21.15.66:0/812295622 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7fd39809c0b0 con 0x7fd39c105370 2024-04-21T09:05:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:33.967 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T09:05:33.967 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 10 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd3880a3c30 msgr2=0x7fd3880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd3880a3c30 0x7fd3880a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0096c0 tx=0x7fd38c000bc0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c105370 msgr2=0x7fd39c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c105370 0x7fd39c198720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd39800d740 tx=0x7fd39800dc00 comp rx=0 tx=0).stop 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 shutdown_connections 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c107ca0 0x7fd39c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd39c105370 0x7fd39c198720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 --2- 172.21.15.66:0/812295622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd39c102a40 0x7fd39c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 >> 172.21.15.66:0/812295622 conn(0x7fd39c0fc470 msgr2=0x7fd39c103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 shutdown_connections 2024-04-21T09:05:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:33.968+0000 7fd3a3c77640 1 -- 172.21.15.66:0/812295622 wait complete. 2024-04-21T09:05:34.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:34 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/812295622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:34.848 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:34 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/812295622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:34.848 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:34 smithi066 systemd[1]: Starting Ceph mds.g for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:34.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:34 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/812295622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:35.120 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:34 smithi066 podman[66847]: 2024-04-21 09:05:34.751781841 +0000 UTC m=+0.018487396 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:35.120 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:34 smithi066 podman[66847]: 2024-04-21 09:05:34.856288126 +0000 UTC m=+0.122993666 container create 7422db8583215f803276163acacf116013fdeac5f219c67f885549974a54783a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-21T09:05:35.430 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:35 smithi066 podman[66847]: 2024-04-21 09:05:35.25066053 +0000 UTC m=+0.517366068 container init 7422db8583215f803276163acacf116013fdeac5f219c67f885549974a54783a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:05:35.430 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:35 smithi066 podman[66847]: 2024-04-21 09:05:35.25364194 +0000 UTC m=+0.520347475 container start 7422db8583215f803276163acacf116013fdeac5f219c67f885549974a54783a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T09:05:35.430 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:35 smithi066 ceph-mds[66866]: -- 172.21.15.66:0/509352808 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598ec911a40 con 0x5598ec8e4c00 2024-04-21T09:05:35.430 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g[66862]: starting mds.g at 2024-04-21T09:05:35.430 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:35 smithi066 bash[66847]: 7422db8583215f803276163acacf116013fdeac5f219c67f885549974a54783a 2024-04-21T09:05:35.430 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:05:35 smithi066 systemd[1]: Started Ceph mds.g for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:35.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:35 smithi186 ceph-mon[31505]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:35 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:35 smithi083 ceph-mon[31413]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:35.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:35.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:35 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:35.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:35 smithi066 ceph-mon[27953]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:35.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:35.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:35 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:35.981 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 -- 172.21.15.66:0/1992742491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748101fc0 msgr2=0x7f0748102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 --2- 172.21.15.66:0/1992742491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748101fc0 0x7f0748102440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0730009900 tx=0x7f073002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 -- 172.21.15.66:0/1992742491 shutdown_connections 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 --2- 172.21.15.66:0/1992742491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748102980 0x7f07481092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 --2- 172.21.15.66:0/1992742491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748101fc0 0x7f0748102440 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 --2- 172.21.15.66:0/1992742491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0748100e70 0x7f0748101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 -- 172.21.15.66:0/1992742491 >> 172.21.15.66:0/1992742491 conn(0x7f07480fc3e0 msgr2=0x7f07480fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 -- 172.21.15.66:0/1992742491 shutdown_connections 2024-04-21T09:05:36.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.085+0000 7f074d39f640 1 -- 172.21.15.66:0/1992742491 wait complete. 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 Processor -- start 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 -- start start 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748100e70 0x7f0748198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0748101fc0 0x7f07481986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748102980 0x7f074819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f074810c9c0 con 0x7f0748100e70 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f074810c840 con 0x7f0748102980 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f074d39f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f074810cb40 con 0x7f0748101fc0 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f0746ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748100e70 0x7f0748198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f0746ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748100e70 0x7f0748198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47674/0 (socket says 172.21.15.66:47674) 2024-04-21T09:05:36.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.086+0000 7f0746ffd640 1 -- 172.21.15.66:0/827454648 learned_addr learned my addr 172.21.15.66:0/827454648 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f07477fe640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748102980 0x7f074819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f0746ffd640 1 -- 172.21.15.66:0/827454648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0748101fc0 msgr2=0x7f07481986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f07467fc640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0748101fc0 0x7f07481986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f0746ffd640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0748101fc0 0x7f07481986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f0746ffd640 1 -- 172.21.15.66:0/827454648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748102980 msgr2=0x7f074819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f0746ffd640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748102980 0x7f074819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f0746ffd640 1 -- 172.21.15.66:0/827454648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074819fd80 con 0x7f0748100e70 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f07477fe640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748102980 0x7f074819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f07467fc640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0748101fc0 0x7f07481986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f0746ffd640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748100e70 0x7f0748198160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f074000eb90 tx=0x7f074000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f073ffff640 1 -- 172.21.15.66:0/827454648 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740004d20 con 0x7f0748100e70 2024-04-21T09:05:36.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07481a0070 con 0x7f0748100e70 2024-04-21T09:05:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f073ffff640 1 -- 172.21.15.66:0/827454648 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074000ee90 con 0x7f0748100e70 2024-04-21T09:05:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07481a05c0 con 0x7f0748100e70 2024-04-21T09:05:36.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.087+0000 7f073ffff640 1 -- 172.21.15.66:0/827454648 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740010670 con 0x7f0748100e70 2024-04-21T09:05:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.089+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07280052c0 con 0x7f0748100e70 2024-04-21T09:05:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.090+0000 7f073ffff640 1 -- 172.21.15.66:0/827454648 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0740005020 con 0x7f0748100e70 2024-04-21T09:05:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.091+0000 7f073ffff640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07340a3bb0 0x7f07340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.091+0000 7f07467fc640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07340a3bb0 0x7f07340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.091+0000 7f073ffff640 1 -- 172.21.15.66:0/827454648 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f07400d0630 con 0x7f0748100e70 2024-04-21T09:05:36.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.091+0000 7f07467fc640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07340a3bb0 0x7f07340a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0730036040 tx=0x7f0730000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.095+0000 7f073ffff640 1 -- 172.21.15.66:0/827454648 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f074009d4b0 con 0x7f0748100e70 2024-04-21T09:05:36.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.325+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0728005880 con 0x7f0748100e70 2024-04-21T09:05:36.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.326+0000 7f073ffff640 1 -- 172.21.15.66:0/827454648 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f074009cdb0 con 0x7f0748100e70 2024-04-21T09:05:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:36.326 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T09:05:36.326 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 11 2024-04-21T09:05:36.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07340a3bb0 msgr2=0x7f07340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:36.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07340a3bb0 0x7f07340a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0730036040 tx=0x7f0730000bc0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748100e70 msgr2=0x7f0748198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:36.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748100e70 0x7f0748198160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f074000eb90 tx=0x7f074000c6a0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 shutdown_connections 2024-04-21T09:05:36.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07340a3bb0 0x7f07340a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748102980 0x7f074819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0748101fc0 0x7f07481986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 --2- 172.21.15.66:0/827454648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0748100e70 0x7f0748198160 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:36.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 >> 172.21.15.66:0/827454648 conn(0x7f07480fc3e0 msgr2=0x7f07480fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:36.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 shutdown_connections 2024-04-21T09:05:36.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:36.328+0000 7f074d39f640 1 -- 172.21.15.66:0/827454648 wait complete. 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.66:6838/2337695414,v1:172.21.15.66:6839/2337695414] up:boot 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: fsmap 10 up:standby 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:36.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: Deploying daemon mds.a on smithi066 2024-04-21T09:05:36.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:36 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/827454648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:36.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.66:6838/2337695414,v1:172.21.15.66:6839/2337695414] up:boot 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: fsmap 10 up:standby 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: Deploying daemon mds.a on smithi066 2024-04-21T09:05:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:36 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/827454648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:36.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.66:6838/2337695414,v1:172.21.15.66:6839/2337695414] up:boot 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: fsmap 10 up:standby 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: Deploying daemon mds.a on smithi066 2024-04-21T09:05:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:36 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/827454648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:37.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:37 smithi066 ceph-mon[27953]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:37.620 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:37 smithi066 systemd[1]: Starting Ceph mds.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:37.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:37 smithi186 ceph-mon[31505]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:37.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:37 smithi083 ceph-mon[31413]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:38.072 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:37 smithi066 podman[67195]: 2024-04-21 09:05:37.761951428 +0000 UTC m=+0.114089471 container create c81f341de8057637ae653f80b2505fb9990934826b76e27b2a5799a6f4862893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, ceph=True, io.buildah.version=1.33.5) 2024-04-21T09:05:38.072 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:37 smithi066 podman[67195]: 2024-04-21 09:05:37.666137921 +0000 UTC m=+0.018275934 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:38.072 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:37 smithi066 podman[67195]: 2024-04-21 09:05:37.972813454 +0000 UTC m=+0.324951455 container init c81f341de8057637ae653f80b2505fb9990934826b76e27b2a5799a6f4862893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T09:05:38.072 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:37 smithi066 podman[67195]: 2024-04-21 09:05:37.975479576 +0000 UTC m=+0.327617584 container start c81f341de8057637ae653f80b2505fb9990934826b76e27b2a5799a6f4862893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-21T09:05:38.072 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:37 smithi066 ceph-mds[67214]: -- 172.21.15.66:0/995584047 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f345653a40 con 0x55f345627000 2024-04-21T09:05:38.072 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a[67210]: starting mds.a at 2024-04-21T09:05:38.072 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:38 smithi066 bash[67195]: c81f341de8057637ae653f80b2505fb9990934826b76e27b2a5799a6f4862893 2024-04-21T09:05:38.341 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:38.387 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:05:38 smithi066 systemd[1]: Started Ceph mds.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:38.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 -- 172.21.15.66:0/3831740454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718103180 msgr2=0x7f8718103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:38.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 --2- 172.21.15.66:0/3831740454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718103180 0x7f8718103600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8704009900 tx=0x7f870402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:38.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 -- 172.21.15.66:0/3831740454 shutdown_connections 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 --2- 172.21.15.66:0/3831740454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718103b40 0x7f871810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 --2- 172.21.15.66:0/3831740454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718103180 0x7f8718103600 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 --2- 172.21.15.66:0/3831740454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718101f90 0x7f87181023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 -- 172.21.15.66:0/3831740454 >> 172.21.15.66:0/3831740454 conn(0x7f87180fd500 msgr2=0x7f87180ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 -- 172.21.15.66:0/3831740454 shutdown_connections 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.492+0000 7f871efa4640 1 -- 172.21.15.66:0/3831740454 wait complete. 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 Processor -- start 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 -- start start 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718101f90 0x7f87181981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718103180 0x7f87181986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:38.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718103b40 0x7f871819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f871810dad0 con 0x7f8718101f90 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f871810d950 con 0x7f8718103b40 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871efa4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f871810dc50 con 0x7f8718103180 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871cd40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718101f90 0x7f87181981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871cd40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718101f90 0x7f87181981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47708/0 (socket says 172.21.15.66:47708) 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871cd40640 1 -- 172.21.15.66:0/526453531 learned_addr learned my addr 172.21.15.66:0/526453531 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871d541640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718103b40 0x7f871819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871cd40640 1 -- 172.21.15.66:0/526453531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718103180 msgr2=0x7f87181986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.493+0000 7f871cd40640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718103180 0x7f87181986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f871cd40640 1 -- 172.21.15.66:0/526453531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718103b40 msgr2=0x7f871819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:38.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f871cd40640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718103b40 0x7f871819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f871cd40640 1 -- 172.21.15.66:0/526453531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871819fdd0 con 0x7f8718101f90 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f871cd40640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718101f90 0x7f87181981b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c970 tx=0x7f870c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f8715ffb640 1 -- 172.21.15.66:0/526453531 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c004220 con 0x7f8718101f90 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f8715ffb640 1 -- 172.21.15.66:0/526453531 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f870c0043c0 con 0x7f8718101f90 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181a00c0 con 0x7f8718101f90 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f8715ffb640 1 -- 172.21.15.66:0/526453531 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c0047c0 con 0x7f8718101f90 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.494+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181a0610 con 0x7f8718101f90 2024-04-21T09:05:38.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.495+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8718109330 con 0x7f8718101f90 2024-04-21T09:05:38.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.496+0000 7f8715ffb640 1 -- 172.21.15.66:0/526453531 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f870c0079b0 con 0x7f8718101f90 2024-04-21T09:05:38.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.497+0000 7f8715ffb640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87080a3c30 0x7f87080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:38.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.497+0000 7f8717fff640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87080a3c30 0x7f87080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:38.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.497+0000 7f8715ffb640 1 -- 172.21.15.66:0/526453531 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f870c0d0040 con 0x7f8718101f90 2024-04-21T09:05:38.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.497+0000 7f8717fff640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87080a3c30 0x7f87080a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8704036040 tx=0x7f8704000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:38.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.500+0000 7f8715ffb640 1 -- 172.21.15.66:0/526453531 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f870c014030 con 0x7f8718101f90 2024-04-21T09:05:38.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.724+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f871802d850 con 0x7f8718101f90 2024-04-21T09:05:38.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.725+0000 7f8715ffb640 1 -- 172.21.15.66:0/526453531 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f870c09cf40 con 0x7f8718101f90 2024-04-21T09:05:38.726 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:38.726 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T09:05:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 12 2024-04-21T09:05:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87080a3c30 msgr2=0x7f87080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87080a3c30 0x7f87080a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8704036040 tx=0x7f8704000bc0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718101f90 msgr2=0x7f87181981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718101f90 0x7f87181981b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c970 tx=0x7f870c00cd70 comp rx=0 tx=0).stop 2024-04-21T09:05:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 shutdown_connections 2024-04-21T09:05:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87080a3c30 0x7f87080a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718103b40 0x7f871819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718103180 0x7f87181986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 --2- 172.21.15.66:0/526453531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718101f90 0x7f87181981b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:38.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 >> 172.21.15.66:0/526453531 conn(0x7f87180fd500 msgr2=0x7f8718106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:38.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 shutdown_connections 2024-04-21T09:05:38.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:38.727+0000 7f871efa4640 1 -- 172.21.15.66:0/526453531 wait complete. 2024-04-21T09:05:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.66:6840/1874916254,v1:172.21.15.66:6841/1874916254] up:boot 2024-04-21T09:05:39.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: fsmap 11 up:standby 2024-04-21T09:05:39.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T09:05:39.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:39 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/526453531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.66:6840/1874916254,v1:172.21.15.66:6841/1874916254] up:boot 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: fsmap 11 up:standby 2024-04-21T09:05:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T09:05:39.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:39 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/526453531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:39.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.66:6840/1874916254,v1:172.21.15.66:6841/1874916254] up:boot 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: fsmap 11 up:standby 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T09:05:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:39 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/526453531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:40.372 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:40 smithi083 ceph-mon[31413]: Deploying daemon mds.b on smithi083 2024-04-21T09:05:40.372 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:40 smithi083 ceph-mon[31413]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:40.372 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 systemd[1]: Starting Ceph mds.b for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:05:40.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:40 smithi186 ceph-mon[31505]: Deploying daemon mds.b on smithi083 2024-04-21T09:05:40.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:40 smithi186 ceph-mon[31505]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:40.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:40 smithi066 ceph-mon[27953]: Deploying daemon mds.b on smithi083 2024-04-21T09:05:40.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:40 smithi066 ceph-mon[27953]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:40.740 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 -- 172.21.15.66:0/2958700050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64101f80 msgr2=0x7f8e641023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2958700050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64101f80 0x7f8e641023a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e5802bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 -- 172.21.15.66:0/2958700050 shutdown_connections 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2958700050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e64103b30 0x7f8e6410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2958700050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e64103170 0x7f8e641035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2958700050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64101f80 0x7f8e641023a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 -- 172.21.15.66:0/2958700050 >> 172.21.15.66:0/2958700050 conn(0x7f8e640fd530 msgr2=0x7f8e640ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 -- 172.21.15.66:0/2958700050 shutdown_connections 2024-04-21T09:05:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 -- 172.21.15.66:0/2958700050 wait complete. 2024-04-21T09:05:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.854+0000 7f8e6923f640 1 Processor -- start 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6923f640 1 -- start start 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6923f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e64101f80 0x7f8e64075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6923f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e64103170 0x7f8e64074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6923f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64103b30 0x7f8e64074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6923f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410db00 con 0x7f8e64103b30 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6923f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410d980 con 0x7f8e64103170 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6923f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410dc80 con 0x7f8e64101f80 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64103b30 0x7f8e64074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64103b30 0x7f8e64074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47726/0 (socket says 172.21.15.66:47726) 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 -- 172.21.15.66:0/2774323291 learned_addr learned my addr 172.21.15.66:0/2774323291 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6259c640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e64103170 0x7f8e64074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:40.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e62d9d640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e64101f80 0x7f8e64075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 -- 172.21.15.66:0/2774323291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e64101f80 msgr2=0x7f8e64075cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e64101f80 0x7f8e64075cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 -- 172.21.15.66:0/2774323291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e64103170 msgr2=0x7f8e64074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e64103170 0x7f8e64074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6359e640 1 -- 172.21.15.66:0/2774323291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e64075140 con 0x7f8e64103b30 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e62d9d640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e64101f80 0x7f8e64075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.855+0000 7f8e6259c640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e64103170 0x7f8e64074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.856+0000 7f8e6359e640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64103b30 0x7f8e64074880 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009fd0 tx=0x7f8e5c00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.856+0000 7f8e4bfff640 1 -- 172.21.15.66:0/2774323291 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c017070 con 0x7f8e64103b30 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.856+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e641a23b0 con 0x7f8e64103b30 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.856+0000 7f8e4bfff640 1 -- 172.21.15.66:0/2774323291 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e5c003e20 con 0x7f8e64103b30 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.856+0000 7f8e4bfff640 1 -- 172.21.15.66:0/2774323291 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c010460 con 0x7f8e64103b30 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.856+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e641a2880 con 0x7f8e64103b30 2024-04-21T09:05:40.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.857+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e641093c0 con 0x7f8e64103b30 2024-04-21T09:05:40.860 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 podman[62441]: 2024-04-21 09:05:40.49277086 +0000 UTC m=+0.018026597 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:05:40.860 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 podman[62441]: 2024-04-21 09:05:40.618295549 +0000 UTC m=+0.143551320 container create 9079479c6bb07150c990bb349f4dcd7bdd6e7ee81f22fd8c768a363222a076c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:05:40.860 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 podman[62441]: 2024-04-21 09:05:40.837554503 +0000 UTC m=+0.362810228 container init 9079479c6bb07150c990bb349f4dcd7bdd6e7ee81f22fd8c768a363222a076c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-21T09:05:40.860 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 podman[62441]: 2024-04-21 09:05:40.840046336 +0000 UTC m=+0.365302063 container start 9079479c6bb07150c990bb349f4dcd7bdd6e7ee81f22fd8c768a363222a076c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-21T09:05:40.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.858+0000 7f8e4bfff640 1 -- 172.21.15.66:0/2774323291 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e5c004020 con 0x7f8e64103b30 2024-04-21T09:05:40.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.859+0000 7f8e4bfff640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8e4c0a3bb0 0x7f8e4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:40.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.859+0000 7f8e62d9d640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8e4c0a3bb0 0x7f8e4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:40.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.859+0000 7f8e4bfff640 1 -- 172.21.15.66:0/2774323291 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8e5c0cfcf0 con 0x7f8e64103b30 2024-04-21T09:05:40.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.859+0000 7f8e62d9d640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8e4c0a3bb0 0x7f8e4c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006010 tx=0x7f8e5802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:40.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:40.863+0000 7f8e4bfff640 1 -- 172.21.15.66:0/2774323291 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e5c09cb70 con 0x7f8e64103b30 2024-04-21T09:05:41.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.096+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e6402d5c0 con 0x7f8e64103b30 2024-04-21T09:05:41.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.097+0000 7f8e4bfff640 1 -- 172.21.15.66:0/2774323291 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f8e5c09c470 con 0x7f8e64103b30 2024-04-21T09:05:41.097 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:41.098 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T09:05:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 12 2024-04-21T09:05:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8e4c0a3bb0 msgr2=0x7f8e4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8e4c0a3bb0 0x7f8e4c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006010 tx=0x7f8e5802c7e0 comp rx=0 tx=0).stop 2024-04-21T09:05:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64103b30 msgr2=0x7f8e64074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64103b30 0x7f8e64074880 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009fd0 tx=0x7f8e5c00def0 comp rx=0 tx=0).stop 2024-04-21T09:05:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 shutdown_connections 2024-04-21T09:05:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8e4c0a3bb0 0x7f8e4c0a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:41.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e64103b30 0x7f8e64074880 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:41.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e64103170 0x7f8e64074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:41.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 --2- 172.21.15.66:0/2774323291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e64101f80 0x7f8e64075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:41.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 >> 172.21.15.66:0/2774323291 conn(0x7f8e640fd530 msgr2=0x7f8e64106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:41.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 shutdown_connections 2024-04-21T09:05:41.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:41.099+0000 7f8e6923f640 1 -- 172.21.15.66:0/2774323291 wait complete. 2024-04-21T09:05:41.359 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 ceph-mds[62459]: -- 172.21.15.83:0/1884877023 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1d8dcba40 con 0x55e1d8d9ec00 2024-04-21T09:05:41.360 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: starting mds.b at 2024-04-21T09:05:41.360 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 bash[62441]: 9079479c6bb07150c990bb349f4dcd7bdd6e7ee81f22fd8c768a363222a076c4 2024-04-21T09:05:41.360 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:40 smithi083 systemd[1]: Started Ceph mds.b for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:05:42.267 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:42.267 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.267 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.267 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.267 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.268 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.268 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2774323291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:42.268 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] up:boot 2024-04-21T09:05:42.268 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: fsmap 12 up:standby 2024-04-21T09:05:42.268 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:05:42.268 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:41 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:42.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:42.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2774323291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] up:boot 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: fsmap 12 up:standby 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:05:42.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:41 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2774323291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] up:boot 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: fsmap 12 up:standby 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:05:42.299 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:41 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:43.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:43.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 -- 172.21.15.66:0/3125333479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 msgr2=0x7f96641092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 --2- 172.21.15.66:0/3125333479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f96641092a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f965c009ca0 tx=0x7f965c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 -- 172.21.15.66:0/3125333479 shutdown_connections 2024-04-21T09:05:43.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 --2- 172.21.15.66:0/3125333479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f96641092a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 --2- 172.21.15.66:0/3125333479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9664101fc0 0x7f9664102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 --2- 172.21.15.66:0/3125333479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9664100e70 0x7f9664101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 -- 172.21.15.66:0/3125333479 >> 172.21.15.66:0/3125333479 conn(0x7f96640fc3e0 msgr2=0x7f96640fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 -- 172.21.15.66:0/3125333479 shutdown_connections 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.215+0000 7f966c523640 1 -- 172.21.15.66:0/3125333479 wait complete. 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 Processor -- start 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 -- start start 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9664100e70 0x7f9664069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9664101fc0 0x7f966419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f966419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f966410c9a0 con 0x7f9664102980 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f966410c820 con 0x7f9664101fc0 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966c523640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f966410cb20 con 0x7f9664100e70 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f966419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f966419f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47744/0 (socket says 172.21.15.66:47744) 2024-04-21T09:05:43.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 -- 172.21.15.66:0/3840226050 learned_addr learned my addr 172.21.15.66:0/3840226050 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966a2bf640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9664100e70 0x7f9664069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f9669abe640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9664101fc0 0x7f966419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 -- 172.21.15.66:0/3840226050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9664100e70 msgr2=0x7f9664069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9664100e70 0x7f9664069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 -- 172.21.15.66:0/3840226050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9664101fc0 msgr2=0x7f966419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9664101fc0 0x7f966419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966aac0640 1 -- 172.21.15.66:0/3840226050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966419fd90 con 0x7f9664102980 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f9669abe640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9664101fc0 0x7f966419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.216+0000 7f966a2bf640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9664100e70 0x7f9664069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.217+0000 7f966aac0640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f966419f480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f965c02c700 tx=0x7f965c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.217+0000 7f965b7fe640 1 -- 172.21.15.66:0/3840226050 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c041020 con 0x7f9664102980 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.217+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966419ffc0 con 0x7f9664102980 2024-04-21T09:05:43.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.217+0000 7f965b7fe640 1 -- 172.21.15.66:0/3840226050 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f965c034d60 con 0x7f9664102980 2024-04-21T09:05:43.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.217+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96641a0480 con 0x7f9664102980 2024-04-21T09:05:43.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.217+0000 7f965b7fe640 1 -- 172.21.15.66:0/3840226050 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c039040 con 0x7f9664102980 2024-04-21T09:05:43.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.218+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9664108180 con 0x7f9664102980 2024-04-21T09:05:43.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.220+0000 7f965b7fe640 1 -- 172.21.15.66:0/3840226050 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f965c044020 con 0x7f9664102980 2024-04-21T09:05:43.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.220+0000 7f965b7fe640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96500a3bb0 0x7f96500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.221+0000 7f966a2bf640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96500a3bb0 0x7f96500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.221+0000 7f965b7fe640 1 -- 172.21.15.66:0/3840226050 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f965c0f23a0 con 0x7f9664102980 2024-04-21T09:05:43.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.221+0000 7f966a2bf640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96500a3bb0 0x7f96500a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9654005d00 tx=0x7f9654005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:43.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.224+0000 7f965b7fe640 1 -- 172.21.15.66:0/3840226050 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f965c04b020 con 0x7f9664102980 2024-04-21T09:05:43.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.469+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f966402d5c0 con 0x7f9664102980 2024-04-21T09:05:43.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.470+0000 7f965b7fe640 1 -- 172.21.15.66:0/3840226050 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f965c0bf220 con 0x7f9664102980 2024-04-21T09:05:43.471 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:43.471 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-21T09:05:43.471 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 13 2024-04-21T09:05:43.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96500a3bb0 msgr2=0x7f96500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96500a3bb0 0x7f96500a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9654005d00 tx=0x7f9654005c70 comp rx=0 tx=0).stop 2024-04-21T09:05:43.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 msgr2=0x7f966419f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f966419f480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f965c02c700 tx=0x7f965c0093f0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 shutdown_connections 2024-04-21T09:05:43.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f96500a3bb0 0x7f96500a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664102980 0x7f966419f480 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9664101fc0 0x7f966419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 --2- 172.21.15.66:0/3840226050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9664100e70 0x7f9664069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 >> 172.21.15.66:0/3840226050 conn(0x7f96640fc3e0 msgr2=0x7f9664105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.472+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 shutdown_connections 2024-04-21T09:05:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.473+0000 7f966c523640 1 -- 172.21.15.66:0/3840226050 wait complete. 2024-04-21T09:05:43.484 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 -- 172.21.15.66:0/4021058960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 msgr2=0x7fe3c4104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 --2- 172.21.15.66:0/4021058960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c4104e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009900 tx=0x7fe3b402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 -- 172.21.15.66:0/4021058960 shutdown_connections 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 --2- 172.21.15.66:0/4021058960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3c4107cb0 0x7fe3c410a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 --2- 172.21.15.66:0/4021058960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3c4105380 0x7fe3c4107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 --2- 172.21.15.66:0/4021058960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c4104e40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 -- 172.21.15.66:0/4021058960 >> 172.21.15.66:0/4021058960 conn(0x7fe3c40fc3e0 msgr2=0x7fe3c40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 -- 172.21.15.66:0/4021058960 shutdown_connections 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.589+0000 7fe3c9255640 1 -- 172.21.15.66:0/4021058960 wait complete. 2024-04-21T09:05:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 Processor -- start 2024-04-21T09:05:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 -- start start 2024-04-21T09:05:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c41023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3c4105380 0x7fe3c4100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3c4107cb0 0x7fe3c4100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c410daf0 con 0x7fe3c4102a50 2024-04-21T09:05:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c410d970 con 0x7fe3c4105380 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c9255640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c410dc70 con 0x7fe3c4107cb0 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c41023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c41023c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47768/0 (socket says 172.21.15.66:47768) 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c2d9d640 1 -- 172.21.15.66:0/3958567156 learned_addr learned my addr 172.21.15.66:0/3958567156 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c259c640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3c4105380 0x7fe3c4100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c359e640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3c4107cb0 0x7fe3c4100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c2d9d640 1 -- 172.21.15.66:0/3958567156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3c4107cb0 msgr2=0x7fe3c4100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c2d9d640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3c4107cb0 0x7fe3c4100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c2d9d640 1 -- 172.21.15.66:0/3958567156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3c4105380 msgr2=0x7fe3c4100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.590+0000 7fe3c2d9d640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3c4105380 0x7fe3c4100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3c2d9d640 1 -- 172.21.15.66:0/3958567156 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c41017d0 con 0x7fe3c4102a50 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3c259c640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3c4105380 0x7fe3c4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:05:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3c2d9d640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c41023c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000c00 tx=0x7fe3b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3abfff640 1 -- 172.21.15.66:0/3958567156 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4043020 con 0x7fe3c4102a50 2024-04-21T09:05:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3abfff640 1 -- 172.21.15.66:0/3958567156 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b4004aa0 con 0x7fe3c4102a50 2024-04-21T09:05:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c41a2310 con 0x7fe3c4102a50 2024-04-21T09:05:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3abfff640 1 -- 172.21.15.66:0/3958567156 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4003d70 con 0x7fe3c4102a50 2024-04-21T09:05:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.591+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a2860 con 0x7fe3c4102a50 2024-04-21T09:05:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.592+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c40fded0 con 0x7fe3c4102a50 2024-04-21T09:05:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.593+0000 7fe3abfff640 1 -- 172.21.15.66:0/3958567156 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe3b4046020 con 0x7fe3c4102a50 2024-04-21T09:05:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.594+0000 7fe3abfff640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3ac0a3bb0 0x7fe3ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.594+0000 7fe3c259c640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3ac0a3bb0 0x7fe3ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.594+0000 7fe3abfff640 1 -- 172.21.15.66:0/3958567156 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe3b40f22c0 con 0x7fe3c4102a50 2024-04-21T09:05:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.594+0000 7fe3c259c640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3ac0a3bb0 0x7fe3ac0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4102130 tx=0x7fe3b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:43.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.601+0000 7fe3abfff640 1 -- 172.21.15.66:0/3958567156 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3b404d020 con 0x7fe3c4102a50 2024-04-21T09:05:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.789+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3c4062b50 con 0x7fe3c4102a50 2024-04-21T09:05:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.790+0000 7fe3abfff640 1 -- 172.21.15.66:0/3958567156 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18418 (secure 0 0 0) 0x7fe3b40bf0a0 con 0x7fe3c4102a50 2024-04-21T09:05:43.791 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:43.791 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:04:57.408258+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:05:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3ac0a3bb0 msgr2=0x7fe3ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3ac0a3bb0 0x7fe3ac0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4102130 tx=0x7fe3b8005f90 comp rx=0 tx=0).stop 2024-04-21T09:05:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 msgr2=0x7fe3c41023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c41023c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000c00 tx=0x7fe3b40042f0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 shutdown_connections 2024-04-21T09:05:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3ac0a3bb0 0x7fe3ac0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3c4107cb0 0x7fe3c4100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3c4105380 0x7fe3c4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 --2- 172.21.15.66:0/3958567156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c4102a50 0x7fe3c41023c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.792+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 >> 172.21.15.66:0/3958567156 conn(0x7fe3c40fc3e0 msgr2=0x7fe3c41032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.793+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 shutdown_connections 2024-04-21T09:05:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.793+0000 7fe3c9255640 1 -- 172.21.15.66:0/3958567156 wait complete. 2024-04-21T09:05:43.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T09:05:43.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:05:43.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.912+0000 7fd876632640 1 -- 172.21.15.66:0/2734449859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8701029a0 msgr2=0x7fd8701092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.912+0000 7fd876632640 1 --2- 172.21.15.66:0/2734449859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8701029a0 0x7fd8701092c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd868009ca0 tx=0x7fd86802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.912+0000 7fd876632640 1 -- 172.21.15.66:0/2734449859 shutdown_connections 2024-04-21T09:05:43.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.912+0000 7fd876632640 1 --2- 172.21.15.66:0/2734449859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8701029a0 0x7fd8701092c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.912+0000 7fd876632640 1 --2- 172.21.15.66:0/2734449859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd870101fe0 0x7fd870102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.912+0000 7fd876632640 1 --2- 172.21.15.66:0/2734449859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870100e90 0x7fd8701012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.912+0000 7fd876632640 1 -- 172.21.15.66:0/2734449859 >> 172.21.15.66:0/2734449859 conn(0x7fd8700fc400 msgr2=0x7fd8700fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.913+0000 7fd876632640 1 -- 172.21.15.66:0/2734449859 shutdown_connections 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.913+0000 7fd876632640 1 -- 172.21.15.66:0/2734449859 wait complete. 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.913+0000 7fd876632640 1 Processor -- start 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.913+0000 7fd876632640 1 -- start start 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.913+0000 7fd876632640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd870100e90 0x7fd870104f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd876632640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd870101fe0 0x7fd870103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd876632640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8701029a0 0x7fd870103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd876632640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd87010c9f0 con 0x7fd870100e90 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd876632640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd87010c870 con 0x7fd870101fe0 2024-04-21T09:05:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd876632640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd87010cb70 con 0x7fd8701029a0 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd870100e90 0x7fd870104f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd870100e90 0x7fd870104f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47792/0 (socket says 172.21.15.66:47792) 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 -- 172.21.15.66:0/1483944709 learned_addr learned my addr 172.21.15.66:0/1483944709 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86f7fe640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd870101fe0 0x7fd870103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd874bcf640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8701029a0 0x7fd870103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 -- 172.21.15.66:0/1483944709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8701029a0 msgr2=0x7fd870103a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8701029a0 0x7fd870103a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 -- 172.21.15.66:0/1483944709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd870101fe0 msgr2=0x7fd870103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd870101fe0 0x7fd870103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 -- 172.21.15.66:0/1483944709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd870104310 con 0x7fd870100e90 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86ffff640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd870100e90 0x7fd870104f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd86000d770 tx=0x7fd86000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86d7fa640 1 -- 172.21.15.66:0/1483944709 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860017070 con 0x7fd870100e90 2024-04-21T09:05:43.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8701a2320 con 0x7fd870100e90 2024-04-21T09:05:43.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86d7fa640 1 -- 172.21.15.66:0/1483944709 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8600054d0 con 0x7fd870100e90 2024-04-21T09:05:43.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd86d7fa640 1 -- 172.21.15.66:0/1483944709 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860004c30 con 0x7fd870100e90 2024-04-21T09:05:43.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.914+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8701a2820 con 0x7fd870100e90 2024-04-21T09:05:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.915+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8701a2ae0 con 0x7fd870100e90 2024-04-21T09:05:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.917+0000 7fd86d7fa640 1 -- 172.21.15.66:0/1483944709 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd860004e50 con 0x7fd870100e90 2024-04-21T09:05:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.917+0000 7fd86d7fa640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd8580a3bb0 0x7fd8580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.917+0000 7fd86f7fe640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd8580a3bb0 0x7fd8580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.917+0000 7fd86d7fa640 1 -- 172.21.15.66:0/1483944709 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd8600cf990 con 0x7fd870100e90 2024-04-21T09:05:43.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.918+0000 7fd86f7fe640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd8580a3bb0 0x7fd8580a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd870104c70 tx=0x7fd86400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:43.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:43.922+0000 7fd86d7fa640 1 -- 172.21.15.66:0/1483944709 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd86009c1c0 con 0x7fd870100e90 2024-04-21T09:05:44.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.112+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8701a2cb0 con 0x7fd870100e90 2024-04-21T09:05:44.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.113+0000 7fd86d7fa640 1 -- 172.21.15.66:0/1483944709 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fd8600a04c0 con 0x7fd870100e90 2024-04-21T09:05:44.113 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:05:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd8580a3bb0 msgr2=0x7fd8580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd8580a3bb0 0x7fd8580a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd870104c70 tx=0x7fd86400b390 comp rx=0 tx=0).stop 2024-04-21T09:05:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd870100e90 msgr2=0x7fd870104f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd870100e90 0x7fd870104f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd86000d770 tx=0x7fd86000dc30 comp rx=0 tx=0).stop 2024-04-21T09:05:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 shutdown_connections 2024-04-21T09:05:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd8580a3bb0 0x7fd8580a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8701029a0 0x7fd870103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd870101fe0 0x7fd870103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 --2- 172.21.15.66:0/1483944709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd870100e90 0x7fd870104f00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 >> 172.21.15.66:0/1483944709 conn(0x7fd8700fc400 msgr2=0x7fd8700690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 shutdown_connections 2024-04-21T09:05:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.115+0000 7fd876632640 1 -- 172.21.15.66:0/1483944709 wait complete. 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3840226050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:43 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3958567156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.127 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3806139072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4107d30 msgr2=0x7f6fe410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3806139072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4107d30 0x7f6fe410a120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009ca0 tx=0x7f6fdc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3806139072 shutdown_connections 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3806139072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4107d30 0x7f6fe410a120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3806139072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4105400 0x7f6fe41077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3806139072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fe4102ad0 0x7f6fe4104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3806139072 >> 172.21.15.66:0/3806139072 conn(0x7f6fe40fc400 msgr2=0x7f6fe40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3806139072 shutdown_connections 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.229+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3806139072 wait complete. 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 Processor -- start 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 -- start start 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fe4102ad0 0x7f6fe41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4105400 0x7f6fe4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4107d30 0x7f6fe419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410dae0 con 0x7f6fe4105400 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410d960 con 0x7f6fe4107d30 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6feb9a4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410dc60 con 0x7f6fe4102ad0 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4105400 0x7f6fe4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4105400 0x7f6fe4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47804/0 (socket says 172.21.15.66:47804) 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 -- 172.21.15.66:0/3710807973 learned_addr learned my addr 172.21.15.66:0/3710807973 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe9f41640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4107d30 0x7f6fe419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe9740640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fe4102ad0 0x7f6fe41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 -- 172.21.15.66:0/3710807973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fe4102ad0 msgr2=0x7f6fe41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fe4102ad0 0x7f6fe41981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 -- 172.21.15.66:0/3710807973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4107d30 msgr2=0x7f6fe419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4107d30 0x7f6fe419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.230+0000 7f6fe8f3f640 1 -- 172.21.15.66:0/3710807973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe419fe00 con 0x7f6fe4105400 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.231+0000 7f6fe8f3f640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4105400 0x7f6fe4198720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0005fd0 tx=0x7f6fe000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.231+0000 7f6fda7fc640 1 -- 172.21.15.66:0/3710807973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0017070 con 0x7f6fe4105400 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.231+0000 7f6fda7fc640 1 -- 172.21.15.66:0/3710807973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe0009c30 con 0x7f6fe4105400 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.231+0000 7f6fda7fc640 1 -- 172.21.15.66:0/3710807973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0003d30 con 0x7f6fe4105400 2024-04-21T09:05:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.231+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe41a00f0 con 0x7f6fe4105400 2024-04-21T09:05:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.231+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41a0640 con 0x7f6fe4105400 2024-04-21T09:05:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.232+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe40fdef0 con 0x7f6fe4105400 2024-04-21T09:05:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.233+0000 7f6fda7fc640 1 -- 172.21.15.66:0/3710807973 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6fe0003ed0 con 0x7f6fe4105400 2024-04-21T09:05:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.234+0000 7f6fda7fc640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6fd00a3bb0 0x7f6fd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.234+0000 7f6fe9740640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6fd00a3bb0 0x7f6fd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.234+0000 7f6fda7fc640 1 -- 172.21.15.66:0/3710807973 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6fe00cfb50 con 0x7f6fe4105400 2024-04-21T09:05:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.234+0000 7f6fe9740640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6fd00a3bb0 0x7f6fd00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009bd0 tx=0x7f6fd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.239+0000 7f6fda7fc640 1 -- 172.21.15.66:0/3710807973 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fe0020030 con 0x7f6fe4105400 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3840226050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:44.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:44.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:43 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3958567156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:44.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3840226050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:43 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3958567156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.444+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6fe402d770 con 0x7f6fe4105400 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.445+0000 7f6fda7fc640 1 -- 172.21.15.66:0/3710807973 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f6fe0009de0 con 0x7f6fe4105400 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-04-21T09:05:44.445 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:05:44.446 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-04-21T09:05:44.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6fd00a3bb0 msgr2=0x7f6fd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6fd00a3bb0 0x7f6fd00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009bd0 tx=0x7f6fd4005f90 comp rx=0 tx=0).stop 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4105400 msgr2=0x7f6fe4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4105400 0x7f6fe4198720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0005fd0 tx=0x7f6fe000dde0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 shutdown_connections 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6fd00a3bb0 0x7f6fd00a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4107d30 0x7f6fe419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fe4105400 0x7f6fe4198720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 --2- 172.21.15.66:0/3710807973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fe4102ad0 0x7f6fe41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 >> 172.21.15.66:0/3710807973 conn(0x7f6fe40fc400 msgr2=0x7f6fe4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 shutdown_connections 2024-04-21T09:05:44.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.447+0000 7f6feb9a4640 1 -- 172.21.15.66:0/3710807973 wait complete. 2024-04-21T09:05:44.458 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-21T09:05:44.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-21T09:05:44.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 -- 172.21.15.66:0/1415726745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 msgr2=0x7f940c107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 --2- 172.21.15.66:0/1415726745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c107790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:44.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 -- 172.21.15.66:0/1415726745 shutdown_connections 2024-04-21T09:05:44.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 --2- 172.21.15.66:0/1415726745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f940c107cd0 0x7f940c10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 --2- 172.21.15.66:0/1415726745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c107790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 --2- 172.21.15.66:0/1415726745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940c102a70 0x7f940c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 -- 172.21.15.66:0/1415726745 >> 172.21.15.66:0/1415726745 conn(0x7f940c0fc400 msgr2=0x7f940c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 -- 172.21.15.66:0/1415726745 shutdown_connections 2024-04-21T09:05:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.563+0000 7f94117b6640 1 -- 172.21.15.66:0/1415726745 wait complete. 2024-04-21T09:05:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 Processor -- start 2024-04-21T09:05:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 -- start start 2024-04-21T09:05:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f940c102a70 0x7f940c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940c107cd0 0x7f940c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f940c10dac0 con 0x7f940c1053a0 2024-04-21T09:05:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f940c10d940 con 0x7f940c107cd0 2024-04-21T09:05:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f94117b6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f940c10dc40 con 0x7f940c102a70 2024-04-21T09:05:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f940a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f940a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47816/0 (socket says 172.21.15.66:47816) 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.564+0000 7f940a7fc640 1 -- 172.21.15.66:0/3736186525 learned_addr learned my addr 172.21.15.66:0/3736186525 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940affd640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f940c102a70 0x7f940c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940b7fe640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940c107cd0 0x7f940c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940a7fc640 1 -- 172.21.15.66:0/3736186525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f940c102a70 msgr2=0x7f940c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940a7fc640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f940c102a70 0x7f940c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940a7fc640 1 -- 172.21.15.66:0/3736186525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940c107cd0 msgr2=0x7f940c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940a7fc640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940c107cd0 0x7f940c19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940a7fc640 1 -- 172.21.15.66:0/3736186525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c19fea0 con 0x7f940c1053a0 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f940a7fc640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c1987c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9400000c00 tx=0x7f9400004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f93f3fff640 1 -- 172.21.15.66:0/3736186525 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400043020 con 0x7f940c1053a0 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940c1a0130 con 0x7f940c1053a0 2024-04-21T09:05:44.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f93f3fff640 1 -- 172.21.15.66:0/3736186525 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9400004440 con 0x7f940c1053a0 2024-04-21T09:05:44.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f93f3fff640 1 -- 172.21.15.66:0/3736186525 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400003c90 con 0x7f940c1053a0 2024-04-21T09:05:44.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.565+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940c1a0680 con 0x7f940c1053a0 2024-04-21T09:05:44.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.566+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940c0fdef0 con 0x7f940c1053a0 2024-04-21T09:05:44.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.567+0000 7f93f3fff640 1 -- 172.21.15.66:0/3736186525 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9400046020 con 0x7f940c1053a0 2024-04-21T09:05:44.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.568+0000 7f93f3fff640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3bb0 0x7f93f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:44.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.568+0000 7f940affd640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3bb0 0x7f93f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:44.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.568+0000 7f93f3fff640 1 -- 172.21.15.66:0/3736186525 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f94000f20d0 con 0x7f940c1053a0 2024-04-21T09:05:44.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.568+0000 7f940affd640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3bb0 0x7f93f40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009bd0 tx=0x7f93fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:44.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.572+0000 7f93f3fff640 1 -- 172.21.15.66:0/3736186525 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f940004d020 con 0x7f940c1053a0 2024-04-21T09:05:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.765+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f940c02d810 con 0x7f940c1053a0 2024-04-21T09:05:44.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.939+0000 7f93f3fff640 1 -- 172.21.15.66:0/3736186525 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f94000bef50 con 0x7f940c1053a0 2024-04-21T09:05:44.940 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3bb0 msgr2=0x7f93f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3bb0 0x7f93f40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009bd0 tx=0x7f93fc005f90 comp rx=0 tx=0).stop 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 msgr2=0x7f940c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c1987c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9400000c00 tx=0x7f9400004040 comp rx=0 tx=0).stop 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 shutdown_connections 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3bb0 0x7f93f40a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940c107cd0 0x7f940c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940c1053a0 0x7f940c1987c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 --2- 172.21.15.66:0/3736186525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f940c102a70 0x7f940c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:44.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 >> 172.21.15.66:0/3736186525 conn(0x7f940c0fc400 msgr2=0x7f940c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:44.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 shutdown_connections 2024-04-21T09:05:44.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:44.942+0000 7f94117b6640 1 -- 172.21.15.66:0/3736186525 wait complete. 2024-04-21T09:05:44.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-21T09:05:45.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 -- 172.21.15.66:0/2348447644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103b50 msgr2=0x7fc4d810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/2348447644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103b50 0x7fc4d810a3e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009ca0 tx=0x7fc4d002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 -- 172.21.15.66:0/2348447644 shutdown_connections 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/2348447644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103b50 0x7fc4d810a3e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/2348447644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8103190 0x7fc4d8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/2348447644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d8101fa0 0x7fc4d81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 -- 172.21.15.66:0/2348447644 >> 172.21.15.66:0/2348447644 conn(0x7fc4d80fd530 msgr2=0x7fc4d80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 -- 172.21.15.66:0/2348447644 shutdown_connections 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 -- 172.21.15.66:0/2348447644 wait complete. 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 Processor -- start 2024-04-21T09:05:45.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.069+0000 7fc4de9b6640 1 -- start start 2024-04-21T09:05:45.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4de9b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8101fa0 0x7fc4d8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4de9b6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103190 0x7fc4d81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4de9b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d8103b50 0x7fc4d819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4de9b6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d810dae0 con 0x7fc4d8103190 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4de9b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d810d960 con 0x7fc4d8103b50 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4de9b6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d810dc60 con 0x7fc4d8101fa0 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103190 0x7fc4d81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8101fa0 0x7fc4d8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103190 0x7fc4d81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36566/0 (socket says 172.21.15.66:36566) 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 -- 172.21.15.66:0/411406429 learned_addr learned my addr 172.21.15.66:0/411406429 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8101fa0 0x7fc4d8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:37036/0 (socket says 172.21.15.66:37036) 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4dcf53640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d8103b50 0x7fc4d819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 -- 172.21.15.66:0/411406429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8101fa0 msgr2=0x7fc4d8198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:45.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8101fa0 0x7fc4d8198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 -- 172.21.15.66:0/411406429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d8103b50 msgr2=0x7fc4d819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d8103b50 0x7fc4d819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d77fe640 1 -- 172.21.15.66:0/411406429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d819fda0 con 0x7fc4d8103190 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4dcf53640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d8103b50 0x7fc4d819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.070+0000 7fc4d7fff640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8101fa0 0x7fc4d8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.071+0000 7fc4d77fe640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103190 0x7fc4d81986c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00e9f0 tx=0x7fc4cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.071+0000 7fc4d57fa640 1 -- 172.21.15.66:0/411406429 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc00cd80 con 0x7fc4d8103190 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.071+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d81a0090 con 0x7fc4d8103190 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.071+0000 7fc4d57fa640 1 -- 172.21.15.66:0/411406429 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4cc0054d0 con 0x7fc4d8103190 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.071+0000 7fc4d57fa640 1 -- 172.21.15.66:0/411406429 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc010690 con 0x7fc4d8103190 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.071+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d81a05e0 con 0x7fc4d8103190 2024-04-21T09:05:45.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.072+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d81092c0 con 0x7fc4d8103190 2024-04-21T09:05:45.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.074+0000 7fc4d57fa640 1 -- 172.21.15.66:0/411406429 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc4cc004e50 con 0x7fc4d8103190 2024-04-21T09:05:45.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.075+0000 7fc4d57fa640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc4c00a3bb0 0x7fc4c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:45.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.075+0000 7fc4d7fff640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc4c00a3bb0 0x7fc4c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:45.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.075+0000 7fc4d57fa640 1 -- 172.21.15.66:0/411406429 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fc4cc0d06f0 con 0x7fc4d8103190 2024-04-21T09:05:45.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.076+0000 7fc4d7fff640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc4c00a3bb0 0x7fc4c00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8102ff0 tx=0x7fc4c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:45.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.079+0000 7fc4d57fa640 1 -- 172.21.15.66:0/411406429 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4cc053ff0 con 0x7fc4d8103190 2024-04-21T09:05:45.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:44 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1483944709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:45.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:44 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3710807973' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T09:05:45.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:44 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3736186525' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T09:05:45.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.274+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fc4d802d5c0 con 0x7fc4d8103190 2024-04-21T09:05:45.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:44 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1483944709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:45.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:44 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3710807973' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T09:05:45.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:44 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3736186525' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T09:05:45.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:44 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1483944709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:05:45.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:44 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3710807973' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T09:05:45.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:44 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3736186525' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T09:05:45.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.947+0000 7fc4d57fa640 1 -- 172.21.15.66:0/411406429 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7fc4cc09d400 con 0x7fc4d8103190 2024-04-21T09:05:45.948 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-04-21T09:05:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc4c00a3bb0 msgr2=0x7fc4c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:45.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc4c00a3bb0 0x7fc4c00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8102ff0 tx=0x7fc4c8005f90 comp rx=0 tx=0).stop 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103190 msgr2=0x7fc4d81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103190 0x7fc4d81986c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00e9f0 tx=0x7fc4cc00edf0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 shutdown_connections 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc4c00a3bb0 0x7fc4c00a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d8103b50 0x7fc4d819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4d8103190 0x7fc4d81986c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 --2- 172.21.15.66:0/411406429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4d8101fa0 0x7fc4d8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 >> 172.21.15.66:0/411406429 conn(0x7fc4d80fd530 msgr2=0x7fc4d8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 shutdown_connections 2024-04-21T09:05:45.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:45.950+0000 7fc4de9b6640 1 -- 172.21.15.66:0/411406429 wait complete. 2024-04-21T09:05:45.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 -- 172.21.15.66:0/1741385254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414102a20 msgr2=0x7f3414104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 --2- 172.21.15.66:0/1741385254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414102a20 0x7f3414104e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3404009900 tx=0x7f340402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 -- 172.21.15.66:0/1741385254 shutdown_connections 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 --2- 172.21.15.66:0/1741385254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3414107c80 0x7f341410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 --2- 172.21.15.66:0/1741385254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3414105350 0x7f3414107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 --2- 172.21.15.66:0/1741385254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414102a20 0x7f3414104e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 -- 172.21.15.66:0/1741385254 >> 172.21.15.66:0/1741385254 conn(0x7f34140fc410 msgr2=0x7f34140fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 -- 172.21.15.66:0/1741385254 shutdown_connections 2024-04-21T09:05:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.091+0000 7f3419a31640 1 -- 172.21.15.66:0/1741385254 wait complete. 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 Processor -- start 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 -- start start 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3414102a20 0x7f3414198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414105350 0x7f3414198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3414107c80 0x7f341419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f341410dae0 con 0x7f3414105350 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f341410d960 con 0x7f3414102a20 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3419a31640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f341410dc60 con 0x7f3414107c80 2024-04-21T09:05:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f34127fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414105350 0x7f3414198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f34127fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414105350 0x7f3414198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36588/0 (socket says 172.21.15.66:36588) 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f34127fc640 1 -- 172.21.15.66:0/2822862838 learned_addr learned my addr 172.21.15.66:0/2822862838 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.092+0000 7f3412ffd640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3414102a20 0x7f3414198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34127fc640 1 -- 172.21.15.66:0/2822862838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3414107c80 msgr2=0x7f341419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34137fe640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3414107c80 0x7f341419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34127fc640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3414107c80 0x7f341419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34127fc640 1 -- 172.21.15.66:0/2822862838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3414102a20 msgr2=0x7f3414198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34127fc640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3414102a20 0x7f3414198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34127fc640 1 -- 172.21.15.66:0/2822862838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341419ff20 con 0x7f3414105350 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34137fe640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3414107c80 0x7f341419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f34127fc640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414105350 0x7f3414198840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f340800d740 tx=0x7f340800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:46.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f3418a2f640 1 -- 172.21.15.66:0/2822862838 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408017070 con 0x7f3414105350 2024-04-21T09:05:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f3418a2f640 1 -- 172.21.15.66:0/2822862838 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340800de50 con 0x7f3414105350 2024-04-21T09:05:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34141a0210 con 0x7f3414105350 2024-04-21T09:05:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f3418a2f640 1 -- 172.21.15.66:0/2822862838 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408004ce0 con 0x7f3414105350 2024-04-21T09:05:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.093+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34141a0710 con 0x7f3414105350 2024-04-21T09:05:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.094+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34140fdf00 con 0x7f3414105350 2024-04-21T09:05:46.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.095+0000 7f3418a2f640 1 -- 172.21.15.66:0/2822862838 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3408004e80 con 0x7f3414105350 2024-04-21T09:05:46.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.096+0000 7f3418a2f640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33fc0a3bb0 0x7f33fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:46.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.096+0000 7f3412ffd640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33fc0a3bb0 0x7f33fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:46.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.096+0000 7f3418a2f640 1 -- 172.21.15.66:0/2822862838 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f34080d0010 con 0x7f3414105350 2024-04-21T09:05:46.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.096+0000 7f3412ffd640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33fc0a3bb0 0x7f33fc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3404009230 tx=0x7f3404005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:46.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.100+0000 7f3418a2f640 1 -- 172.21.15.66:0/2822862838 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f340809cba0 con 0x7f3414105350 2024-04-21T09:05:46.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.328+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3414062bc0 con 0x7f3414105350 2024-04-21T09:05:46.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3736186525' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/411406429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:46.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:45 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3736186525' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/411406429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:46.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:46.361 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:45 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3736186525' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/411406429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:05:46.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:45 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:05:46.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.962+0000 7f3418a2f640 1 -- 172.21.15.66:0/2822862838 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f340809c4a0 con 0x7f3414105350 2024-04-21T09:05:46.963 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 2 and data pool 3 2024-04-21T09:05:46.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.966+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33fc0a3bb0 msgr2=0x7f33fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:46.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.966+0000 7f3419a31640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33fc0a3bb0 0x7f33fc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3404009230 tx=0x7f3404005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.966+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414105350 msgr2=0x7f3414198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:46.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.966+0000 7f3419a31640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414105350 0x7f3414198840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f340800d740 tx=0x7f340800dc00 comp rx=0 tx=0).stop 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 shutdown_connections 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33fc0a3bb0 0x7f33fc0a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3414107c80 0x7f341419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3414105350 0x7f3414198840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 --2- 172.21.15.66:0/2822862838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3414102a20 0x7f3414198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 >> 172.21.15.66:0/2822862838 conn(0x7f34140fc410 msgr2=0x7f3414103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 shutdown_connections 2024-04-21T09:05:46.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:46.967+0000 7f3419a31640 1 -- 172.21.15.66:0/2822862838 wait complete. 2024-04-21T09:05:46.988 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-21T09:05:46.989 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 -- 172.21.15.66:0/227444913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770102a00 msgr2=0x7ff770109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 --2- 172.21.15.66:0/227444913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770102a00 0x7ff770109320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009900 tx=0x7ff76c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 -- 172.21.15.66:0/227444913 shutdown_connections 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 --2- 172.21.15.66:0/227444913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770102a00 0x7ff770109320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 --2- 172.21.15.66:0/227444913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff770102040 0x7ff7701024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 --2- 172.21.15.66:0/227444913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff770100e60 0x7ff770101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 -- 172.21.15.66:0/227444913 >> 172.21.15.66:0/227444913 conn(0x7ff7700fc440 msgr2=0x7ff7700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 -- 172.21.15.66:0/227444913 shutdown_connections 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.098+0000 7ff7778a5640 1 -- 172.21.15.66:0/227444913 wait complete. 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 Processor -- start 2024-04-21T09:05:47.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 -- start start 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770100e60 0x7ff7701981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff770102040 0x7ff7701986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff770102a00 0x7ff77019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff77010c9f0 con 0x7ff770100e60 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff77010c870 con 0x7ff770102a00 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff7778a5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff77010cb70 con 0x7ff770102040 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff775641640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770100e60 0x7ff7701981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff775641640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770100e60 0x7ff7701981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36604/0 (socket says 172.21.15.66:36604) 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff775641640 1 -- 172.21.15.66:0/3177736880 learned_addr learned my addr 172.21.15.66:0/3177736880 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff775e42640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff770102a00 0x7ff77019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.099+0000 7ff775641640 1 -- 172.21.15.66:0/3177736880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff770102040 msgr2=0x7ff7701986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff775641640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff770102040 0x7ff7701986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff775641640 1 -- 172.21.15.66:0/3177736880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff770102a00 msgr2=0x7ff77019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff775641640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff770102a00 0x7ff77019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff775641640 1 -- 172.21.15.66:0/3177736880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77019fdd0 con 0x7ff770100e60 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff775e42640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff770102a00 0x7ff77019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff775641640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770100e60 0x7ff7701981b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff76800ebc0 tx=0x7ff76800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff7667fc640 1 -- 172.21.15.66:0/3177736880 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768003b70 con 0x7ff770100e60 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff7667fc640 1 -- 172.21.15.66:0/3177736880 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff768003d10 con 0x7ff770100e60 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7701a00c0 con 0x7ff770100e60 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff7667fc640 1 -- 172.21.15.66:0/3177736880 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768019800 con 0x7ff770100e60 2024-04-21T09:05:47.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.100+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7701a0610 con 0x7ff770100e60 2024-04-21T09:05:47.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.101+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7701a08d0 con 0x7ff770100e60 2024-04-21T09:05:47.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.103+0000 7ff7667fc640 1 -- 172.21.15.66:0/3177736880 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff7680199a0 con 0x7ff770100e60 2024-04-21T09:05:47.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.103+0000 7ff7667fc640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff75c0a3bb0 0x7ff75c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:47.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.103+0000 7ff774e40640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff75c0a3bb0 0x7ff75c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:47.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.103+0000 7ff7667fc640 1 -- 172.21.15.66:0/3177736880 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff7680d0d40 con 0x7ff770100e60 2024-04-21T09:05:47.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.104+0000 7ff774e40640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff75c0a3bb0 0x7ff75c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff760005fd0 tx=0x7ff760005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:47.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.107+0000 7ff7667fc640 1 -- 172.21.15.66:0/3177736880 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff76809d900 con 0x7ff770100e60 2024-04-21T09:05:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.301+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff7701a0aa0 con 0x7ff770100e60 2024-04-21T09:05:47.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:46 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/411406429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T09:05:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:46 smithi186 ceph-mon[31505]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T09:05:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:46 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2822862838' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T09:05:47.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:46 smithi186 ceph-mon[31505]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T09:05:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:46 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/411406429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T09:05:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:46 smithi083 ceph-mon[31413]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T09:05:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:46 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2822862838' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T09:05:47.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:46 smithi083 ceph-mon[31413]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T09:05:47.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:46 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/411406429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T09:05:47.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:46 smithi066 ceph-mon[27953]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T09:05:47.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:46 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2822862838' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T09:05:47.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:46 smithi066 ceph-mon[27953]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T09:05:47.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.950+0000 7ff7667fc640 1 -- 172.21.15.66:0/3177736880 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7ff76809d200 con 0x7ff770100e60 2024-04-21T09:05:47.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff75c0a3bb0 msgr2=0x7ff75c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:47.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff75c0a3bb0 0x7ff75c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff760005fd0 tx=0x7ff760005c70 comp rx=0 tx=0).stop 2024-04-21T09:05:47.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770100e60 msgr2=0x7ff7701981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770100e60 0x7ff7701981b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff76800ebc0 tx=0x7ff76800c6a0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 shutdown_connections 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff75c0a3bb0 0x7ff75c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff770102a00 0x7ff77019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff770102040 0x7ff7701986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 --2- 172.21.15.66:0/3177736880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff770100e60 0x7ff7701981b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 >> 172.21.15.66:0/3177736880 conn(0x7ff7700fc440 msgr2=0x7ff7700fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 shutdown_connections 2024-04-21T09:05:47.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:47.954+0000 7ff7778a5640 1 -- 172.21.15.66:0/3177736880 wait complete. 2024-04-21T09:05:47.968 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-21T09:05:48.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 -- 172.21.15.66:0/3451159902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b0103b70 msgr2=0x7fb9b010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/3451159902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b0103b70 0x7fb9b010a400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009ca0 tx=0x7fb9a802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 -- 172.21.15.66:0/3451159902 shutdown_connections 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/3451159902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b0103b70 0x7fb9b010a400 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/3451159902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b01031b0 0x7fb9b0103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/3451159902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b0101fc0 0x7fb9b01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 -- 172.21.15.66:0/3451159902 >> 172.21.15.66:0/3451159902 conn(0x7fb9b00fd550 msgr2=0x7fb9b00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 -- 172.21.15.66:0/3451159902 shutdown_connections 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 -- 172.21.15.66:0/3451159902 wait complete. 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 Processor -- start 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.084+0000 7fb9b69b2640 1 -- start start 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b69b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b0101fc0 0x7fb9b01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:48.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b69b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b01031b0 0x7fb9b0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b69b2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0103b70 0x7fb9b019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b69b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010db40 con 0x7fb9b01031b0 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b69b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010d9c0 con 0x7fb9b0101fc0 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b69b2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010dcc0 con 0x7fb9b0103b70 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b01031b0 0x7fb9b0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b01031b0 0x7fb9b0198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36618/0 (socket says 172.21.15.66:36618) 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 -- 172.21.15.66:0/876285089 learned_addr learned my addr 172.21.15.66:0/876285089 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9affff640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b0101fc0 0x7fb9b01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b4f4f640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0103b70 0x7fb9b019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 -- 172.21.15.66:0/876285089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0103b70 msgr2=0x7fb9b019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0103b70 0x7fb9b019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 -- 172.21.15.66:0/876285089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b0101fc0 msgr2=0x7fb9b01981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b0101fc0 0x7fb9b01981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:48.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 -- 172.21.15.66:0/876285089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b019fe10 con 0x7fb9b01031b0 2024-04-21T09:05:48.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9af7fe640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b01031b0 0x7fb9b0198730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b4a0 tx=0x7fb9a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:48.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9ad7fa640 1 -- 172.21.15.66:0/876285089 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4014070 con 0x7fb9b01031b0 2024-04-21T09:05:48.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b01a0100 con 0x7fb9b01031b0 2024-04-21T09:05:48.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.085+0000 7fb9ad7fa640 1 -- 172.21.15.66:0/876285089 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a4010ac0 con 0x7fb9b01031b0 2024-04-21T09:05:48.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.086+0000 7fb9ad7fa640 1 -- 172.21.15.66:0/876285089 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4017c30 con 0x7fb9b01031b0 2024-04-21T09:05:48.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.086+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a0650 con 0x7fb9b01031b0 2024-04-21T09:05:48.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.086+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b01092e0 con 0x7fb9b01031b0 2024-04-21T09:05:48.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.088+0000 7fb9ad7fa640 1 -- 172.21.15.66:0/876285089 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9a4017070 con 0x7fb9b01031b0 2024-04-21T09:05:48.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.088+0000 7fb9ad7fa640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb9980a3bb0 0x7fb9980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:48.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.088+0000 7fb9affff640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb9980a3bb0 0x7fb9980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:48.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.088+0000 7fb9ad7fa640 1 -- 172.21.15.66:0/876285089 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fb9a40cfb20 con 0x7fb9b01031b0 2024-04-21T09:05:48.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.089+0000 7fb9affff640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb9980a3bb0 0x7fb9980a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0103010 tx=0x7fb9a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:48.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.093+0000 7fb9ad7fa640 1 -- 172.21.15.66:0/876285089 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9a409c4d0 con 0x7fb9b01031b0 2024-04-21T09:05:48.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:48.279+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb9b0062b50 con 0x7fb9b01031b0 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: pgmap v288: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2822862838' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3177736880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T09:05:48.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3177736880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T09:05:48.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:47 smithi186 ceph-mon[31505]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T09:05:48.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: pgmap v288: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:48.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T09:05:48.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T09:05:48.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2822862838' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T09:05:48.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T09:05:48.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T09:05:48.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3177736880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T09:05:48.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3177736880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T09:05:48.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:47 smithi083 ceph-mon[31413]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T09:05:48.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: pgmap v288: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2822862838' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3177736880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3177736880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T09:05:48.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:47 smithi066 ceph-mon[27953]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T09:05:49.242 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:48 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] up:active 2024-04-21T09:05:49.242 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:48 smithi186 ceph-mon[31505]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:49.242 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:48 smithi186 ceph-mon[31505]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:49.242 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:48 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/876285089' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T09:05:49.261 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:48 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] up:active 2024-04-21T09:05:49.261 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:48 smithi083 ceph-mon[31413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:49.261 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:48 smithi083 ceph-mon[31413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:49.261 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:48 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/876285089' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T09:05:49.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:48 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] up:active 2024-04-21T09:05:49.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:48 smithi066 ceph-mon[27953]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:49.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:48 smithi066 ceph-mon[27953]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:49.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:48 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/876285089' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T09:05:49.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.985+0000 7fb9ad7fa640 1 -- 172.21.15.66:0/876285089 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fb9a409bdd0 con 0x7fb9b01031b0 2024-04-21T09:05:49.986 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' created 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb9980a3bb0 msgr2=0x7fb9980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb9980a3bb0 0x7fb9980a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0103010 tx=0x7fb9a000b390 comp rx=0 tx=0).stop 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b01031b0 msgr2=0x7fb9b0198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b01031b0 0x7fb9b0198730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b4a0 tx=0x7fb9a400b960 comp rx=0 tx=0).stop 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 shutdown_connections 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb9980a3bb0 0x7fb9980a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9b0103b70 0x7fb9b019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9b01031b0 0x7fb9b0198730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 --2- 172.21.15.66:0/876285089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b0101fc0 0x7fb9b01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:49.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 >> 172.21.15.66:0/876285089 conn(0x7fb9b00fd550 msgr2=0x7fb9b01064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:49.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 shutdown_connections 2024-04-21T09:05:49.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:49.988+0000 7fb9b69b2640 1 -- 172.21.15.66:0/876285089 wait complete. 2024-04-21T09:05:50.008 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 -- 172.21.15.66:0/2270923811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c101fe0 msgr2=0x7ff51c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 --2- 172.21.15.66:0/2270923811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c101fe0 0x7ff51c102460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff504009900 tx=0x7ff50402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 -- 172.21.15.66:0/2270923811 shutdown_connections 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 --2- 172.21.15.66:0/2270923811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff51c1029a0 0x7ff51c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 --2- 172.21.15.66:0/2270923811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c101fe0 0x7ff51c102460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 --2- 172.21.15.66:0/2270923811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c100e90 0x7ff51c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 -- 172.21.15.66:0/2270923811 >> 172.21.15.66:0/2270923811 conn(0x7ff51c0fc400 msgr2=0x7ff51c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 -- 172.21.15.66:0/2270923811 shutdown_connections 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 -- 172.21.15.66:0/2270923811 wait complete. 2024-04-21T09:05:50.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 Processor -- start 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.119+0000 7ff5214ba640 1 -- start start 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c100e90 0x7ff51c069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff51c101fe0 0x7ff51c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c1029a0 0x7ff51c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c10c9e0 con 0x7ff51c100e90 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c10c860 con 0x7ff51c1029a0 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c10cb60 con 0x7ff51c101fe0 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c100e90 0x7ff51c069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c100e90 0x7ff51c069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36644/0 (socket says 172.21.15.66:36644) 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 -- 172.21.15.66:0/3391669536 learned_addr learned my addr 172.21.15.66:0/3391669536 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51b7fe640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c1029a0 0x7ff51c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51a7fc640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff51c101fe0 0x7ff51c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:50.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 -- 172.21.15.66:0/3391669536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff51c101fe0 msgr2=0x7ff51c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff51c101fe0 0x7ff51c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 -- 172.21.15.66:0/3391669536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c1029a0 msgr2=0x7ff51c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c1029a0 0x7ff51c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 -- 172.21.15.66:0/3391669536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c19fe70 con 0x7ff51c100e90 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51a7fc640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff51c101fe0 0x7ff51c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff51affd640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c100e90 0x7ff51c069c70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e9f0 tx=0x7ff51400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff513fff640 1 -- 172.21.15.66:0/3391669536 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51400cd80 con 0x7ff51c100e90 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff513fff640 1 -- 172.21.15.66:0/3391669536 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5140108e0 con 0x7ff51c100e90 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c1a0160 con 0x7ff51c100e90 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff513fff640 1 -- 172.21.15.66:0/3391669536 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514018980 con 0x7ff51c100e90 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.120+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1a06d0 con 0x7ff51c100e90 2024-04-21T09:05:50.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.122+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c108290 con 0x7ff51c100e90 2024-04-21T09:05:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.123+0000 7ff513fff640 1 -- 172.21.15.66:0/3391669536 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff514010430 con 0x7ff51c100e90 2024-04-21T09:05:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.123+0000 7ff513fff640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff5080a3c30 0x7ff5080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.123+0000 7ff51a7fc640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff5080a3c30 0x7ff5080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.123+0000 7ff513fff640 1 -- 172.21.15.66:0/3391669536 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7ff5140d0990 con 0x7ff51c100e90 2024-04-21T09:05:50.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.124+0000 7ff51a7fc640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff5080a3c30 0x7ff5080a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff504006010 tx=0x7ff50402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:50.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.128+0000 7ff513fff640 1 -- 172.21.15.66:0/3391669536 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff51409d150 con 0x7ff51c100e90 2024-04-21T09:05:50.256 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:49 smithi186 ceph-mon[31505]: pgmap v291: 97 pgs: 16 creating+peering, 50 active+clean, 31 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-21T09:05:50.257 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:49 smithi186 ceph-mon[31505]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-21T09:05:50.257 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:49 smithi186 ceph-mon[31505]: Cluster is now healthy 2024-04-21T09:05:50.257 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:49 smithi186 ceph-mon[31505]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T09:05:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.315+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ff51c062bc0 con 0x7ff51c100e90 2024-04-21T09:05:50.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:49 smithi083 ceph-mon[31413]: pgmap v291: 97 pgs: 16 creating+peering, 50 active+clean, 31 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-21T09:05:50.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:49 smithi083 ceph-mon[31413]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-21T09:05:50.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:49 smithi083 ceph-mon[31413]: Cluster is now healthy 2024-04-21T09:05:50.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:49 smithi083 ceph-mon[31413]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T09:05:50.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:49 smithi066 ceph-mon[27953]: pgmap v291: 97 pgs: 16 creating+peering, 50 active+clean, 31 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-21T09:05:50.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:49 smithi066 ceph-mon[27953]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-21T09:05:50.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:49 smithi066 ceph-mon[27953]: Cluster is now healthy 2024-04-21T09:05:50.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:49 smithi066 ceph-mon[27953]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T09:05:50.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.991+0000 7ff513fff640 1 -- 172.21.15.66:0/3391669536 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7ff51409ca50 con 0x7ff51c100e90 2024-04-21T09:05:50.992 INFO:teuthology.orchestra.run.smithi066.stderr:set pool 4 allow_ec_overwrites to true 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.994+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff5080a3c30 msgr2=0x7ff5080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.994+0000 7ff5214ba640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff5080a3c30 0x7ff5080a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff504006010 tx=0x7ff50402c7e0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.994+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c100e90 msgr2=0x7ff51c069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.994+0000 7ff5214ba640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c100e90 0x7ff51c069c70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e9f0 tx=0x7ff51400eeb0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 shutdown_connections 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff5080a3c30 0x7ff5080a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c1029a0 0x7ff51c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff51c101fe0 0x7ff51c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 --2- 172.21.15.66:0/3391669536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51c100e90 0x7ff51c069c70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:50.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 >> 172.21.15.66:0/3391669536 conn(0x7ff51c0fc400 msgr2=0x7ff51c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:50.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 shutdown_connections 2024-04-21T09:05:50.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:50.995+0000 7ff5214ba640 1 -- 172.21.15.66:0/3391669536 wait complete. 2024-04-21T09:05:51.018 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-21T09:05:51.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1866751657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e4101ac0 msgr2=0x7fc5e4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1866751657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e4101ac0 0x7fc5e4109360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009ca0 tx=0x7fc5dc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1866751657 shutdown_connections 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1866751657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e4101ac0 0x7fc5e4109360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1866751657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e4101100 0x7fc5e4101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1866751657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e41024b0 0x7fc5e41028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1866751657 >> 172.21.15.66:0/1866751657 conn(0x7fc5e40fbb90 msgr2=0x7fc5e40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1866751657 shutdown_connections 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.128+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1866751657 wait complete. 2024-04-21T09:05:51.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 Processor -- start 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 -- start start 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e4101100 0x7fc5e41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e4101ac0 0x7fc5e4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e41024b0 0x7fc5e419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410db50 con 0x7fc5e41024b0 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410d9d0 con 0x7fc5e4101100 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e8b04640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410dcd0 con 0x7fc5e4101ac0 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e41024b0 0x7fc5e419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e41024b0 0x7fc5e419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36656/0 (socket says 172.21.15.66:36656) 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e2d9d640 1 -- 172.21.15.66:0/1957468543 learned_addr learned my addr 172.21.15.66:0/1957468543 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:51.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.129+0000 7fc5e1d9b640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e4101ac0 0x7fc5e4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e259c640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e4101100 0x7fc5e41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e2d9d640 1 -- 172.21.15.66:0/1957468543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e4101ac0 msgr2=0x7fc5e4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e2d9d640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e4101ac0 0x7fc5e4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e2d9d640 1 -- 172.21.15.66:0/1957468543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e4101100 msgr2=0x7fc5e41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e2d9d640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e4101100 0x7fc5e41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e2d9d640 1 -- 172.21.15.66:0/1957468543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e419fdf0 con 0x7fc5e41024b0 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e259c640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e4101100 0x7fc5e41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e2d9d640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e41024b0 0x7fc5e419f780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc032520 tx=0x7fc5dc0347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5d37fe640 1 -- 172.21.15.66:0/1957468543 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc039040 con 0x7fc5e41024b0 2024-04-21T09:05:51.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e41a0080 con 0x7fc5e41024b0 2024-04-21T09:05:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5d37fe640 1 -- 172.21.15.66:0/1957468543 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5dc044070 con 0x7fc5e41024b0 2024-04-21T09:05:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5d37fe640 1 -- 172.21.15.66:0/1957468543 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0042a0 con 0x7fc5e41024b0 2024-04-21T09:05:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.130+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41a05c0 con 0x7fc5e41024b0 2024-04-21T09:05:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.131+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e41082e0 con 0x7fc5e41024b0 2024-04-21T09:05:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.132+0000 7fc5d37fe640 1 -- 172.21.15.66:0/1957468543 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc5dc003970 con 0x7fc5e41024b0 2024-04-21T09:05:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.133+0000 7fc5d37fe640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc5cc0a3bb0 0x7fc5cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.133+0000 7fc5e259c640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc5cc0a3bb0 0x7fc5cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.133+0000 7fc5d37fe640 1 -- 172.21.15.66:0/1957468543 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fc5dc0f2f60 con 0x7fc5e41024b0 2024-04-21T09:05:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.133+0000 7fc5e259c640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc5cc0a3bb0 0x7fc5cc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5e406a370 tx=0x7fc5d4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.137+0000 7fc5d37fe640 1 -- 172.21.15.66:0/1957468543 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5dc035d30 con 0x7fc5e41024b0 2024-04-21T09:05:51.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:50 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/876285089' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T09:05:51.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:50 smithi186 ceph-mon[31505]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T09:05:51.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:50 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3391669536' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T09:05:51.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:50 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/876285089' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T09:05:51.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:50 smithi083 ceph-mon[31413]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T09:05:51.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:50 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3391669536' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T09:05:51.360 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:05:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: 2024-04-21T09:05:51.134+0000 7f30a2500700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-21T09:05:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.365+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fc5e4062b50 con 0x7fc5e41024b0 2024-04-21T09:05:51.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:50 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/876285089' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T09:05:51.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:50 smithi066 ceph-mon[27953]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T09:05:51.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:50 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3391669536' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T09:05:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:51.999+0000 7fc5d37fe640 1 -- 172.21.15.66:0/1957468543 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fc5dc0bf720 con 0x7fc5e41024b0 2024-04-21T09:05:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:added data pool 4 to fsmap 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc5cc0a3bb0 msgr2=0x7fc5cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc5cc0a3bb0 0x7fc5cc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5e406a370 tx=0x7fc5d4005d30 comp rx=0 tx=0).stop 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e41024b0 msgr2=0x7fc5e419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e41024b0 0x7fc5e419f780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc032520 tx=0x7fc5dc0347b0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 shutdown_connections 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fc5cc0a3bb0 0x7fc5cc0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5e41024b0 0x7fc5e419f780 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e4101ac0 0x7fc5e4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 --2- 172.21.15.66:0/1957468543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e4101100 0x7fc5e41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 >> 172.21.15.66:0/1957468543 conn(0x7fc5e40fbb90 msgr2=0x7fc5e41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.002+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 shutdown_connections 2024-04-21T09:05:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.003+0000 7fc5e8b04640 1 -- 172.21.15.66:0/1957468543 wait complete. 2024-04-21T09:05:52.017 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 -- 172.21.15.66:0/3660941807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd930100ea0 msgr2=0x7fd9301012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 --2- 172.21.15.66:0/3660941807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd930100ea0 0x7fd9301012c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd920009900 tx=0x7fd92002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 -- 172.21.15.66:0/3660941807 shutdown_connections 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 --2- 172.21.15.66:0/3660941807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9301029b0 0x7fd9301092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 --2- 172.21.15.66:0/3660941807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd930101ff0 0x7fd930102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 --2- 172.21.15.66:0/3660941807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd930100ea0 0x7fd9301012c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 -- 172.21.15.66:0/3660941807 >> 172.21.15.66:0/3660941807 conn(0x7fd9300fc410 msgr2=0x7fd9300fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 -- 172.21.15.66:0/3660941807 shutdown_connections 2024-04-21T09:05:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.126+0000 7fd934d59640 1 -- 172.21.15.66:0/3660941807 wait complete. 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 Processor -- start 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 -- start start 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd930100ea0 0x7fd930197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd930101ff0 0x7fd930197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9301029b0 0x7fd93019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010cac0 con 0x7fd9301029b0 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010c940 con 0x7fd930100ea0 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd934d59640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010cc40 con 0x7fd930101ff0 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9301029b0 0x7fd93019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9301029b0 0x7fd93019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36676/0 (socket says 172.21.15.66:36676) 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd92ed9d640 1 -- 172.21.15.66:0/652499182 learned_addr learned my addr 172.21.15.66:0/652499182 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd92dd9b640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd930101ff0 0x7fd930197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.127+0000 7fd92e59c640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd930100ea0 0x7fd930197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd92ed9d640 1 -- 172.21.15.66:0/652499182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd930101ff0 msgr2=0x7fd930197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd92ed9d640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd930101ff0 0x7fd930197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd92ed9d640 1 -- 172.21.15.66:0/652499182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd930100ea0 msgr2=0x7fd930197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd92ed9d640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd930100ea0 0x7fd930197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd92ed9d640 1 -- 172.21.15.66:0/652499182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93019ed30 con 0x7fd9301029b0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd92ed9d640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9301029b0 0x7fd93019e6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd92800b970 tx=0x7fd92800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd91f7fe640 1 -- 172.21.15.66:0/652499182 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928019070 con 0x7fd9301029b0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd91f7fe640 1 -- 172.21.15.66:0/652499182 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd928014440 con 0x7fd9301029b0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd91f7fe640 1 -- 172.21.15.66:0/652499182 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9280075d0 con 0x7fd9301029b0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93019efc0 con 0x7fd9301029b0 2024-04-21T09:05:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.128+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93019f560 con 0x7fd9301029b0 2024-04-21T09:05:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.129+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9301081b0 con 0x7fd9301029b0 2024-04-21T09:05:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.130+0000 7fd91f7fe640 1 -- 172.21.15.66:0/652499182 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd92801c070 con 0x7fd9301029b0 2024-04-21T09:05:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.131+0000 7fd91f7fe640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd9180a3c30 0x7fd9180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.131+0000 7fd92e59c640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd9180a3c30 0x7fd9180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.131+0000 7fd91f7fe640 1 -- 172.21.15.66:0/652499182 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd9280cfb30 con 0x7fd9301029b0 2024-04-21T09:05:52.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.131+0000 7fd92e59c640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd9180a3c30 0x7fd9180a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd92002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:52.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.136+0000 7fd91f7fe640 1 -- 172.21.15.66:0/652499182 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd92809bc80 con 0x7fd9301029b0 2024-04-21T09:05:52.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:51 smithi186 ceph-mon[31505]: pgmap v294: 129 pgs: 13 creating+peering, 72 active+clean, 44 unknown; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-04-21T09:05:52.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:51 smithi186 ceph-mon[31505]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T09:05:52.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:51 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3391669536' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T09:05:52.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:51 smithi186 ceph-mon[31505]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T09:05:52.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:51 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1957468543' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T09:05:52.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:51 smithi083 ceph-mon[31413]: pgmap v294: 129 pgs: 13 creating+peering, 72 active+clean, 44 unknown; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-04-21T09:05:52.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:51 smithi083 ceph-mon[31413]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T09:05:52.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:51 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3391669536' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T09:05:52.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:51 smithi083 ceph-mon[31413]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T09:05:52.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:51 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1957468543' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T09:05:52.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.362+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd93002d5c0 con 0x7fd9301029b0 2024-04-21T09:05:52.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.362+0000 7fd91f7fe640 1 -- 172.21.15.66:0/652499182 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7fd92809ff80 con 0x7fd9301029b0 2024-04-21T09:05:52.363 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:52.363 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:05:51.367785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24512,"qdb_cluster":[24512]},"id":1}]} 2024-04-21T09:05:52.363 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.364+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd9180a3c30 msgr2=0x7fd9180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.364+0000 7fd934d59640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd9180a3c30 0x7fd9180a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd92002c7e0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.364+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9301029b0 msgr2=0x7fd93019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.364+0000 7fd934d59640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9301029b0 0x7fd93019e6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd92800b970 tx=0x7fd92800be30 comp rx=0 tx=0).stop 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 shutdown_connections 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd9180a3c30 0x7fd9180a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9301029b0 0x7fd93019e6c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd930101ff0 0x7fd930197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 --2- 172.21.15.66:0/652499182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd930100ea0 0x7fd930197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 >> 172.21.15.66:0/652499182 conn(0x7fd9300fc410 msgr2=0x7fd930105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:52.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 shutdown_connections 2024-04-21T09:05:52.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.365+0000 7fd934d59640 1 -- 172.21.15.66:0/652499182 wait complete. 2024-04-21T09:05:52.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:51 smithi066 ceph-mon[27953]: pgmap v294: 129 pgs: 13 creating+peering, 72 active+clean, 44 unknown; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-04-21T09:05:52.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:51 smithi066 ceph-mon[27953]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T09:05:52.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:51 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3391669536' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T09:05:52.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:51 smithi066 ceph-mon[27953]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T09:05:52.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:51 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1957468543' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T09:05:52.376 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:05:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 -- 172.21.15.66:0/2270272446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102a10 msgr2=0x7f81fc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 --2- 172.21.15.66:0/2270272446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102a10 0x7f81fc109330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009ca0 tx=0x7f81f402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 -- 172.21.15.66:0/2270272446 shutdown_connections 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 --2- 172.21.15.66:0/2270272446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102a10 0x7f81fc109330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 --2- 172.21.15.66:0/2270272446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc102050 0x7f81fc1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 --2- 172.21.15.66:0/2270272446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81fc100e70 0x7f81fc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 -- 172.21.15.66:0/2270272446 >> 172.21.15.66:0/2270272446 conn(0x7f81fc0fc470 msgr2=0x7f81fc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 -- 172.21.15.66:0/2270272446 shutdown_connections 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.477+0000 7f8202ba4640 1 -- 172.21.15.66:0/2270272446 wait complete. 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 Processor -- start 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 -- start start 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc100e70 0x7f81fc06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102050 0x7f81fc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81fc102a10 0x7f81fc19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc10ca20 con 0x7f81fc102050 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc10c8a0 con 0x7f81fc100e70 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8202ba4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc10cba0 con 0x7f81fc102a10 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102050 0x7f81fc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102050 0x7f81fc19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36690/0 (socket says 172.21.15.66:36690) 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 -- 172.21.15.66:0/3214147891 learned_addr learned my addr 172.21.15.66:0/3214147891 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8200940640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc100e70 0x7f81fc06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f8201141640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81fc102a10 0x7f81fc19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 -- 172.21.15.66:0/3214147891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81fc102a10 msgr2=0x7f81fc19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81fc102a10 0x7f81fc19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 -- 172.21.15.66:0/3214147891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc100e70 msgr2=0x7f81fc06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc100e70 0x7f81fc06b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 -- 172.21.15.66:0/3214147891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc19fdc0 con 0x7f81fc102050 2024-04-21T09:05:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81fbfff640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102050 0x7f81fc19ce90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81e800d950 tx=0x7f81e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.478+0000 7f81f9ffb640 1 -- 172.21.15.66:0/3214147891 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8017030 con 0x7f81fc102050 2024-04-21T09:05:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.479+0000 7f81f9ffb640 1 -- 172.21.15.66:0/3214147891 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e8004020 con 0x7f81fc102050 2024-04-21T09:05:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.479+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc19fff0 con 0x7f81fc102050 2024-04-21T09:05:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.479+0000 7f81f9ffb640 1 -- 172.21.15.66:0/3214147891 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80047a0 con 0x7f81fc102050 2024-04-21T09:05:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.479+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc1a0510 con 0x7f81fc102050 2024-04-21T09:05:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.480+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc108210 con 0x7f81fc102050 2024-04-21T09:05:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.481+0000 7f81f9ffb640 1 -- 172.21.15.66:0/3214147891 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81e8009a30 con 0x7f81fc102050 2024-04-21T09:05:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.481+0000 7f81f9ffb640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81ec0a3bb0 0x7f81ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.481+0000 7f8200940640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81ec0a3bb0 0x7f81ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.481+0000 7f81f9ffb640 1 -- 172.21.15.66:0/3214147891 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f81e80d0040 con 0x7f81fc102050 2024-04-21T09:05:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.482+0000 7f8200940640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81ec0a3bb0 0x7f81ec0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81f0005fd0 tx=0x7f81f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:52.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.486+0000 7f81f9ffb640 1 -- 172.21.15.66:0/3214147891 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81e809c7e0 con 0x7f81fc102050 2024-04-21T09:05:52.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.675+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81fc062bf0 con 0x7f81fc102050 2024-04-21T09:05:52.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.677+0000 7f81f9ffb640 1 -- 172.21.15.66:0/3214147891 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24103 (secure 0 0 0) 0x7f81e809c0e0 con 0x7f81fc102050 2024-04-21T09:05:52.677 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:52.678 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":90,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:51.367590+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:05:52.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81ec0a3bb0 msgr2=0x7f81ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81ec0a3bb0 0x7f81ec0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81f0005fd0 tx=0x7f81f000b390 comp rx=0 tx=0).stop 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102050 msgr2=0x7f81fc19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102050 0x7f81fc19ce90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81e800d950 tx=0x7f81e800de10 comp rx=0 tx=0).stop 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 shutdown_connections 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81ec0a3bb0 0x7f81ec0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81fc102a10 0x7f81fc19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81fc102050 0x7f81fc19ce90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 --2- 172.21.15.66:0/3214147891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc100e70 0x7f81fc06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 >> 172.21.15.66:0/3214147891 conn(0x7f81fc0fc470 msgr2=0x7f81fc1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 shutdown_connections 2024-04-21T09:05:52.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.679+0000 7f8202ba4640 1 -- 172.21.15.66:0/3214147891 wait complete. 2024-04-21T09:05:52.691 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:05:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 -- 172.21.15.66:0/3981104609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea81053c0 msgr2=0x7f3ea81077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/3981104609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea81053c0 0x7f3ea81077b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009900 tx=0x7f3e9c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 -- 172.21.15.66:0/3981104609 shutdown_connections 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/3981104609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8107cf0 0x7f3ea810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/3981104609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea81053c0 0x7f3ea81077b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/3981104609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ea8102a90 0x7f3ea8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 -- 172.21.15.66:0/3981104609 >> 172.21.15.66:0/3981104609 conn(0x7f3ea80fc3e0 msgr2=0x7f3ea80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 -- 172.21.15.66:0/3981104609 shutdown_connections 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 -- 172.21.15.66:0/3981104609 wait complete. 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.792+0000 7f3eae4d2640 1 Processor -- start 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eae4d2640 1 -- start start 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eae4d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8102a90 0x7f3ea8198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eae4d2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ea81053c0 0x7f3ea8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eae4d2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea8107cf0 0x7f3ea819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eae4d2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea810da80 con 0x7f3ea8107cf0 2024-04-21T09:05:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eae4d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea810d900 con 0x7f3ea8102a90 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eae4d2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea810dc00 con 0x7f3ea81053c0 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea8107cf0 0x7f3ea819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea8107cf0 0x7f3ea819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36702/0 (socket says 172.21.15.66:36702) 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 -- 172.21.15.66:0/2413011892 learned_addr learned my addr 172.21.15.66:0/2413011892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3ea77fe640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ea81053c0 0x7f3ea8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3ea7fff640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8102a90 0x7f3ea8198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 -- 172.21.15.66:0/2413011892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ea81053c0 msgr2=0x7f3ea8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ea81053c0 0x7f3ea8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 -- 172.21.15.66:0/2413011892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8102a90 msgr2=0x7f3ea8198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8102a90 0x7f3ea8198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 -- 172.21.15.66:0/2413011892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea819fd70 con 0x7f3ea8107cf0 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.793+0000 7f3eaca6f640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea8107cf0 0x7f3ea819f700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000eaa0 tx=0x7f3ea000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.794+0000 7f3ea57fa640 1 -- 172.21.15.66:0/2413011892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000cdb0 con 0x7f3ea8107cf0 2024-04-21T09:05:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.794+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81a0060 con 0x7f3ea8107cf0 2024-04-21T09:05:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.794+0000 7f3ea57fa640 1 -- 172.21.15.66:0/2413011892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea0005020 con 0x7f3ea8107cf0 2024-04-21T09:05:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.794+0000 7f3ea57fa640 1 -- 172.21.15.66:0/2413011892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea00116d0 con 0x7f3ea8107cf0 2024-04-21T09:05:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.794+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81a0580 con 0x7f3ea8107cf0 2024-04-21T09:05:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.795+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea80fded0 con 0x7f3ea8107cf0 2024-04-21T09:05:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.796+0000 7f3ea57fa640 1 -- 172.21.15.66:0/2413011892 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ea00051d0 con 0x7f3ea8107cf0 2024-04-21T09:05:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.796+0000 7f3ea57fa640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e900a3bb0 0x7f3e900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.796+0000 7f3ea7fff640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e900a3bb0 0x7f3e900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.796+0000 7f3ea57fa640 1 -- 172.21.15.66:0/2413011892 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3ea00d08f0 con 0x7f3ea8107cf0 2024-04-21T09:05:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.797+0000 7f3ea7fff640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e900a3bb0 0x7f3e900a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800ba40 tx=0x7f3e98005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:52.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.801+0000 7f3ea57fa640 1 -- 172.21.15.66:0/2413011892 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ea0016030 con 0x7f3ea8107cf0 2024-04-21T09:05:52.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.988+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ea8062b50 con 0x7f3ea8107cf0 2024-04-21T09:05:52.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.990+0000 7f3ea57fa640 1 -- 172.21.15.66:0/2413011892 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24103 (secure 0 0 0) 0x7f3ea009d090 con 0x7f3ea8107cf0 2024-04-21T09:05:52.991 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:52.991 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":90,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:51.367590+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e900a3bb0 msgr2=0x7f3e900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e900a3bb0 0x7f3e900a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800ba40 tx=0x7f3e98005cc0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea8107cf0 msgr2=0x7f3ea819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea8107cf0 0x7f3ea819f700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000eaa0 tx=0x7f3ea000ef60 comp rx=0 tx=0).stop 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 shutdown_connections 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e900a3bb0 0x7f3e900a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ea8107cf0 0x7f3ea819f700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ea81053c0 0x7f3ea8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 --2- 172.21.15.66:0/2413011892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8102a90 0x7f3ea8198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 >> 172.21.15.66:0/2413011892 conn(0x7f3ea80fc3e0 msgr2=0x7f3ea81034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:52.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 shutdown_connections 2024-04-21T09:05:52.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:52.992+0000 7f3eae4d2640 1 -- 172.21.15.66:0/2413011892 wait complete. 2024-04-21T09:05:53.024 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 -- 172.21.15.66:0/526378236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8100e70 msgr2=0x7f8fb8101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/526378236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8100e70 0x7f8fb8101290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009900 tx=0x7f8fa802bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 -- 172.21.15.66:0/526378236 shutdown_connections 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/526378236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fb8102980 0x7f8fb81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/526378236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb8101fc0 0x7f8fb8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/526378236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8100e70 0x7f8fb8101290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 -- 172.21.15.66:0/526378236 >> 172.21.15.66:0/526378236 conn(0x7f8fb80fc3e0 msgr2=0x7f8fb80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 -- 172.21.15.66:0/526378236 shutdown_connections 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 -- 172.21.15.66:0/526378236 wait complete. 2024-04-21T09:05:53.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 Processor -- start 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.126+0000 7f8fbe22c640 1 -- start start 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb8100e70 0x7f8fb8069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8101fc0 0x7f8fb819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fb8102980 0x7f8fb819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810c9a0 con 0x7f8fb8101fc0 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810c820 con 0x7f8fb8102980 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810cb20 con 0x7f8fb8100e70 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8101fc0 0x7f8fb819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8101fc0 0x7f8fb819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36720/0 (socket says 172.21.15.66:36720) 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 -- 172.21.15.66:0/1736853299 learned_addr learned my addr 172.21.15.66:0/1736853299 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb77fe640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb8100e70 0x7f8fb8069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb7fff640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fb8102980 0x7f8fb819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 -- 172.21.15.66:0/1736853299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb8100e70 msgr2=0x7f8fb8069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb8100e70 0x7f8fb8069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 -- 172.21.15.66:0/1736853299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fb8102980 msgr2=0x7f8fb819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fb8102980 0x7f8fb819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 -- 172.21.15.66:0/1736853299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb819fd70 con 0x7f8fb8101fc0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb7fff640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fb8102980 0x7f8fb819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb77fe640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb8100e70 0x7f8fb8069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb6ffd640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8101fc0 0x7f8fb819ce70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c970 tx=0x7f8fac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb4ff9640 1 -- 172.21.15.66:0/1736853299 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac007c10 con 0x7f8fb8101fc0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb4ff9640 1 -- 172.21.15.66:0/1736853299 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fac007db0 con 0x7f8fb8101fc0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81a0060 con 0x7f8fb8101fc0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fb4ff9640 1 -- 172.21.15.66:0/1736853299 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00f870 con 0x7f8fb8101fc0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.127+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a05b0 con 0x7f8fb8101fc0 2024-04-21T09:05:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.129+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8108180 con 0x7f8fb8101fc0 2024-04-21T09:05:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.130+0000 7f8fb4ff9640 1 -- 172.21.15.66:0/1736853299 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fac00fa10 con 0x7f8fb8101fc0 2024-04-21T09:05:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.130+0000 7f8fb4ff9640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fa00a3bb0 0x7f8fa00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.131+0000 7f8fb77fe640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fa00a3bb0 0x7f8fa00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.131+0000 7f8fb4ff9640 1 -- 172.21.15.66:0/1736853299 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f8fac0d0a80 con 0x7f8fb8101fc0 2024-04-21T09:05:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.131+0000 7f8fb77fe640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fa00a3bb0 0x7f8fa00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8006010 tx=0x7f8fa802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.135+0000 7f8fb4ff9640 1 -- 172.21.15.66:0/1736853299 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fac09d2b0 con 0x7f8fb8101fc0 2024-04-21T09:05:53.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:53 smithi186 ceph-mon[31505]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T09:05:53.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:53 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1957468543' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T09:05:53.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:53 smithi186 ceph-mon[31505]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:53.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:53 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/652499182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:53 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3214147891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:53 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2413011892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:53 smithi083 ceph-mon[31413]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T09:05:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:53 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1957468543' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T09:05:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:53 smithi083 ceph-mon[31413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:53 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/652499182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:53 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3214147891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:53 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2413011892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.361+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fb802d5c0 con 0x7f8fb8101fc0 2024-04-21T09:05:53.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.361+0000 7f8fb4ff9640 1 -- 172.21.15.66:0/1736853299 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7f8fac09cbb0 con 0x7f8fb8101fc0 2024-04-21T09:05:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:05:51.367785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24512,"qdb_cluster":[24512]},"id":1}]} 2024-04-21T09:05:53.362 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-04-21T09:05:53.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fa00a3bb0 msgr2=0x7f8fa00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fa00a3bb0 0x7f8fa00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8006010 tx=0x7f8fa802c7e0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8101fc0 msgr2=0x7f8fb819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8101fc0 0x7f8fb819ce70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c970 tx=0x7f8fac00ce30 comp rx=0 tx=0).stop 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 shutdown_connections 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fa00a3bb0 0x7f8fa00a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fb8102980 0x7f8fb819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fb8101fc0 0x7f8fb819ce70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 --2- 172.21.15.66:0/1736853299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb8100e70 0x7f8fb8069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 >> 172.21.15.66:0/1736853299 conn(0x7f8fb80fc3e0 msgr2=0x7f8fb8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 shutdown_connections 2024-04-21T09:05:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.364+0000 7f8fbe22c640 1 -- 172.21.15.66:0/1736853299 wait complete. 2024-04-21T09:05:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:53 smithi066 ceph-mon[27953]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T09:05:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:53 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1957468543' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T09:05:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:53 smithi066 ceph-mon[27953]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:05:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:53 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/652499182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:53 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3214147891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:53 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2413011892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:53.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:05:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 -- 172.21.15.66:0/249931758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10102a80 msgr2=0x7fad10104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 --2- 172.21.15.66:0/249931758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10102a80 0x7fad10104e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad00009900 tx=0x7fad0002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 -- 172.21.15.66:0/249931758 shutdown_connections 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 --2- 172.21.15.66:0/249931758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10107ce0 0x7fad1010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 --2- 172.21.15.66:0/249931758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad101053b0 0x7fad101077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 --2- 172.21.15.66:0/249931758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10102a80 0x7fad10104e70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 -- 172.21.15.66:0/249931758 >> 172.21.15.66:0/249931758 conn(0x7fad100fc410 msgr2=0x7fad100fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 -- 172.21.15.66:0/249931758 shutdown_connections 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 -- 172.21.15.66:0/249931758 wait complete. 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.477+0000 7fad155a4640 1 Processor -- start 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad155a4640 1 -- start start 2024-04-21T09:05:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad155a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10102a80 0x7fad1019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad155a4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad101053b0 0x7fad1019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad155a4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10107ce0 0x7fad101a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad155a4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad1010dae0 con 0x7fad10107ce0 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad155a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad1010d960 con 0x7fad10102a80 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad155a4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fad1010dc60 con 0x7fad101053b0 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10107ce0 0x7fad101a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10107ce0 0x7fad101a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36734/0 (socket says 172.21.15.66:36734) 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 -- 172.21.15.66:0/2568994407 learned_addr learned my addr 172.21.15.66:0/2568994407 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0e7fc640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad101053b0 0x7fad1019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0effd640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10102a80 0x7fad1019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 -- 172.21.15.66:0/2568994407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad101053b0 msgr2=0x7fad1019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad101053b0 0x7fad1019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 -- 172.21.15.66:0/2568994407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10102a80 msgr2=0x7fad1019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10102a80 0x7fad1019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 -- 172.21.15.66:0/2568994407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad101a1fb0 con 0x7fad10107ce0 2024-04-21T09:05:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.478+0000 7fad0f7fe640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10107ce0 0x7fad101a1940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad0800cad0 tx=0x7fad0800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.479+0000 7facf7fff640 1 -- 172.21.15.66:0/2568994407 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800ad60 con 0x7fad10107ce0 2024-04-21T09:05:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.479+0000 7facf7fff640 1 -- 172.21.15.66:0/2568994407 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad08005020 con 0x7fad10107ce0 2024-04-21T09:05:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.479+0000 7facf7fff640 1 -- 172.21.15.66:0/2568994407 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800f6b0 con 0x7fad10107ce0 2024-04-21T09:05:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.479+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad101a22a0 con 0x7fad10107ce0 2024-04-21T09:05:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.479+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a27a0 con 0x7fad10107ce0 2024-04-21T09:05:53.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.480+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad101097d0 con 0x7fad10107ce0 2024-04-21T09:05:53.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.481+0000 7facf7fff640 1 -- 172.21.15.66:0/2568994407 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fad080051d0 con 0x7fad10107ce0 2024-04-21T09:05:53.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.481+0000 7facf7fff640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7facf80a3bb0 0x7facf80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.482+0000 7fad0effd640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7facf80a3bb0 0x7facf80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.482+0000 7facf7fff640 1 -- 172.21.15.66:0/2568994407 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fad080d0640 con 0x7fad10107ce0 2024-04-21T09:05:53.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.482+0000 7fad0effd640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7facf80a3bb0 0x7facf80a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fad00006010 tx=0x7fad000095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.486+0000 7facf7fff640 1 -- 172.21.15.66:0/2568994407 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad08014030 con 0x7fad10107ce0 2024-04-21T09:05:53.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.674+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad10062b50 con 0x7fad10107ce0 2024-04-21T09:05:53.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.676+0000 7facf7fff640 1 -- 172.21.15.66:0/2568994407 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7fad0809cde0 con 0x7fad10107ce0 2024-04-21T09:05:53.676 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:53.676 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:05:53.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.677+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7facf80a3bb0 msgr2=0x7facf80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.677+0000 7fad155a4640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7facf80a3bb0 0x7facf80a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fad00006010 tx=0x7fad000095a0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.677+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10107ce0 msgr2=0x7fad101a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.677+0000 7fad155a4640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10107ce0 0x7fad101a1940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad0800cad0 tx=0x7fad0800ced0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 shutdown_connections 2024-04-21T09:05:53.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7facf80a3bb0 0x7facf80a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad10107ce0 0x7fad101a1940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad101053b0 0x7fad1019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 --2- 172.21.15.66:0/2568994407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10102a80 0x7fad1019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 >> 172.21.15.66:0/2568994407 conn(0x7fad100fc410 msgr2=0x7fad101021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 shutdown_connections 2024-04-21T09:05:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.678+0000 7fad155a4640 1 -- 172.21.15.66:0/2568994407 wait complete. 2024-04-21T09:05:53.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-21T09:05:46.330734+0000', 'modified': '2024-04-21T09:05:51.367785+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/4140026829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 4140026829}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 4140026829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24512, 'qdb_cluster': [24512]} 2024-04-21T09:05:53.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-21T09:05:53.690 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T09:05:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 -- 172.21.15.66:0/1859315725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa441031a0 msgr2=0x7faa44103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1859315725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa441031a0 0x7faa44103620 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faa30009900 tx=0x7faa3002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 -- 172.21.15.66:0/1859315725 shutdown_connections 2024-04-21T09:05:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1859315725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa44103b60 0x7faa4410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1859315725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa441031a0 0x7faa44103620 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1859315725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44101fb0 0x7faa441023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 -- 172.21.15.66:0/1859315725 >> 172.21.15.66:0/1859315725 conn(0x7faa440fd520 msgr2=0x7faa440ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 -- 172.21.15.66:0/1859315725 shutdown_connections 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.793+0000 7faa4aad2640 1 -- 172.21.15.66:0/1859315725 wait complete. 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 Processor -- start 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 -- start start 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa44101fb0 0x7faa441981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031a0 0x7faa44198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103b60 0x7faa4419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa4410daf0 con 0x7faa44101fb0 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa4410d970 con 0x7faa44103b60 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4aad2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faa4410dc70 con 0x7faa441031a0 2024-04-21T09:05:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa44101fb0 0x7faa441981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa44101fb0 0x7faa441981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36744/0 (socket says 172.21.15.66:36744) 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 -- 172.21.15.66:0/1873555551 learned_addr learned my addr 172.21.15.66:0/1873555551 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa43fff640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031a0 0x7faa44198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 -- 172.21.15.66:0/1873555551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031a0 msgr2=0x7faa44198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031a0 0x7faa44198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4906f640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103b60 0x7faa4419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 -- 172.21.15.66:0/1873555551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103b60 msgr2=0x7faa4419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103b60 0x7faa4419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 -- 172.21.15.66:0/1873555551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4419fdf0 con 0x7faa44101fb0 2024-04-21T09:05:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa4886e640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa44101fb0 0x7faa441981d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa3800e9c0 tx=0x7faa3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.794+0000 7faa41ffb640 1 -- 172.21.15.66:0/1873555551 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3800cd80 con 0x7faa44101fb0 2024-04-21T09:05:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.795+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa441a00e0 con 0x7faa44101fb0 2024-04-21T09:05:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.795+0000 7faa41ffb640 1 -- 172.21.15.66:0/1873555551 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa380035f0 con 0x7faa44101fb0 2024-04-21T09:05:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.795+0000 7faa41ffb640 1 -- 172.21.15.66:0/1873555551 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38010690 con 0x7faa44101fb0 2024-04-21T09:05:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.795+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa441a0680 con 0x7faa44101fb0 2024-04-21T09:05:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa41ffb640 1 -- 172.21.15.66:0/1873555551 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faa380037a0 con 0x7faa44101fb0 2024-04-21T09:05:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 Processor -- start 2024-04-21T09:05:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 -- start start 2024-04-21T09:05:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa41ffb640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7faa340a3bb0 0x7faa340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b5c40 0x55a9718b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b66e0 0x55a9718bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a971815b10 con 0x55a9718b5250 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9717b3240 con 0x55a9718b5c40 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a971811ad0 con 0x55a9718b66e0 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.797+0000 7faa4906f640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7faa340a3bb0 0x7faa340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa41ffb640 1 -- 172.21.15.66:0/1873555551 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7faa380d0b10 con 0x7faa44101fb0 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36752/0 (socket says 172.21.15.66:36752) 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 -- 172.21.15.66:0/705807611 learned_addr learned my addr 172.21.15.66:0/705807611 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 -- 172.21.15.66:0/705807611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b66e0 msgr2=0x55a9718bd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 --2- 172.21.15.66:0/705807611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b66e0 0x55a9718bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 -- 172.21.15.66:0/705807611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b5c40 msgr2=0x55a9718b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 --2- 172.21.15.66:0/705807611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b5c40 0x55a9718b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 -- 172.21.15.66:0/705807611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9715282d0 con 0x55a9718b5250 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4906f640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7faa340a3bb0 0x7faa340a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005fd0 tx=0x7faa3c00f730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa4886e640 1 --2- 172.21.15.66:0/705807611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718b5670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa38099580 tx=0x7faa3809c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b36a00291ad4eda server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa2effd640 1 -- 172.21.15.66:0/705807611 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3809d2b0 con 0x55a9718b5250 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa2effd640 1 -- 172.21.15.66:0/705807611 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3809cbb0 con 0x55a9718b5250 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.798+0000 7faa2effd640 1 -- 172.21.15.66:0/705807611 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa380bd110 con 0x55a9718b5250 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 -- 172.21.15.66:0/705807611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 msgr2=0x55a9718b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/705807611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718b5670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa38099580 tx=0x7faa3809c510 comp rx=0 tx=0).stop 2024-04-21T09:05:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 -- 172.21.15.66:0/705807611 shutdown_connections 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/705807611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b66e0 0x55a9718bd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/705807611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b5c40 0x55a9718b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/705807611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718b5670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 -- 172.21.15.66:0/705807611 >> 172.21.15.66:0/705807611 conn(0x55a9718acf40 msgr2=0x55a9718ab490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 -- 172.21.15.66:0/705807611 shutdown_connections 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 -- 172.21.15.66:0/705807611 wait complete. 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 -- start start 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.799+0000 7faa4d7a0740 1 Processor -- start 2024-04-21T09:05:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718bf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9718bd8c0 con 0x55a9718b5c40 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a97179e190 con 0x55a9718b66e0 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a9718bda40 con 0x55a9718b5250 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 10 client.0 _ll_register_callbacks cb 0x55a9717998b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718bf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36756/0 (socket says 172.21.15.66:36756) 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 -- 172.21.15.66:0/1731196173 learned_addr learned my addr 172.21.15.66:0/1731196173 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4886e640 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4906f640 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 -- 172.21.15.66:0/1731196173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 msgr2=0x55a9718bf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718bf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 -- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 msgr2=0x55a9718c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 -- 172.21.15.66:0/1731196173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9715f26b0 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa43fff640 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718bf810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa3002c840 tx=0x7faa30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792549c199871af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa23fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa23fff640 1 -- 172.21.15.66:0/1731196173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30043020 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa23fff640 1 -- 172.21.15.66:0/1731196173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7faa30005040 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.800+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa3809cbb0 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa23fff640 1 -- 172.21.15.66:0/1731196173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7faa30005d50 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 10 client.14748 fetch_fsmap learned FSMap version 17 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a9718c9c30 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa23fff640 1 -- 172.21.15.66:0/1731196173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3809cbb0 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa23fff640 1 -- 172.21.15.66:0/1731196173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7faa300035f0 con 0x55a9718b5c40 2024-04-21T09:05:53.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 10 client.14748 fetch_fsmap finished waiting for FSMap version 17 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 10 client.14748 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 20 client.14748 populate_metadata read hostname 'smithi066' 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9718cc330 0x55a9718cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 4 client.14748 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a971823530 con 0x55a9718cc330 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.801+0000 7faa4886e640 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9718cc330 0x55a9718cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.802+0000 7faa4886e640 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9718cc330 0x55a9718cd7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e09d20ed5561e74 server_cookie=5362b0ec9b1185fe in_seq=0 out_seq=0 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.802+0000 7faa23fff640 10 client.14748 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa23fff640 1 -- 172.21.15.66:0/1731196173 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a971823530 con 0x55a9718cc330 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa23fff640 10 client.14748 handle_command_reply: tid=0 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 client.14748 shutdown 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 msgr2=0x55a9718bf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718bf810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa3002c840 tx=0x7faa30004270 comp rx=0 tx=0).stop 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 shutdown_connections 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9718cc330 0x55a9718cd7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718bf810 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/1731196173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718bf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 >> 172.21.15.66:0/1731196173 conn(0x55a9718acf40 msgr2=0x55a9718aaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa23fff640 0 client.14748 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 shutdown_connections 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 1 -- 172.21.15.66:0/1731196173 wait complete. 2024-04-21T09:05:53.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.803+0000 7faa4d7a0740 20 client.14748 trim_cache size 0 max 16384 2024-04-21T09:05:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 Processor -- start 2024-04-21T09:05:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 -- start start 2024-04-21T09:05:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5c40 0x55a9718bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9718cbae0 con 0x55a9718b5250 2024-04-21T09:05:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9718cc1b0 con 0x55a9718b66e0 2024-04-21T09:05:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a9718cc880 con 0x55a9718b5c40 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4886e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4886e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718bd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36758/0 (socket says 172.21.15.66:36758) 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.821+0000 7faa4886e640 1 -- 172.21.15.66:0/3802796137 learned_addr learned my addr 172.21.15.66:0/3802796137 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa43fff640 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5c40 0x55a9718bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4886e640 1 -- 172.21.15.66:0/3802796137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5c40 msgr2=0x55a9718bd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4886e640 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5c40 0x55a9718bd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4886e640 1 -- 172.21.15.66:0/3802796137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 msgr2=0x55a9718c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4886e640 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4886e640 1 -- 172.21.15.66:0/3802796137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9718ac230 con 0x55a9718b5250 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa43fff640 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5c40 0x55a9718bd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4886e640 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718bd270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faa380058c0 tx=0x7faa3809c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7c0a276b0e8a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa2e7fc640 1 -- 172.21.15.66:0/3802796137 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa380b5070 con 0x55a9718b5250 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa2e7fc640 1 -- 172.21.15.66:0/3802796137 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa380a0f60 con 0x55a9718b5250 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa2e7fc640 1 -- 172.21.15.66:0/3802796137 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa380a55f0 con 0x55a9718b5250 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4d7a0740 1 -- 172.21.15.66:0/3802796137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 msgr2=0x55a9718bd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718bd270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faa380058c0 tx=0x7faa3809c640 comp rx=0 tx=0).stop 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4d7a0740 1 -- 172.21.15.66:0/3802796137 shutdown_connections 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.822+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5c40 0x55a9718bd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/3802796137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5250 0x55a9718bd270 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 -- 172.21.15.66:0/3802796137 >> 172.21.15.66:0/3802796137 conn(0x55a9718acf40 msgr2=0x55a9718b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 -- 172.21.15.66:0/3802796137 shutdown_connections 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 -- 172.21.15.66:0/3802796137 wait complete. 2024-04-21T09:05:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 -- start start 2024-04-21T09:05:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 Processor -- start 2024-04-21T09:05:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9718cd620 con 0x55a9718b5c40 2024-04-21T09:05:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9718ccf50 con 0x55a9718b66e0 2024-04-21T09:05:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a9718cdcf0 con 0x55a9718b5250 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa4d7a0740 10 client.0 _ll_register_callbacks cb 0x7faa38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36774/0 (socket says 172.21.15.66:36774) 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.823+0000 7faa43fff640 1 -- 172.21.15.66:0/2160009685 learned_addr learned my addr 172.21.15.66:0/2160009685 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa4906f640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa4886e640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa43fff640 1 -- 172.21.15.66:0/2160009685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 msgr2=0x55a9718c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa43fff640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718c27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa43fff640 1 -- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 msgr2=0x55a9718c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:05:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa43fff640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa43fff640 1 -- 172.21.15.66:0/2160009685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9718c57c0 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa4906f640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa43fff640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718c4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa3002c170 tx=0x7faa30003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8883d518ec53888 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa23fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa23fff640 1 -- 172.21.15.66:0/2160009685 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30056020 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa23fff640 1 -- 172.21.15.66:0/2160009685 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7faa30005650 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa380a0f60 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa23fff640 1 -- 172.21.15.66:0/2160009685 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7faa3005e050 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa4d7a0740 10 client.14754 fetch_fsmap learned FSMap version 17 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a9718c6a60 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.824+0000 7faa23fff640 1 -- 172.21.15.66:0/2160009685 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa380a0f60 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa23fff640 1 -- 172.21.15.66:0/2160009685 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7faa30052560 con 0x55a9718b5c40 2024-04-21T09:05:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4d7a0740 10 client.14754 fetch_fsmap finished waiting for FSMap version 17 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4d7a0740 10 client.14754 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4d7a0740 20 client.14754 populate_metadata read hostname 'smithi066' 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a971936a90 0x55a9719335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4d7a0740 4 client.14754 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "status"}) -- 0x55a97181d3a0 con 0x55a971936a90 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4886e640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a971936a90 0x55a9719335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa4886e640 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a971936a90 0x55a9719335a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc34da7b734385a server_cookie=48aba7ed588e22a5 in_seq=0 out_seq=0 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.825+0000 7faa23fff640 10 client.14754 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa23fff640 1 -- 172.21.15.66:0/2160009685 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55a97181d3a0 con 0x55a971936a90 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa23fff640 10 client.14754 handle_command_reply: tid=0 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 client.14754 shutdown 2024-04-21T09:05:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 msgr2=0x55a9718c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718c4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa3002c170 tx=0x7faa30003ed0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 shutdown_connections 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a971936a90 0x55a9719335a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9718b66e0 0x55a9718c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa23fff640 0 client.14754 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9718b5c40 0x55a9718c4d40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 --2- 172.21.15.66:0/2160009685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9718b5250 0x55a9718c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 >> 172.21.15.66:0/2160009685 conn(0x55a9718acf40 msgr2=0x55a9718aa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 shutdown_connections 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 1 -- 172.21.15.66:0/2160009685 wait complete. 2024-04-21T09:05:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4d7a0740 20 client.14754 trim_cache size 0 max 16384 2024-04-21T09:05:53.839 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:05:53.839 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:05:53.839 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-04-21T09:05:53.839 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24512, 2024-04-21T09:05:53.839 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:05:53.839 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:05:53.840 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:05:53.840 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.8617921839999996, 2024-04-21T09:05:53.840 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 17, 2024-04-21T09:05:53.840 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 91, 2024-04-21T09:05:53.840 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T09:05:53.840 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 12.961658554 2024-04-21T09:05:53.840 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:05:53.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.826+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7faa340a3bb0 msgr2=0x7faa340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7faa340a3bb0 0x7faa340a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005fd0 tx=0x7faa3c00f730 comp rx=0 tx=0).stop 2024-04-21T09:05:53.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa44101fb0 msgr2=0x7faa441981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:53.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa44101fb0 0x7faa441981d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa3800e9c0 tx=0x7faa3800ee80 comp rx=0 tx=0).stop 2024-04-21T09:05:53.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 shutdown_connections 2024-04-21T09:05:53.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7faa340a3bb0 0x7faa340a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103b60 0x7faa4419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031a0 0x7faa44198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 --2- 172.21.15.66:0/1873555551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa44101fb0 0x7faa441981d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:53.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 >> 172.21.15.66:0/1873555551 conn(0x7faa440fd520 msgr2=0x7faa441065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:53.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 shutdown_connections 2024-04-21T09:05:53.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:53.827+0000 7faa4aad2640 1 -- 172.21.15.66:0/1873555551 wait complete. 2024-04-21T09:05:53.845 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T09:05:53.845 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:05:53.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T09:05:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:05:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:54 smithi186 ceph-mon[31505]: pgmap v297: 129 pgs: 2 creating+activating, 6 creating+peering, 109 active+clean, 12 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-21T09:05:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:54 smithi186 ceph-mon[31505]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T09:05:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:54 smithi186 ceph-mon[31505]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T09:05:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:54 smithi186 ceph-mon[31505]: Cluster is now healthy 2024-04-21T09:05:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:54 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1736853299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:54.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:54 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2568994407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:54.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:54 smithi066 ceph-mon[27953]: pgmap v297: 129 pgs: 2 creating+activating, 6 creating+peering, 109 active+clean, 12 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-21T09:05:54.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:54 smithi066 ceph-mon[27953]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T09:05:54.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:54 smithi066 ceph-mon[27953]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T09:05:54.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:54 smithi066 ceph-mon[27953]: Cluster is now healthy 2024-04-21T09:05:54.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:54 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1736853299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:54.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:54 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2568994407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:54.362 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:54 smithi083 ceph-mon[31413]: pgmap v297: 129 pgs: 2 creating+activating, 6 creating+peering, 109 active+clean, 12 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-21T09:05:54.362 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:54 smithi083 ceph-mon[31413]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T09:05:54.362 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:54 smithi083 ceph-mon[31413]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T09:05:54.362 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:54 smithi083 ceph-mon[31413]: Cluster is now healthy 2024-04-21T09:05:54.362 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:54 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1736853299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:05:54.362 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:54 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2568994407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:55.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.766+0000 7f042cf4b700 1 -- 172.21.15.66:0/3086520527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428101d20 msgr2=0x7f04281021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:55.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.766+0000 7f042cf4b700 1 --2- 172.21.15.66:0/3086520527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428101d20 0x7f04281021a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0418009990 tx=0x7f0418009ca0 comp rx=0 tx=0).stop 2024-04-21T09:05:55.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 -- 172.21.15.66:0/3086520527 shutdown_connections 2024-04-21T09:05:55.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 --2- 172.21.15.66:0/3086520527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04281026e0 0x7f042810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:55.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 --2- 172.21.15.66:0/3086520527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0428106990 0x7f0428106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:55.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 --2- 172.21.15.66:0/3086520527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0428101d20 0x7f04281021a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:55.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 -- 172.21.15.66:0/3086520527 >> 172.21.15.66:0/3086520527 conn(0x7f04280788f0 msgr2=0x7f0428078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:55.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 -- 172.21.15.66:0/3086520527 shutdown_connections 2024-04-21T09:05:55.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 -- 172.21.15.66:0/3086520527 wait complete. 2024-04-21T09:05:55.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 Processor -- start 2024-04-21T09:05:55.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.767+0000 7f042cf4b700 1 -- start start 2024-04-21T09:05:55.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042cf4b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0428101d20 0x7f04281035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:55.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042cf4b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04281026e0 0x7f0428103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:55.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042cf4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0428106990 0x7f0428104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:55.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042cf4b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0428111b90 con 0x7f04281026e0 2024-04-21T09:05:55.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042cf4b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0428111a10 con 0x7f0428106990 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042cf4b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0428111d10 con 0x7f0428101d20 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04281026e0 0x7f0428103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04281026e0 0x7f0428103b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59270/0 (socket says 172.21.15.66:59270) 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 -- 172.21.15.66:0/4291966808 learned_addr learned my addr 172.21.15.66:0/4291966808 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042659c700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0428101d20 0x7f04281035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0426d9d700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0428106990 0x7f0428104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 -- 172.21.15.66:0/4291966808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0428101d20 msgr2=0x7f04281035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0428101d20 0x7f04281035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 -- 172.21.15.66:0/4291966808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0428106990 msgr2=0x7f0428104060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0428106990 0x7f0428104060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 -- 172.21.15.66:0/4291966808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04281a7690 con 0x7f04281026e0 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f042659c700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0428101d20 0x7f04281035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:05:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.768+0000 7f0425d9b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04281026e0 0x7f0428103b20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0418005910 tx=0x7f0418009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.769+0000 7f041f7fe700 1 -- 172.21.15.66:0/4291966808 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418027070 con 0x7f04281026e0 2024-04-21T09:05:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.769+0000 7f041f7fe700 1 -- 172.21.15.66:0/4291966808 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0418003f20 con 0x7f04281026e0 2024-04-21T09:05:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.769+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04281a7920 con 0x7f04281026e0 2024-04-21T09:05:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.769+0000 7f041f7fe700 1 -- 172.21.15.66:0/4291966808 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418022c50 con 0x7f04281026e0 2024-04-21T09:05:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.769+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04281a7dd0 con 0x7f04281026e0 2024-04-21T09:05:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.770+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0428103080 con 0x7f04281026e0 2024-04-21T09:05:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.772+0000 7f041f7fe700 1 -- 172.21.15.66:0/4291966808 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f041802a070 con 0x7f04281026e0 2024-04-21T09:05:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.772+0000 7f041f7fe700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f04140a3cc0 0x7f04140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.772+0000 7f042659c700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f04140a3cc0 0x7f04140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.772+0000 7f041f7fe700 1 -- 172.21.15.66:0/4291966808 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f04180d1c10 con 0x7f04281026e0 2024-04-21T09:05:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.773+0000 7f042659c700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f04140a3cc0 0x7f04140a6180 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04100099d0 tx=0x7f0410008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:55.775+0000 7f041f7fe700 1 -- 172.21.15.66:0/4291966808 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0418031070 con 0x7f04281026e0 2024-04-21T09:05:56.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.032+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f042802df80 con 0x7f04281026e0 2024-04-21T09:05:56.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:56 smithi066 ceph-mon[27953]: pgmap v299: 129 pgs: 2 creating+activating, 6 creating+peering, 117 active+clean, 4 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-04-21T09:05:56.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.032+0000 7f041f7fe700 1 -- 172.21.15.66:0/4291966808 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f0418013070 con 0x7f04281026e0 2024-04-21T09:05:56.034 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f04140a3cc0 msgr2=0x7f04140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f04140a3cc0 0x7f04140a6180 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04100099d0 tx=0x7f0410008040 comp rx=0 tx=0).stop 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04281026e0 msgr2=0x7f0428103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04281026e0 0x7f0428103b20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0418005910 tx=0x7f0418009f90 comp rx=0 tx=0).stop 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 shutdown_connections 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f04140a3cc0 0x7f04140a6180 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0428106990 0x7f0428104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0428101d20 0x7f04281035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 --2- 172.21.15.66:0/4291966808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04281026e0 0x7f0428103b20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.034+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 >> 172.21.15.66:0/4291966808 conn(0x7f04280788f0 msgr2=0x7f042806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:56.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.035+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 shutdown_connections 2024-04-21T09:05:56.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56.035+0000 7f042cf4b700 1 -- 172.21.15.66:0/4291966808 wait complete. 2024-04-21T09:05:56.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:56 smithi186 ceph-mon[31505]: pgmap v299: 129 pgs: 2 creating+activating, 6 creating+peering, 117 active+clean, 4 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-04-21T09:05:56.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:56 smithi083 ceph-mon[31413]: pgmap v299: 129 pgs: 2 creating+activating, 6 creating+peering, 117 active+clean, 4 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-04-21T09:05:56.679 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-21T09:05:56.679 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T09:05:56.679 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-04-21T09:05:56.680 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-21T09:05:56.680 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:05:56.680 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-21T09:05:56.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T09:05:56.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T09:05:56.680 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T09:05:56.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:56.680 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T09:05:56.697 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42180sec preferred_lft 42180sec 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::37c:f2c2:71f0:1287/64 scope link noprefixroute 2024-04-21T09:05:56.698 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-21T09:05:56.699 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T09:05:56.699 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:05:56.699 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:05:56.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-04-21T09:05:56.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-04-21T09:05:56.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-04-21T09:05:56.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T09:05:56.699 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:05:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:05:56.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:05:56.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:56.896 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T09:05:56.920 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-04-21T09:05:56.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:56.922 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-04-21T09:05:56.990 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-04-21T09:05:56.990 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T09:05:56.990 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-21T09:05:56.990 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-21T09:05:56.990 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T09:05:56.991 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:05:56.991 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:05:56.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-21T09:05:56.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-21T09:05:56.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-21T09:05:56.991 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:05:57.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:05:57.165 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:57 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4291966808' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T09:05:57.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:05:57.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:57.271 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-21T09:05:57.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:57.287 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-04-21T09:05:57.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:57 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4291966808' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T09:05:57.342 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:05:57.342 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:05:57.342 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T09:05:57.343 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-21T09:05:57.343 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:05:57.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:57 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4291966808' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T09:05:57.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:05:57.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:05:57.459 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-21T09:05:57.460 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:05:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:05:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:05:57.584 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:05:57.584 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:05:57.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-04-21T09:05:57.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T09:05:57.584 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:05:57.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:05:57.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:05:57.661 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-21T09:05:57.661 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T09:05:57.661 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:05:57.676 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T09:05:57.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T09:05:57.676 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:05:57.731 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-04-21T09:05:57.796 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T09:05:57.852 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-04-21T09:05:57.852 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-04-21T09:05:57.852 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-04-21T09:05:57.852 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-04-21T09:05:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/6e6e8460c85900d354f99492666cf45a112a4c5e5d80ed4505d0d695213e4ebf/merged 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/28c2811ee65339a5f6133fcaf26e0478427064b9e5134460eb14662089134e0a/merged 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/0 2024-04-21T09:05:57.854 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/ebacaecddd897b0717be8c5e994f713a59d4b59b4b7cbec947de8056306d4257/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/2a64c64163cddf5406e2227fa5928cb0de234046eb8cf1bdd2bd03ab1dbd4c0d/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/87782a91e1792af1c9f3f137312fc27f25b7f0a2dcd2e14a0064fb2a445c47d2/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/50a23de37d368ae1a88b0b10fe7d274914c26f92e162b6ee414be1b1d59b2e1e/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/e20f8fb5db133b7fd76f44f49c63a760ed8e0091b627ba44d420e609eb3c7e15/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/8c6fbdd2a573cc394556328d9a55b214b5ee0521d1a07bb71891179e6c2c42cc/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b677b9811d912a62a75447145c4c165f37ad64285d7c9a01b156f4eb128b4fcf/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/bacf92df245ca9e132a8fde2aca28cef4652195e21fac7b75aff553fce0adadd/merged 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T09:05:57.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:57.856 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-04-21T09:05:57.907 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-21T09:05:57.907 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-21T09:05:57.949 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-04-21T09:05:57.973 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T09:05:58.030 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-04-21T09:05:58.030 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-04-21T09:05:58.030 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-04-21T09:05:58.030 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-04-21T09:05:58.030 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-04-21T09:05:58.030 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-04-21T09:05:58.030 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-04-21T09:05:58.031 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/6e6e8460c85900d354f99492666cf45a112a4c5e5d80ed4505d0d695213e4ebf/merged 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/28c2811ee65339a5f6133fcaf26e0478427064b9e5134460eb14662089134e0a/merged 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/0 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/ebacaecddd897b0717be8c5e994f713a59d4b59b4b7cbec947de8056306d4257/merged 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/2a64c64163cddf5406e2227fa5928cb0de234046eb8cf1bdd2bd03ab1dbd4c0d/merged 2024-04-21T09:05:58.032 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/87782a91e1792af1c9f3f137312fc27f25b7f0a2dcd2e14a0064fb2a445c47d2/merged 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/50a23de37d368ae1a88b0b10fe7d274914c26f92e162b6ee414be1b1d59b2e1e/merged 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/e20f8fb5db133b7fd76f44f49c63a760ed8e0091b627ba44d420e609eb3c7e15/merged 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/8c6fbdd2a573cc394556328d9a55b214b5ee0521d1a07bb71891179e6c2c42cc/merged 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b677b9811d912a62a75447145c4c165f37ad64285d7c9a01b156f4eb128b4fcf/merged 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/bacf92df245ca9e132a8fde2aca28cef4652195e21fac7b75aff553fce0adadd/merged 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T09:05:58.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:58.033 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-04-21T09:05:58.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:2024-04-21T09:05:58.065+0000 7f90f9b33480 -1 init, newargv = 0x55cf8e368e90 newargc=15 2024-04-21T09:05:58.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[69478]: starting ceph client 2024-04-21T09:05:58.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[69478]: starting fuse 2024-04-21T09:05:58.080 INFO:teuthology.orchestra.run.smithi066.stdout:115 2024-04-21T09:05:58.080 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-21T09:05:58.081 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.admin")) 2024-04-21T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-04-21T09:05:58.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:58 smithi066 ceph-mon[27953]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-21T09:05:58.198 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.admin.69478.asok 2024-04-21T09:05:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:05:58.209 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.69478.asok 2024-04-21T09:05:58.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:58.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.69478.asok status 2024-04-21T09:05:58.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:58 smithi186 ceph-mon[31505]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-21T09:05:58.346 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:05:58.346 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-04-21T09:05:58.346 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "a630765c1f93c6e61d9b40c4b76437611d5ef29e", 2024-04-21T09:05:58.346 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T09:05:58.346 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "admin", 2024-04-21T09:05:58.346 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-04-21T09:05:58.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "69478", 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 14766, 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 14766 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-04-21T09:05:58.347 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1338949376 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1338949376 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.14766 192.168.0.1:0/1338949376", 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1338949376", 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 17, 2024-04-21T09:05:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 91, 2024-04-21T09:05:58.349 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-04-21T09:05:58.349 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-04-21T09:05:58.349 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-04-21T09:05:58.349 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:05:58.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:58.355 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-04-21T09:05:58.355 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:05:58.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:58 smithi083 ceph-mon[31413]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-21T09:05:58.370 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-04-21T09:05:58.370 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:05:58.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:05:58.371 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:05:58.436 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-21T09:05:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T09:05:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:# file: . 2024-04-21T09:05:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-21T09:05:58.545 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:05:58.551 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:05:58.551 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T09:05:58.568 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:05:58.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 -- 172.21.15.66:0/4137344464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110103180 msgr2=0x7f7110103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 --2- 172.21.15.66:0/4137344464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110103180 0x7f7110103600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7100009900 tx=0x7f710002bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 -- 172.21.15.66:0/4137344464 shutdown_connections 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 --2- 172.21.15.66:0/4137344464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7110103b40 0x7f711010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 --2- 172.21.15.66:0/4137344464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110103180 0x7f7110103600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 --2- 172.21.15.66:0/4137344464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7110101f90 0x7f71101023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 -- 172.21.15.66:0/4137344464 >> 172.21.15.66:0/4137344464 conn(0x7f71100fd500 msgr2=0x7f71100ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 -- 172.21.15.66:0/4137344464 shutdown_connections 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 -- 172.21.15.66:0/4137344464 wait complete. 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 Processor -- start 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.709+0000 7f7116359640 1 -- start start 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f7116359640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110101f90 0x7f7110075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f7116359640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7110103180 0x7f7110074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f7116359640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7110103b40 0x7f7110074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f7116359640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f711010dad0 con 0x7f7110101f90 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f7116359640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f711010d950 con 0x7f7110103180 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f7116359640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f711010dc50 con 0x7f7110103b40 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110101f90 0x7f7110075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110101f90 0x7f7110075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59282/0 (socket says 172.21.15.66:59282) 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 -- 172.21.15.66:0/1043121044 learned_addr learned my addr 172.21.15.66:0/1043121044 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710f7fe640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7110103180 0x7f7110074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f71148f6640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7110103b40 0x7f7110074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 -- 172.21.15.66:0/1043121044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7110103b40 msgr2=0x7f7110074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7110103b40 0x7f7110074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 -- 172.21.15.66:0/1043121044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7110103180 msgr2=0x7f7110074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7110103180 0x7f7110074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 -- 172.21.15.66:0/1043121044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7110075110 con 0x7f7110101f90 2024-04-21T09:05:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.710+0000 7f710ffff640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110101f90 0x7f7110075d00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70f800d950 tx=0x7f70f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.711+0000 7f710d7fa640 1 -- 172.21.15.66:0/1043121044 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8017070 con 0x7f7110101f90 2024-04-21T09:05:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.711+0000 7f710d7fa640 1 -- 172.21.15.66:0/1043121044 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70f8009c30 con 0x7f7110101f90 2024-04-21T09:05:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.711+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71101a22a0 con 0x7f7110101f90 2024-04-21T09:05:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.711+0000 7f710d7fa640 1 -- 172.21.15.66:0/1043121044 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f800f620 con 0x7f7110101f90 2024-04-21T09:05:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.711+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71101a2770 con 0x7f7110101f90 2024-04-21T09:05:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.712+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71101092b0 con 0x7f7110101f90 2024-04-21T09:05:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.713+0000 7f710d7fa640 1 -- 172.21.15.66:0/1043121044 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f70f801a070 con 0x7f7110101f90 2024-04-21T09:05:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.714+0000 7f710d7fa640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.714+0000 7f710f7fe640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.714+0000 7f710d7fa640 1 -- 172.21.15.66:0/1043121044 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f70f80cffb0 con 0x7f7110101f90 2024-04-21T09:05:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.714+0000 7f710f7fe640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7100009230 tx=0x7f7100005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.717+0000 7f710d7fa640 1 -- 172.21.15.66:0/1043121044 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70f8009de0 con 0x7f7110101f90 2024-04-21T09:05:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.904+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f711002d5c0 con 0x7f7110101f90 2024-04-21T09:05:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.906+0000 7f710d7fa640 1 -- 172.21.15.66:0/1043121044 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7f70f809c100 con 0x7f7110101f90 2024-04-21T09:05:58.907 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:05:58.907 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:05:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70fc0a3bb0 msgr2=0x7f70fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7100009230 tx=0x7f7100005fd0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110101f90 msgr2=0x7f7110075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110101f90 0x7f7110075d00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70f800d950 tx=0x7f70f800de10 comp rx=0 tx=0).stop 2024-04-21T09:05:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 shutdown_connections 2024-04-21T09:05:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70fc0a3bb0 0x7f70fc0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7110103b40 0x7f7110074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7110103180 0x7f7110074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 --2- 172.21.15.66:0/1043121044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7110101f90 0x7f7110075d00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 >> 172.21.15.66:0/1043121044 conn(0x7f71100fd500 msgr2=0x7f7110106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 shutdown_connections 2024-04-21T09:05:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:58.908+0000 7f7116359640 1 -- 172.21.15.66:0/1043121044 wait complete. 2024-04-21T09:05:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:05:58.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:05:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 -- 172.21.15.66:0/899491616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4107cf0 msgr2=0x7f8ce410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/899491616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4107cf0 0x7f8ce410a0e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009ca0 tx=0x7f8cdc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 -- 172.21.15.66:0/899491616 shutdown_connections 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/899491616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4107cf0 0x7f8ce410a0e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/899491616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce41053c0 0x7f8ce41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/899491616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce4102a90 0x7f8ce4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 -- 172.21.15.66:0/899491616 >> 172.21.15.66:0/899491616 conn(0x7f8ce40fc440 msgr2=0x7f8ce40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 -- 172.21.15.66:0/899491616 shutdown_connections 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 -- 172.21.15.66:0/899491616 wait complete. 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.028+0000 7f8ce8a55640 1 Processor -- start 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 -- start start 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4102a90 0x7f8ce4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce41053c0 0x7f8ce4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce4107cf0 0x7f8ce419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410daf0 con 0x7f8ce4102a90 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410d970 con 0x7f8ce41053c0 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410dc70 con 0x7f8ce4107cf0 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4102a90 0x7f8ce4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4102a90 0x7f8ce4198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59300/0 (socket says 172.21.15.66:59300) 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 -- 172.21.15.66:0/2212281498 learned_addr learned my addr 172.21.15.66:0/2212281498 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce2d9d640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce4107cf0 0x7f8ce419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce1d9b640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce41053c0 0x7f8ce4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 -- 172.21.15.66:0/2212281498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce4107cf0 msgr2=0x7f8ce419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce4107cf0 0x7f8ce419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 -- 172.21.15.66:0/2212281498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce41053c0 msgr2=0x7f8ce4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce41053c0 0x7f8ce4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 -- 172.21.15.66:0/2212281498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce419fe60 con 0x7f8ce4102a90 2024-04-21T09:05:59.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce259c640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4102a90 0x7f8ce4198240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400d950 tx=0x7f8cd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8cd37fe640 1 -- 172.21.15.66:0/2212281498 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4017030 con 0x7f8ce4102a90 2024-04-21T09:05:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce41a0150 con 0x7f8ce4102a90 2024-04-21T09:05:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.029+0000 7f8cd37fe640 1 -- 172.21.15.66:0/2212281498 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd4004020 con 0x7f8ce4102a90 2024-04-21T09:05:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.030+0000 7f8cd37fe640 1 -- 172.21.15.66:0/2212281498 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40047a0 con 0x7f8ce4102a90 2024-04-21T09:05:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.030+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce41a06a0 con 0x7f8ce4102a90 2024-04-21T09:05:59.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.031+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce40fdf30 con 0x7f8ce4102a90 2024-04-21T09:05:59.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.032+0000 7f8cd37fe640 1 -- 172.21.15.66:0/2212281498 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8cd4009a30 con 0x7f8ce4102a90 2024-04-21T09:05:59.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.032+0000 7f8cd37fe640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ccc0a3bb0 0x7f8ccc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.033+0000 7f8cd37fe640 1 -- 172.21.15.66:0/2212281498 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f8cd40d0050 con 0x7f8ce4102a90 2024-04-21T09:05:59.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.033+0000 7f8ce1d9b640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ccc0a3bb0 0x7f8ccc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.033+0000 7f8ce1d9b640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ccc0a3bb0 0x7f8ccc0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4199750 tx=0x7f8cd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.037+0000 7f8cd37fe640 1 -- 172.21.15.66:0/2212281498 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cd409c7f0 con 0x7f8ce4102a90 2024-04-21T09:05:59.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.222+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ce4062b50 con 0x7f8ce4102a90 2024-04-21T09:05:59.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.223+0000 7f8cd37fe640 1 -- 172.21.15.66:0/2212281498 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f8cd409c0f0 con 0x7f8ce4102a90 2024-04-21T09:05:59.224 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:05:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ccc0a3bb0 msgr2=0x7f8ccc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ccc0a3bb0 0x7f8ccc0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4199750 tx=0x7f8cd8005f90 comp rx=0 tx=0).stop 2024-04-21T09:05:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4102a90 msgr2=0x7f8ce4198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4102a90 0x7f8ce4198240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400d950 tx=0x7f8cd400de10 comp rx=0 tx=0).stop 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 shutdown_connections 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ccc0a3bb0 0x7f8ccc0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce4107cf0 0x7f8ce419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce41053c0 0x7f8ce4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 --2- 172.21.15.66:0/2212281498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ce4102a90 0x7f8ce4198240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 >> 172.21.15.66:0/2212281498 conn(0x7f8ce40fc440 msgr2=0x7f8ce4103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 shutdown_connections 2024-04-21T09:05:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.225+0000 7f8ce8a55640 1 -- 172.21.15.66:0/2212281498 wait complete. 2024-04-21T09:05:59.235 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:05:59 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1043121044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:59.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:05:59.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:05:59 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1043121044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.342+0000 7f9462606640 1 -- 172.21.15.66:0/737600600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 msgr2=0x7f945c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.342+0000 7f9462606640 1 --2- 172.21.15.66:0/737600600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c10a0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9454009ca0 tx=0x7f945402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.342+0000 7f9462606640 1 -- 172.21.15.66:0/737600600 shutdown_connections 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.342+0000 7f9462606640 1 --2- 172.21.15.66:0/737600600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c10a0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.342+0000 7f9462606640 1 --2- 172.21.15.66:0/737600600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c105340 0x7f945c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 --2- 172.21.15.66:0/737600600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945c102a10 0x7f945c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 -- 172.21.15.66:0/737600600 >> 172.21.15.66:0/737600600 conn(0x7f945c0fc400 msgr2=0x7f945c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 -- 172.21.15.66:0/737600600 shutdown_connections 2024-04-21T09:05:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 -- 172.21.15.66:0/737600600 wait complete. 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 Processor -- start 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 -- start start 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c102a10 0x7f945c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.343+0000 7f9462606640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945c105340 0x7f945c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9462606640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9462606640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f945c10dad0 con 0x7f945c107c70 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9462606640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f945c10d950 con 0x7f945c105340 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9462606640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f945c10dc50 con 0x7f945c102a10 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59316/0 (socket says 172.21.15.66:59316) 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 -- 172.21.15.66:0/3131453368 learned_addr learned my addr 172.21.15.66:0/3131453368 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f945bfff640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c102a10 0x7f945c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 -- 172.21.15.66:0/3131453368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c102a10 msgr2=0x7f945c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c102a10 0x7f945c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 -- 172.21.15.66:0/3131453368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945c105340 msgr2=0x7f945c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f945b7fe640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945c105340 0x7f945c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945c105340 0x7f945c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 -- 172.21.15.66:0/3131453368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c19fdf0 con 0x7f945c107c70 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9460ba3640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c19f780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f945402c700 tx=0x7f94540093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f94597fa640 1 -- 172.21.15.66:0/3131453368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454041020 con 0x7f945c107c70 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c1a0080 con 0x7f945c107c70 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f94597fa640 1 -- 172.21.15.66:0/3131453368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9454034d60 con 0x7f945c107c70 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c1a0570 con 0x7f945c107c70 2024-04-21T09:05:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.344+0000 7f94597fa640 1 -- 172.21.15.66:0/3131453368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454039040 con 0x7f945c107c70 2024-04-21T09:05:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.345+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945c0fdef0 con 0x7f945c107c70 2024-04-21T09:05:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.347+0000 7f94597fa640 1 -- 172.21.15.66:0/3131453368 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9454044020 con 0x7f945c107c70 2024-04-21T09:05:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.347+0000 7f94597fa640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f94440a3bb0 0x7f94440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.348+0000 7f945bfff640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f94440a3bb0 0x7f94440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.348+0000 7f94597fa640 1 -- 172.21.15.66:0/3131453368 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f94540f2640 con 0x7f945c107c70 2024-04-21T09:05:59.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.348+0000 7f945bfff640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f94440a3bb0 0x7f94440a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9450009aa0 tx=0x7f9450005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.352+0000 7f94597fa640 1 -- 172.21.15.66:0/3131453368 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f945404b020 con 0x7f945c107c70 2024-04-21T09:05:59.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:05:59 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1043121044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:05:59.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.539+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f945c062b50 con 0x7f945c107c70 2024-04-21T09:05:59.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.540+0000 7f94597fa640 1 -- 172.21.15.66:0/3131453368 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f94540bede0 con 0x7f945c107c70 2024-04-21T09:05:59.540 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:05:59.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.541+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f94440a3bb0 msgr2=0x7f94440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.541+0000 7f9462606640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f94440a3bb0 0x7f94440a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9450009aa0 tx=0x7f9450005f90 comp rx=0 tx=0).stop 2024-04-21T09:05:59.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 msgr2=0x7f945c19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c19f780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f945402c700 tx=0x7f94540093f0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 shutdown_connections 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f94440a3bb0 0x7f94440a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c107c70 0x7f945c19f780 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945c105340 0x7f945c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 --2- 172.21.15.66:0/3131453368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c102a10 0x7f945c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 >> 172.21.15.66:0/3131453368 conn(0x7f945c0fc400 msgr2=0x7f945c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 shutdown_connections 2024-04-21T09:05:59.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.542+0000 7f9462606640 1 -- 172.21.15.66:0/3131453368 wait complete. 2024-04-21T09:05:59.553 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:05:59.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 -- 172.21.15.66:0/2843130462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 msgr2=0x7f6868107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 --2- 172.21.15.66:0/2843130462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868107790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 -- 172.21.15.66:0/2843130462 shutdown_connections 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 --2- 172.21.15.66:0/2843130462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6868107cd0 0x7f686810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 --2- 172.21.15.66:0/2843130462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868107790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 --2- 172.21.15.66:0/2843130462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6868102a70 0x7f6868104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 -- 172.21.15.66:0/2843130462 >> 172.21.15.66:0/2843130462 conn(0x7f68680fc440 msgr2=0x7f68680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 -- 172.21.15.66:0/2843130462 shutdown_connections 2024-04-21T09:05:59.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 -- 172.21.15.66:0/2843130462 wait complete. 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.663+0000 7f686dbfb640 1 Processor -- start 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f686dbfb640 1 -- start start 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f686dbfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6868102a70 0x7f6868198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f686dbfb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f686dbfb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6868107cd0 0x7f686819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f686dbfb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f686810daf0 con 0x7f68681053a0 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f686dbfb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f686810d970 con 0x7f6868102a70 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f686dbfb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f686810dc70 con 0x7f6868107cd0 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59330/0 (socket says 172.21.15.66:59330) 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 -- 172.21.15.66:0/1106400139 learned_addr learned my addr 172.21.15.66:0/1106400139 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:59.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6867fff640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6868107cd0 0x7f686819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f68677fe640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6868102a70 0x7f6868198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 -- 172.21.15.66:0/1106400139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6868107cd0 msgr2=0x7f686819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6868107cd0 0x7f686819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 -- 172.21.15.66:0/1106400139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6868102a70 msgr2=0x7f6868198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6868102a70 0x7f6868198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.664+0000 7f6866ffd640 1 -- 172.21.15.66:0/1106400139 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686819fd50 con 0x7f68681053a0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.665+0000 7f6866ffd640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868198670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f685c036040 tx=0x7f685c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.665+0000 7f6864ff9640 1 -- 172.21.15.66:0/1106400139 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c043020 con 0x7f68681053a0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.665+0000 7f6864ff9640 1 -- 172.21.15.66:0/1106400139 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685c004aa0 con 0x7f68681053a0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.665+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686819ffe0 con 0x7f68681053a0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.665+0000 7f6864ff9640 1 -- 172.21.15.66:0/1106400139 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c003de0 con 0x7f68681053a0 2024-04-21T09:05:59.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.665+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68681a04a0 con 0x7f68681053a0 2024-04-21T09:05:59.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.666+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68680fdf30 con 0x7f68681053a0 2024-04-21T09:05:59.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.667+0000 7f6864ff9640 1 -- 172.21.15.66:0/1106400139 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f685c046020 con 0x7f68681053a0 2024-04-21T09:05:59.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.667+0000 7f6864ff9640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f68500a3bb0 0x7f68500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.668+0000 7f68677fe640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f68500a3bb0 0x7f68500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.668+0000 7f6864ff9640 1 -- 172.21.15.66:0/1106400139 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f685c0f2af0 con 0x7f68681053a0 2024-04-21T09:05:59.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.668+0000 7f68677fe640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f68500a3bb0 0x7f68500a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6858005e10 tx=0x7f6858005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.672+0000 7f6864ff9640 1 -- 172.21.15.66:0/1106400139 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f685c04d020 con 0x7f68681053a0 2024-04-21T09:05:59.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.860+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6868062b50 con 0x7f68681053a0 2024-04-21T09:05:59.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.861+0000 7f6864ff9640 1 -- 172.21.15.66:0/1106400139 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f685c0bf290 con 0x7f68681053a0 2024-04-21T09:05:59.862 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:05:59.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f68500a3bb0 msgr2=0x7f68500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f68500a3bb0 0x7f68500a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6858005e10 tx=0x7f6858005da0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 msgr2=0x7f6868198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868198670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f685c036040 tx=0x7f685c0042f0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 shutdown_connections 2024-04-21T09:05:59.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f68500a3bb0 0x7f68500a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6868107cd0 0x7f686819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681053a0 0x7f6868198670 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 --2- 172.21.15.66:0/1106400139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6868102a70 0x7f6868198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 >> 172.21.15.66:0/1106400139 conn(0x7f68680fc440 msgr2=0x7f6868103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:59.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 shutdown_connections 2024-04-21T09:05:59.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.863+0000 7f686dbfb640 1 -- 172.21.15.66:0/1106400139 wait complete. 2024-04-21T09:05:59.875 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:05:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.980+0000 7f31746d2640 1 -- 172.21.15.66:0/2468327242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c102a10 msgr2=0x7f316c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.980+0000 7f31746d2640 1 --2- 172.21.15.66:0/2468327242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c102a10 0x7f316c104e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3164009900 tx=0x7f316402bc30 comp rx=0 tx=0).stop 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- 172.21.15.66:0/2468327242 shutdown_connections 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 --2- 172.21.15.66:0/2468327242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c107c70 0x7f316c10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 --2- 172.21.15.66:0/2468327242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c105340 0x7f316c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 --2- 172.21.15.66:0/2468327242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c102a10 0x7f316c104e00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- 172.21.15.66:0/2468327242 >> 172.21.15.66:0/2468327242 conn(0x7f316c0fc400 msgr2=0x7f316c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- 172.21.15.66:0/2468327242 shutdown_connections 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- 172.21.15.66:0/2468327242 wait complete. 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 Processor -- start 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- start start 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c102a10 0x7f316c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c105340 0x7f316c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107c70 0x7f316c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10dae0 con 0x7f316c107c70 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10d960 con 0x7f316c105340 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.981+0000 7f31746d2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10dc60 con 0x7f316c102a10 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107c70 0x7f316c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107c70 0x7f316c19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59346/0 (socket says 172.21.15.66:59346) 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 -- 172.21.15.66:0/2997271363 learned_addr learned my addr 172.21.15.66:0/2997271363 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3171c6d640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c105340 0x7f316c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f317246e640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c102a10 0x7f316c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 -- 172.21.15.66:0/2997271363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c102a10 msgr2=0x7f316c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c102a10 0x7f316c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 -- 172.21.15.66:0/2997271363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c105340 msgr2=0x7f316c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c105340 0x7f316c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:05:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 -- 172.21.15.66:0/2997271363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c19fe60 con 0x7f316c107c70 2024-04-21T09:05:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f3172c6f640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107c70 0x7f316c19f7f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f316800eb10 tx=0x7f316800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f31637fe640 1 -- 172.21.15.66:0/2997271363 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168004b20 con 0x7f316c107c70 2024-04-21T09:05:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f31637fe640 1 -- 172.21.15.66:0/2997271363 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3168004cc0 con 0x7f316c107c70 2024-04-21T09:05:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c1a0150 con 0x7f316c107c70 2024-04-21T09:05:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f31637fe640 1 -- 172.21.15.66:0/2997271363 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168010670 con 0x7f316c107c70 2024-04-21T09:05:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.982+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a06f0 con 0x7f316c107c70 2024-04-21T09:05:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.983+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c0fdef0 con 0x7f316c107c70 2024-04-21T09:05:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.984+0000 7f31637fe640 1 -- 172.21.15.66:0/2997271363 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3168005020 con 0x7f316c107c70 2024-04-21T09:05:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.985+0000 7f31637fe640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f31580a3c30 0x7f31580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:05:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.985+0000 7f317246e640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f31580a3c30 0x7f31580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:05:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.985+0000 7f31637fe640 1 -- 172.21.15.66:0/2997271363 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f31680d0bb0 con 0x7f316c107c70 2024-04-21T09:05:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.986+0000 7f317246e640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f31580a3c30 0x7f31580a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3164000c00 tx=0x7f3164005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:05:59.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:05:59.990+0000 7f31637fe640 1 -- 172.21.15.66:0/2997271363 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f316809d350 con 0x7f316c107c70 2024-04-21T09:06:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.177+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f316c062b50 con 0x7f316c107c70 2024-04-21T09:06:00.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.177+0000 7f31637fe640 1 -- 172.21.15.66:0/2997271363 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f316809cc50 con 0x7f316c107c70 2024-04-21T09:06:00.178 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:06:00.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f31580a3c30 msgr2=0x7f31580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f31580a3c30 0x7f31580a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3164000c00 tx=0x7f3164005be0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107c70 msgr2=0x7f316c19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107c70 0x7f316c19f7f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f316800eb10 tx=0x7f316800c2d0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 shutdown_connections 2024-04-21T09:06:00.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f31580a3c30 0x7f31580a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107c70 0x7f316c19f7f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c105340 0x7f316c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 --2- 172.21.15.66:0/2997271363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c102a10 0x7f316c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 >> 172.21.15.66:0/2997271363 conn(0x7f316c0fc400 msgr2=0x7f316c1048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:00.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.179+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 shutdown_connections 2024-04-21T09:06:00.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.180+0000 7f31746d2640 1 -- 172.21.15.66:0/2997271363 wait complete. 2024-04-21T09:06:00.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:00 smithi066 ceph-mon[27953]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-21T09:06:00.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:00 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2212281498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:00 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3131453368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:00 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1106400139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.191 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T09:06:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- 172.21.15.66:0/4188972802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf4107cd0 msgr2=0x7f0cf410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/4188972802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf4107cd0 0x7f0cf410a0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009ca0 tx=0x7f0cec02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- 172.21.15.66:0/4188972802 shutdown_connections 2024-04-21T09:06:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/4188972802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf4107cd0 0x7f0cf410a0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/4188972802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cf41053a0 0x7f0cf4107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/4188972802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf4102a70 0x7f0cf4104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- 172.21.15.66:0/4188972802 >> 172.21.15.66:0/4188972802 conn(0x7f0cf40fc400 msgr2=0x7f0cf40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- 172.21.15.66:0/4188972802 shutdown_connections 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- 172.21.15.66:0/4188972802 wait complete. 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 Processor -- start 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- start start 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cf4102a70 0x7f0cf4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf41053a0 0x7f0cf41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf4107cd0 0x7f0cf419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410dac0 con 0x7f0cf41053a0 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410d940 con 0x7f0cf4107cd0 2024-04-21T09:06:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.295+0000 7f0cfc29a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410dc40 con 0x7f0cf4102a70 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf41053a0 0x7f0cf41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf41053a0 0x7f0cf41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59352/0 (socket says 172.21.15.66:59352) 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 -- 172.21.15.66:0/831852053 learned_addr learned my addr 172.21.15.66:0/831852053 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cfa837640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf4107cd0 0x7f0cf419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cfa036640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cf4102a70 0x7f0cf4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 -- 172.21.15.66:0/831852053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cf4102a70 msgr2=0x7f0cf4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cf4102a70 0x7f0cf4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 -- 172.21.15.66:0/831852053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf4107cd0 msgr2=0x7f0cf419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf4107cd0 0x7f0cf419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 -- 172.21.15.66:0/831852053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf419fd90 con 0x7f0cf41053a0 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.296+0000 7f0cf9835640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf41053a0 0x7f0cf41986b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c970 tx=0x7f0ce400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.297+0000 7f0ceb7fe640 1 -- 172.21.15.66:0/831852053 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4014070 con 0x7f0cf41053a0 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.297+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41a0080 con 0x7f0cf41053a0 2024-04-21T09:06:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.297+0000 7f0ceb7fe640 1 -- 172.21.15.66:0/831852053 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce4004440 con 0x7f0cf41053a0 2024-04-21T09:06:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.297+0000 7f0ceb7fe640 1 -- 172.21.15.66:0/831852053 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4003e50 con 0x7f0cf41053a0 2024-04-21T09:06:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.297+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a05d0 con 0x7f0cf41053a0 2024-04-21T09:06:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.298+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf40fdef0 con 0x7f0cf41053a0 2024-04-21T09:06:00.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.299+0000 7f0ceb7fe640 1 -- 172.21.15.66:0/831852053 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ce4017070 con 0x7f0cf41053a0 2024-04-21T09:06:00.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.300+0000 7f0ceb7fe640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0ce00a3d50 0x7f0ce00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.300+0000 7f0cfa036640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0ce00a3d50 0x7f0ce00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.300+0000 7f0ceb7fe640 1 -- 172.21.15.66:0/831852053 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f0ce40d01e0 con 0x7f0cf41053a0 2024-04-21T09:06:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.300+0000 7f0cfa036640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0ce00a3d50 0x7f0ce00a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0005fd0 tx=0x7f0cf0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:00.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.304+0000 7f0ceb7fe640 1 -- 172.21.15.66:0/831852053 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ce401e030 con 0x7f0cf41053a0 2024-04-21T09:06:00.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:00 smithi186 ceph-mon[31505]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-21T09:06:00.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:00 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2212281498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:00 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3131453368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:00 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1106400139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:00 smithi083 ceph-mon[31413]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-21T09:06:00.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:00 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2212281498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:00 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3131453368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:00 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1106400139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:06:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.504+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0cf402d5c0 con 0x7f0cf41053a0 2024-04-21T09:06:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.505+0000 7f0ceb7fe640 1 -- 172.21.15.66:0/831852053 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+588 (secure 0 0 0) 0x7f0ce409c980 con 0x7f0cf41053a0 2024-04-21T09:06:00.505 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/612283217 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:00.505 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/763565357 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:00.505 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3624224927 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2187684654 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3374606173 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/763565357 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/502604018 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3624224927 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2365699589 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3868723687 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:00.506 INFO:teuthology.orchestra.run.smithi066.stderr:listed 10 entries 2024-04-21T09:06:00.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0ce00a3d50 msgr2=0x7f0ce00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0ce00a3d50 0x7f0ce00a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0005fd0 tx=0x7f0cf0005850 comp rx=0 tx=0).stop 2024-04-21T09:06:00.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf41053a0 msgr2=0x7f0cf41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf41053a0 0x7f0cf41986b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c970 tx=0x7f0ce400ce30 comp rx=0 tx=0).stop 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 shutdown_connections 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0ce00a3d50 0x7f0ce00a6210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf4107cd0 0x7f0cf419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cf41053a0 0x7f0cf41986b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 --2- 172.21.15.66:0/831852053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cf4102a70 0x7f0cf4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 >> 172.21.15.66:0/831852053 conn(0x7f0cf40fc400 msgr2=0x7f0cf41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 shutdown_connections 2024-04-21T09:06:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.507+0000 7f0cfc29a640 1 -- 172.21.15.66:0/831852053 wait complete. 2024-04-21T09:06:00.518 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:06:00.518 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T09:06:00.534 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:06:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 -- 172.21.15.66:0/774145798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 msgr2=0x7fcc6410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 --2- 172.21.15.66:0/774145798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc6410a0f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009ca0 tx=0x7fcc5c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 -- 172.21.15.66:0/774145798 shutdown_connections 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 --2- 172.21.15.66:0/774145798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc6410a0f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 --2- 172.21.15.66:0/774145798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc641053d0 0x7fcc641077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 --2- 172.21.15.66:0/774145798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc64102aa0 0x7fcc64104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 -- 172.21.15.66:0/774145798 >> 172.21.15.66:0/774145798 conn(0x7fcc640fc470 msgr2=0x7fcc640fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 -- 172.21.15.66:0/774145798 shutdown_connections 2024-04-21T09:06:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 -- 172.21.15.66:0/774145798 wait complete. 2024-04-21T09:06:00.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.675+0000 7fcc69206640 1 Processor -- start 2024-04-21T09:06:00.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc69206640 1 -- start start 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc69206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc64102aa0 0x7fcc64102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc69206640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc641053d0 0x7fcc64100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc69206640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc64100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc69206640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410db00 con 0x7fcc64107d00 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc69206640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410d980 con 0x7fcc64102aa0 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc69206640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410dc80 con 0x7fcc641053d0 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc64100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc64100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59370/0 (socket says 172.21.15.66:59370) 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 -- 172.21.15.66:0/646074587 learned_addr learned my addr 172.21.15.66:0/646074587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6259c640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc641053d0 0x7fcc64100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc62d9d640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc64102aa0 0x7fcc64102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 -- 172.21.15.66:0/646074587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc641053d0 msgr2=0x7fcc64100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc641053d0 0x7fcc64100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 -- 172.21.15.66:0/646074587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc64102aa0 msgr2=0x7fcc64102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc64102aa0 0x7fcc64102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6359e640 1 -- 172.21.15.66:0/646074587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc64101820 con 0x7fcc64107d00 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc6259c640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc641053d0 0x7fcc64100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.676+0000 7fcc62d9d640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc64102aa0 0x7fcc64102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.677+0000 7fcc6359e640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc64100f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c02c700 tx=0x7fcc5c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.677+0000 7fcc5bfff640 1 -- 172.21.15.66:0/646074587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c039040 con 0x7fcc64107d00 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.677+0000 7fcc5bfff640 1 -- 172.21.15.66:0/646074587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5c044070 con 0x7fcc64107d00 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.677+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc641a2260 con 0x7fcc64107d00 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.677+0000 7fcc5bfff640 1 -- 172.21.15.66:0/646074587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0042a0 con 0x7fcc64107d00 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.677+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641a27b0 con 0x7fcc64107d00 2024-04-21T09:06:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.678+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc640fdf60 con 0x7fcc64107d00 2024-04-21T09:06:00.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.680+0000 7fcc5bfff640 1 -- 172.21.15.66:0/646074587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc5c003970 con 0x7fcc64107d00 2024-04-21T09:06:00.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.680+0000 7fcc5bfff640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcc500a3bb0 0x7fcc500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.680+0000 7fcc62d9d640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcc500a3bb0 0x7fcc500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.680+0000 7fcc5bfff640 1 -- 172.21.15.66:0/646074587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fcc5c0f3550 con 0x7fcc64107d00 2024-04-21T09:06:00.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.681+0000 7fcc62d9d640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcc500a3bb0 0x7fcc500a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c005d00 tx=0x7fcc4c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.684+0000 7fcc5bfff640 1 -- 172.21.15.66:0/646074587 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc5c0bfcf0 con 0x7fcc64107d00 2024-04-21T09:06:00.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.870+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc64062b50 con 0x7fcc64107d00 2024-04-21T09:06:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.872+0000 7fcc5bfff640 1 -- 172.21.15.66:0/646074587 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7fcc5c0bf5f0 con 0x7fcc64107d00 2024-04-21T09:06:00.872 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:00.873 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcc500a3bb0 msgr2=0x7fcc500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcc500a3bb0 0x7fcc500a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c005d00 tx=0x7fcc4c005c70 comp rx=0 tx=0).stop 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 msgr2=0x7fcc64100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc64100f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c02c700 tx=0x7fcc5c034760 comp rx=0 tx=0).stop 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 shutdown_connections 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcc500a3bb0 0x7fcc500a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc64107d00 0x7fcc64100f90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc641053d0 0x7fcc64100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 --2- 172.21.15.66:0/646074587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc64102aa0 0x7fcc64102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 >> 172.21.15.66:0/646074587 conn(0x7fcc640fc470 msgr2=0x7fcc641033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 shutdown_connections 2024-04-21T09:06:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.874+0000 7fcc69206640 1 -- 172.21.15.66:0/646074587 wait complete. 2024-04-21T09:06:00.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:06:00.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:06:00.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 -- 172.21.15.66:0/1249660270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 msgr2=0x7f0010103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 --2- 172.21.15.66:0/1249660270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010103610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efff8009900 tx=0x7efff802bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 -- 172.21.15.66:0/1249660270 shutdown_connections 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 --2- 172.21.15.66:0/1249660270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0010103b50 0x7f001010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 --2- 172.21.15.66:0/1249660270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010103610 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 --2- 172.21.15.66:0/1249660270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0010101fa0 0x7f00101023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 -- 172.21.15.66:0/1249660270 >> 172.21.15.66:0/1249660270 conn(0x7f00100fd530 msgr2=0x7f00100ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 -- 172.21.15.66:0/1249660270 shutdown_connections 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 -- 172.21.15.66:0/1249660270 wait complete. 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 Processor -- start 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.990+0000 7f001705c640 1 -- start start 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f001705c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0010101fa0 0x7f0010198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f001705c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f001705c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0010103b50 0x7f001019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f001705c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f001010dae0 con 0x7f0010103190 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f001705c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f001010d960 con 0x7f0010101fa0 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f001705c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f001010dc60 con 0x7f0010103b50 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59374/0 (socket says 172.21.15.66:59374) 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 -- 172.21.15.66:0/3044403097 learned_addr learned my addr 172.21.15.66:0/3044403097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f00155f9640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0010103b50 0x7f001019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f0014df8640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0010101fa0 0x7f0010198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 -- 172.21.15.66:0/3044403097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0010103b50 msgr2=0x7f001019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0010103b50 0x7f001019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 -- 172.21.15.66:0/3044403097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0010101fa0 msgr2=0x7f0010198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0010101fa0 0x7f0010198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 -- 172.21.15.66:0/3044403097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001019fe30 con 0x7f0010103190 2024-04-21T09:06:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f00155f9640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0010103b50 0x7f001019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f0014df8640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0010101fa0 0x7f0010198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000ffff640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010198750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efff8006010 tx=0x7efff8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000dffb640 1 -- 172.21.15.66:0/3044403097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8043020 con 0x7f0010103190 2024-04-21T09:06:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000dffb640 1 -- 172.21.15.66:0/3044403097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff8004440 con 0x7f0010103190 2024-04-21T09:06:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00101a00c0 con 0x7f0010103190 2024-04-21T09:06:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.991+0000 7f000dffb640 1 -- 172.21.15.66:0/3044403097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8003bf0 con 0x7f0010103190 2024-04-21T09:06:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.992+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00101a0610 con 0x7f0010103190 2024-04-21T09:06:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.993+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0010109430 con 0x7f0010103190 2024-04-21T09:06:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.994+0000 7f000dffb640 1 -- 172.21.15.66:0/3044403097 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efff8046020 con 0x7f0010103190 2024-04-21T09:06:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.995+0000 7f000dffb640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efffc0a3bb0 0x7efffc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.995+0000 7f000dffb640 1 -- 172.21.15.66:0/3044403097 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7efff80f30d0 con 0x7f0010103190 2024-04-21T09:06:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.995+0000 7f0014df8640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efffc0a3bb0 0x7efffc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.996+0000 7f0014df8640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efffc0a3bb0 0x7efffc0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0010102ff0 tx=0x7f0000005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:00.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:00.999+0000 7f000dffb640 1 -- 172.21.15.66:0/3044403097 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efff804d020 con 0x7f0010103190 2024-04-21T09:06:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.188+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f001002d810 con 0x7f0010103190 2024-04-21T09:06:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.188+0000 7f000dffb640 1 -- 172.21.15.66:0/3044403097 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7efff80bf870 con 0x7f0010103190 2024-04-21T09:06:01.189 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:06:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efffc0a3bb0 msgr2=0x7efffc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efffc0a3bb0 0x7efffc0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0010102ff0 tx=0x7f0000005e70 comp rx=0 tx=0).stop 2024-04-21T09:06:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 msgr2=0x7f0010198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010198750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efff8006010 tx=0x7efff8004040 comp rx=0 tx=0).stop 2024-04-21T09:06:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 shutdown_connections 2024-04-21T09:06:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7efffc0a3bb0 0x7efffc0a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0010103b50 0x7f001019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0010103190 0x7f0010198750 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.190+0000 7f001705c640 1 --2- 172.21.15.66:0/3044403097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0010101fa0 0x7f0010198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.191+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 >> 172.21.15.66:0/3044403097 conn(0x7f00100fd530 msgr2=0x7f0010106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.191+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 shutdown_connections 2024-04-21T09:06:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.191+0000 7f001705c640 1 -- 172.21.15.66:0/3044403097 wait complete. 2024-04-21T09:06:01.201 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:01 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2997271363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:06:01.201 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:01 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/831852053' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:06:01.201 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:01 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/646074587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:01.202 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:06:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.305+0000 7f760d92c640 1 -- 172.21.15.66:0/2904300260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 msgr2=0x7f7608101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.305+0000 7f760d92c640 1 --2- 172.21.15.66:0/2904300260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f7608101280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7600009ca0 tx=0x7f760002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.305+0000 7f760d92c640 1 -- 172.21.15.66:0/2904300260 shutdown_connections 2024-04-21T09:06:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.305+0000 7f760d92c640 1 --2- 172.21.15.66:0/2904300260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102a00 0x7f7608109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.305+0000 7f760d92c640 1 --2- 172.21.15.66:0/2904300260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7608102040 0x7f76081024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.305+0000 7f760d92c640 1 --2- 172.21.15.66:0/2904300260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f7608101280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 -- 172.21.15.66:0/2904300260 >> 172.21.15.66:0/2904300260 conn(0x7f76080fc440 msgr2=0x7f76080fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 -- 172.21.15.66:0/2904300260 shutdown_connections 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 -- 172.21.15.66:0/2904300260 wait complete. 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 Processor -- start 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 -- start start 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f760806b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7608102040 0x7f760819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102a00 0x7f760819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f760810c9f0 con 0x7f7608100e60 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f760810c870 con 0x7f7608102a00 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.306+0000 7f760d92c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f760810cb70 con 0x7f7608102040 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f760806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f760806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59382/0 (socket says 172.21.15.66:59382) 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 -- 172.21.15.66:0/3047669111 learned_addr learned my addr 172.21.15.66:0/3047669111 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f76077fe640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102a00 0x7f760819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f76067fc640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7608102040 0x7f760819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 -- 172.21.15.66:0/3047669111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7608102040 msgr2=0x7f760819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7608102040 0x7f760819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 -- 172.21.15.66:0/3047669111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102a00 msgr2=0x7f760819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102a00 0x7f760819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 -- 172.21.15.66:0/3047669111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760819fe60 con 0x7f7608100e60 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f76077fe640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102a00 0x7f760819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f76067fc640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7608102040 0x7f760819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f7606ffd640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f760806b120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7600032380 tx=0x7f76000347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f760c92a640 1 -- 172.21.15.66:0/3047669111 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600039040 con 0x7f7608100e60 2024-04-21T09:06:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76081a00f0 con 0x7f7608100e60 2024-04-21T09:06:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f760c92a640 1 -- 172.21.15.66:0/3047669111 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7600044070 con 0x7f7608100e60 2024-04-21T09:06:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f760c92a640 1 -- 172.21.15.66:0/3047669111 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000042a0 con 0x7f7608100e60 2024-04-21T09:06:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.307+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76081a05e0 con 0x7f7608100e60 2024-04-21T09:06:01.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.308+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76081082c0 con 0x7f7608100e60 2024-04-21T09:06:01.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.309+0000 7f760c92a640 1 -- 172.21.15.66:0/3047669111 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7600003a00 con 0x7f7608100e60 2024-04-21T09:06:01.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.310+0000 7f760c92a640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75f40a3bb0 0x7f75f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.310+0000 7f76067fc640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75f40a3bb0 0x7f75f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.310+0000 7f760c92a640 1 -- 172.21.15.66:0/3047669111 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f76000f2ab0 con 0x7f7608100e60 2024-04-21T09:06:01.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.310+0000 7f76067fc640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75f40a3bb0 0x7f75f40a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f760806ae90 tx=0x7f75f0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:01.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.315+0000 7f760c92a640 1 -- 172.21.15.66:0/3047669111 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76000bf250 con 0x7f7608100e60 2024-04-21T09:06:01.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:01 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2997271363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:06:01.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:01 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/831852053' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:06:01.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:01 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/646074587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:01.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:01 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2997271363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:06:01.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:01 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/831852053' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:06:01.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:01 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/646074587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:01.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.500+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7608062b50 con 0x7f7608100e60 2024-04-21T09:06:01.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.500+0000 7f760c92a640 1 -- 172.21.15.66:0/3047669111 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f76000beb50 con 0x7f7608100e60 2024-04-21T09:06:01.501 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:06:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75f40a3bb0 msgr2=0x7f75f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75f40a3bb0 0x7f75f40a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f760806ae90 tx=0x7f75f0005e50 comp rx=0 tx=0).stop 2024-04-21T09:06:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 msgr2=0x7f760806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f760806b120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7600032380 tx=0x7f76000347b0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 shutdown_connections 2024-04-21T09:06:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75f40a3bb0 0x7f75f40a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102a00 0x7f760819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7608102040 0x7f760819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.502+0000 7f760d92c640 1 --2- 172.21.15.66:0/3047669111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7608100e60 0x7f760806b120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.503+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 >> 172.21.15.66:0/3047669111 conn(0x7f76080fc440 msgr2=0x7f7608105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.503+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 shutdown_connections 2024-04-21T09:06:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.503+0000 7f760d92c640 1 -- 172.21.15.66:0/3047669111 wait complete. 2024-04-21T09:06:01.514 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:06:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 -- 172.21.15.66:0/878154165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c103b60 msgr2=0x7fb85c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 --2- 172.21.15.66:0/878154165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c103b60 0x7fb85c10a3f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb854009ca0 tx=0x7fb85402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 -- 172.21.15.66:0/878154165 shutdown_connections 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 --2- 172.21.15.66:0/878154165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c103b60 0x7fb85c10a3f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 --2- 172.21.15.66:0/878154165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb85c1031a0 0x7fb85c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 --2- 172.21.15.66:0/878154165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85c101fb0 0x7fb85c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 -- 172.21.15.66:0/878154165 >> 172.21.15.66:0/878154165 conn(0x7fb85c0fd520 msgr2=0x7fb85c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 -- 172.21.15.66:0/878154165 shutdown_connections 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.619+0000 7fb860e14640 1 -- 172.21.15.66:0/878154165 wait complete. 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 Processor -- start 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 -- start start 2024-04-21T09:06:01.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85c101fb0 0x7fb85c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c1031a0 0x7fb85c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb85c103b60 0x7fb85c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c10daf0 con 0x7fb85c1031a0 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c10d970 con 0x7fb85c101fb0 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb860e14640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c10dc70 con 0x7fb85c103b60 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c1031a0 0x7fb85c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c1031a0 0x7fb85c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59400/0 (socket says 172.21.15.66:59400) 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 -- 172.21.15.66:0/2155448154 learned_addr learned my addr 172.21.15.66:0/2155448154 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb85a59c640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85c101fb0 0x7fb85c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb85ad9d640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb85c103b60 0x7fb85c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 -- 172.21.15.66:0/2155448154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb85c103b60 msgr2=0x7fb85c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb85c103b60 0x7fb85c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 -- 172.21.15.66:0/2155448154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85c101fb0 msgr2=0x7fb85c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85c101fb0 0x7fb85c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 -- 172.21.15.66:0/2155448154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c19fdc0 con 0x7fb85c1031a0 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.620+0000 7fb859d9b640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c1031a0 0x7fb85c1986e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d770 tx=0x7fb84c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.621+0000 7fb8537fe640 1 -- 172.21.15.66:0/2155448154 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c017070 con 0x7fb85c1031a0 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.621+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c1a00b0 con 0x7fb85c1031a0 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.621+0000 7fb8537fe640 1 -- 172.21.15.66:0/2155448154 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb84c0054d0 con 0x7fb85c1031a0 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.621+0000 7fb8537fe640 1 -- 172.21.15.66:0/2155448154 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c004c30 con 0x7fb85c1031a0 2024-04-21T09:06:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.621+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c1a05b0 con 0x7fb85c1031a0 2024-04-21T09:06:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.622+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85c1092f0 con 0x7fb85c1031a0 2024-04-21T09:06:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.623+0000 7fb8537fe640 1 -- 172.21.15.66:0/2155448154 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb84c004dd0 con 0x7fb85c1031a0 2024-04-21T09:06:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.623+0000 7fb8537fe640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8480a3bb0 0x7fb8480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.624+0000 7fb85a59c640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8480a3bb0 0x7fb8480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.624+0000 7fb8537fe640 1 -- 172.21.15.66:0/2155448154 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fb84c0d03f0 con 0x7fb85c1031a0 2024-04-21T09:06:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.624+0000 7fb85a59c640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8480a3bb0 0x7fb8480a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb844005fd0 tx=0x7fb84400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:01.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.627+0000 7fb8537fe640 1 -- 172.21.15.66:0/2155448154 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb84c09c540 con 0x7fb85c1031a0 2024-04-21T09:06:01.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.811+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb85c02d810 con 0x7fb85c1031a0 2024-04-21T09:06:01.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.811+0000 7fb8537fe640 1 -- 172.21.15.66:0/2155448154 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fb84c0a0840 con 0x7fb85c1031a0 2024-04-21T09:06:01.812 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:06:01.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8480a3bb0 msgr2=0x7fb8480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8480a3bb0 0x7fb8480a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb844005fd0 tx=0x7fb84400b410 comp rx=0 tx=0).stop 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c1031a0 msgr2=0x7fb85c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c1031a0 0x7fb85c1986e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d770 tx=0x7fb84c00dc30 comp rx=0 tx=0).stop 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 shutdown_connections 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8480a3bb0 0x7fb8480a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb85c103b60 0x7fb85c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb85c1031a0 0x7fb85c1986e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 --2- 172.21.15.66:0/2155448154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85c101fb0 0x7fb85c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 >> 172.21.15.66:0/2155448154 conn(0x7fb85c0fd520 msgr2=0x7fb85c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 shutdown_connections 2024-04-21T09:06:01.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.813+0000 7fb860e14640 1 -- 172.21.15.66:0/2155448154 wait complete. 2024-04-21T09:06:01.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:06:01.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 -- 172.21.15.66:0/2810736202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa8101fe0 msgr2=0x7f7aa8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2810736202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa8101fe0 0x7f7aa8102460 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009900 tx=0x7f7a9c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 -- 172.21.15.66:0/2810736202 shutdown_connections 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2810736202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aa81029a0 0x7f7aa81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2810736202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa8101fe0 0x7f7aa8102460 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2810736202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8100e90 0x7f7aa81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 -- 172.21.15.66:0/2810736202 >> 172.21.15.66:0/2810736202 conn(0x7f7aa80fc400 msgr2=0x7f7aa80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 -- 172.21.15.66:0/2810736202 shutdown_connections 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.928+0000 7f7ab0681640 1 -- 172.21.15.66:0/2810736202 wait complete. 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 Processor -- start 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 -- start start 2024-04-21T09:06:01.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8100e90 0x7f7aa8104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aa8101fe0 0x7f7aa8103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa81029a0 0x7f7aa8103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa810c9f0 con 0x7f7aa81029a0 2024-04-21T09:06:01.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa810c870 con 0x7f7aa8101fe0 2024-04-21T09:06:01.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7ab0681640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa810cb70 con 0x7f7aa8100e90 2024-04-21T09:06:01.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aaec1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa81029a0 0x7f7aa8103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aaec1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa81029a0 0x7f7aa8103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59416/0 (socket says 172.21.15.66:59416) 2024-04-21T09:06:01.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aaec1e640 1 -- 172.21.15.66:0/2685724733 learned_addr learned my addr 172.21.15.66:0/2685724733 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:01.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aadc1c640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aa8101fe0 0x7f7aa8103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aae41d640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8100e90 0x7f7aa8104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aaec1e640 1 -- 172.21.15.66:0/2685724733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8100e90 msgr2=0x7f7aa8104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aaec1e640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8100e90 0x7f7aa8104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.929+0000 7f7aaec1e640 1 -- 172.21.15.66:0/2685724733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aa8101fe0 msgr2=0x7f7aa8103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7aaec1e640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aa8101fe0 0x7f7aa8103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7aaec1e640 1 -- 172.21.15.66:0/2685724733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa8104220 con 0x7f7aa81029a0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7aaec1e640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa81029a0 0x7f7aa8103990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009fd0 tx=0x7f7aa000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7a9b7fe640 1 -- 172.21.15.66:0/2685724733 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0017070 con 0x7f7aa81029a0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7a9b7fe640 1 -- 172.21.15.66:0/2685724733 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0003e20 con 0x7f7aa81029a0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa81a23a0 con 0x7f7aa81029a0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7a9b7fe640 1 -- 172.21.15.66:0/2685724733 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00106c0 con 0x7f7aa81029a0 2024-04-21T09:06:01.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.930+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa81a2870 con 0x7f7aa81029a0 2024-04-21T09:06:01.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.931+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa81a2b30 con 0x7f7aa81029a0 2024-04-21T09:06:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.933+0000 7f7a9b7fe640 1 -- 172.21.15.66:0/2685724733 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7aa0004140 con 0x7f7aa81029a0 2024-04-21T09:06:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.933+0000 7f7a9b7fe640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7a940a3bb0 0x7f7a940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.933+0000 7f7aae41d640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7a940a3bb0 0x7f7a940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.933+0000 7f7a9b7fe640 1 -- 172.21.15.66:0/2685724733 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f7aa00d0320 con 0x7f7aa81029a0 2024-04-21T09:06:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.934+0000 7f7aae41d640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7a940a3bb0 0x7f7a940a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005fd0 tx=0x7f7aa4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:01.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:01.937+0000 7f7a9b7fe640 1 -- 172.21.15.66:0/2685724733 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7aa00047e0 con 0x7f7aa81029a0 2024-04-21T09:06:02.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:02 smithi066 ceph-mon[27953]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 370 B/s wr, 0 op/s 2024-04-21T09:06:02.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3044403097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3047669111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2155448154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.125+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7aa81a2e20 con 0x7f7aa81029a0 2024-04-21T09:06:02.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.126+0000 7f7a9b7fe640 1 -- 172.21.15.66:0/2685724733 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f7aa009cac0 con 0x7f7aa81029a0 2024-04-21T09:06:02.126 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:06:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.127+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7a940a3bb0 msgr2=0x7f7a940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.127+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7a940a3bb0 0x7f7a940a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005fd0 tx=0x7f7aa4005850 comp rx=0 tx=0).stop 2024-04-21T09:06:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.127+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa81029a0 msgr2=0x7f7aa8103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.127+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa81029a0 0x7f7aa8103990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009fd0 tx=0x7f7aa000b260 comp rx=0 tx=0).stop 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 shutdown_connections 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f7a940a3bb0 0x7f7a940a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa81029a0 0x7f7aa8103990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aa8101fe0 0x7f7aa8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 --2- 172.21.15.66:0/2685724733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8100e90 0x7f7aa8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 >> 172.21.15.66:0/2685724733 conn(0x7f7aa80fc400 msgr2=0x7f7aa8069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 shutdown_connections 2024-04-21T09:06:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.128+0000 7f7ab0681640 1 -- 172.21.15.66:0/2685724733 wait complete. 2024-04-21T09:06:02.139 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T09:06:02.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/2118467906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 msgr2=0x7fdc8810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:02.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/2118467906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc8810a0e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009900 tx=0x7fdc8002bc50 comp rx=0 tx=0).stop 2024-04-21T09:06:02.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/2118467906 shutdown_connections 2024-04-21T09:06:02.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/2118467906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc8810a0e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/2118467906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc881053c0 0x7fdc881077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/2118467906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc88102a90 0x7fdc88104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/2118467906 >> 172.21.15.66:0/2118467906 conn(0x7fdc880fc440 msgr2=0x7fdc880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/2118467906 shutdown_connections 2024-04-21T09:06:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.241+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/2118467906 wait complete. 2024-04-21T09:06:02.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 Processor -- start 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 -- start start 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc88102a90 0x7fdc88102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc881053c0 0x7fdc88100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc88100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8810daf0 con 0x7fdc88107cf0 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8810d970 con 0x7fdc88102a90 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.242+0000 7fdc8d0eb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8810dc70 con 0x7fdc881053c0 2024-04-21T09:06:02.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc88100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc88100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59428/0 (socket says 172.21.15.66:59428) 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 -- 172.21.15.66:0/202921860 learned_addr learned my addr 172.21.15.66:0/202921860 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc86d9d640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc88102a90 0x7fdc88102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8659c640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc881053c0 0x7fdc88100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 -- 172.21.15.66:0/202921860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc881053c0 msgr2=0x7fdc88100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc881053c0 0x7fdc88100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 -- 172.21.15.66:0/202921860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc88102a90 msgr2=0x7fdc88102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc88102a90 0x7fdc88102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 -- 172.21.15.66:0/202921860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc88101810 con 0x7fdc88107cf0 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc86d9d640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc88102a90 0x7fdc88102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:02.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8759e640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc88100f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009230 tx=0x7fdc800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:02.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc6ffff640 1 -- 172.21.15.66:0/202921860 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80043020 con 0x7fdc88107cf0 2024-04-21T09:06:02.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc881a2230 con 0x7fdc88107cf0 2024-04-21T09:06:02.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc6ffff640 1 -- 172.21.15.66:0/202921860 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc80004aa0 con 0x7fdc88107cf0 2024-04-21T09:06:02.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a27d0 con 0x7fdc88107cf0 2024-04-21T09:06:02.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.243+0000 7fdc6ffff640 1 -- 172.21.15.66:0/202921860 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80003c70 con 0x7fdc88107cf0 2024-04-21T09:06:02.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.244+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc880fdf30 con 0x7fdc88107cf0 2024-04-21T09:06:02.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.245+0000 7fdc6ffff640 1 -- 172.21.15.66:0/202921860 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdc80046020 con 0x7fdc88107cf0 2024-04-21T09:06:02.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.246+0000 7fdc6ffff640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdc700a3bb0 0x7fdc700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:02.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.246+0000 7fdc86d9d640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdc700a3bb0 0x7fdc700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:02.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.246+0000 7fdc6ffff640 1 -- 172.21.15.66:0/202921860 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fdc800f2e90 con 0x7fdc88107cf0 2024-04-21T09:06:02.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.247+0000 7fdc86d9d640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdc700a3bb0 0x7fdc700a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc78008e00 tx=0x7fdc78008350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:02.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.250+0000 7fdc6ffff640 1 -- 172.21.15.66:0/202921860 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc8004d020 con 0x7fdc88107cf0 2024-04-21T09:06:02.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:02 smithi186 ceph-mon[31505]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 370 B/s wr, 0 op/s 2024-04-21T09:06:02.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3044403097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3047669111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2155448154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:02 smithi083 ceph-mon[31413]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 370 B/s wr, 0 op/s 2024-04-21T09:06:02.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3044403097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3047669111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2155448154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:06:02.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.450+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc8802d5c0 con 0x7fdc88107cf0 2024-04-21T09:06:02.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.451+0000 7fdc6ffff640 1 -- 172.21.15.66:0/202921860 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+588 (secure 0 0 0) 0x7fdc800bf630 con 0x7fdc88107cf0 2024-04-21T09:06:02.451 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/612283217 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:02.451 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/763565357 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:02.451 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3624224927 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:02.451 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2187684654 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:02.451 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3374606173 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:02.451 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/763565357 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:02.452 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/502604018 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:02.452 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3624224927 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:02.452 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2365699589 2024-04-22T08:58:38.211749+0000 2024-04-21T09:06:02.452 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3868723687 2024-04-22T08:58:12.038096+0000 2024-04-21T09:06:02.452 INFO:teuthology.orchestra.run.smithi066.stderr:listed 10 entries 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdc700a3bb0 msgr2=0x7fdc700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdc700a3bb0 0x7fdc700a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc78008e00 tx=0x7fdc78008350 comp rx=0 tx=0).stop 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 msgr2=0x7fdc88100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc88100f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009230 tx=0x7fdc800042f0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 shutdown_connections 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdc700a3bb0 0x7fdc700a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc88107cf0 0x7fdc88100f80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc881053c0 0x7fdc88100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 --2- 172.21.15.66:0/202921860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc88102a90 0x7fdc88102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:02.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.452+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 >> 172.21.15.66:0/202921860 conn(0x7fdc880fc440 msgr2=0x7fdc881033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:02.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.453+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 shutdown_connections 2024-04-21T09:06:02.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:02.453+0000 7fdc8d0eb640 1 -- 172.21.15.66:0/202921860 wait complete. 2024-04-21T09:06:02.464 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-04-21T09:06:02.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:06:02.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-21T09:06:02.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[69478]: fuse finished with error 0 and tester_r 0 2024-04-21T09:06:02.488 INFO:teuthology.orchestra.run:waiting for 300 2024-04-21T09:06:03.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:03 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2685724733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:06:03.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:03 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/202921860' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:06:03.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:03 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2685724733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:06:03.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:03 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/202921860' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:06:03.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:03 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2685724733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:06:03.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:03 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/202921860' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:06:04.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:04 smithi186 ceph-mon[31505]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-21T09:06:04.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:04 smithi083 ceph-mon[31413]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-21T09:06:04.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:04 smithi066 ceph-mon[27953]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-21T09:06:06.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:06 smithi186 ceph-mon[31505]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 271 B/s wr, 0 op/s 2024-04-21T09:06:06.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:06 smithi083 ceph-mon[31413]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 271 B/s wr, 0 op/s 2024-04-21T09:06:06.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:06 smithi066 ceph-mon[27953]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 271 B/s wr, 0 op/s 2024-04-21T09:06:08.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:08 smithi186 ceph-mon[31505]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:08.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:08 smithi083 ceph-mon[31413]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:08.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:08 smithi066 ceph-mon[27953]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:08.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:06:08.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:06:08.590 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-21T09:06:08.622 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:06:08.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4140941755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 msgr2=0x7f9c9c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4140941755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c1012c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009900 tx=0x7f9c9002bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4140941755 shutdown_connections 2024-04-21T09:06:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4140941755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9c1029b0 0x7f9c9c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4140941755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9c101ff0 0x7f9c9c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4140941755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c1012c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4140941755 >> 172.21.15.66:0/4140941755 conn(0x7f9c9c0fc450 msgr2=0x7f9c9c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:08.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4140941755 shutdown_connections 2024-04-21T09:06:08.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.775+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4140941755 wait complete. 2024-04-21T09:06:08.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 Processor -- start 2024-04-21T09:06:08.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 -- start start 2024-04-21T09:06:08.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c06b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:08.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9c101ff0 0x7f9c9c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:08.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9c1029b0 0x7f9c9c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:08.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c10ca20 con 0x7f9c9c100ea0 2024-04-21T09:06:08.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c10c8a0 con 0x7f9c9c1029b0 2024-04-21T09:06:08.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca3e11640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c10cba0 con 0x7f9c9c101ff0 2024-04-21T09:06:08.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c06b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c06b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42970/0 (socket says 172.21.15.66:42970) 2024-04-21T09:06:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 -- 172.21.15.66:0/4102504400 learned_addr learned my addr 172.21.15.66:0/4102504400 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca23ae640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9c1029b0 0x7f9c9c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca13ac640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9c101ff0 0x7f9c9c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 -- 172.21.15.66:0/4102504400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9c101ff0 msgr2=0x7f9c9c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9c101ff0 0x7f9c9c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 -- 172.21.15.66:0/4102504400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9c1029b0 msgr2=0x7f9c9c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9c1029b0 0x7f9c9c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.776+0000 7f9ca1bad640 1 -- 172.21.15.66:0/4102504400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c19fe10 con 0x7f9c9c100ea0 2024-04-21T09:06:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.777+0000 7f9ca13ac640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9c101ff0 0x7f9c9c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.777+0000 7f9ca1bad640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c06b190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000c00 tx=0x7f9c900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.777+0000 7f9c8effd640 1 -- 172.21.15.66:0/4102504400 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90043020 con 0x7f9c9c100ea0 2024-04-21T09:06:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.777+0000 7f9c8effd640 1 -- 172.21.15.66:0/4102504400 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c900054c0 con 0x7f9c9c100ea0 2024-04-21T09:06:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.777+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c1a0040 con 0x7f9c9c100ea0 2024-04-21T09:06:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.777+0000 7f9c8effd640 1 -- 172.21.15.66:0/4102504400 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90003db0 con 0x7f9c9c100ea0 2024-04-21T09:06:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.777+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c1a05e0 con 0x7f9c9c100ea0 2024-04-21T09:06:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.778+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c1081b0 con 0x7f9c9c100ea0 2024-04-21T09:06:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.779+0000 7f9c8effd640 1 -- 172.21.15.66:0/4102504400 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9c90046020 con 0x7f9c9c100ea0 2024-04-21T09:06:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.780+0000 7f9c8effd640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9c880a3c30 0x7f9c880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.780+0000 7f9ca13ac640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9c880a3c30 0x7f9c880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.780+0000 7f9c8effd640 1 -- 172.21.15.66:0/4102504400 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f9c900f2e10 con 0x7f9c9c100ea0 2024-04-21T09:06:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.780+0000 7f9ca13ac640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9c880a3c30 0x7f9c880a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06af00 tx=0x7f9c98005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.784+0000 7f9c8effd640 1 -- 172.21.15.66:0/4102504400 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c9004d020 con 0x7f9c9c100ea0 2024-04-21T09:06:08.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.971+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c9c02d810 con 0x7f9c9c100ea0 2024-04-21T09:06:08.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.973+0000 7f9c8effd640 1 -- 172.21.15.66:0/4102504400 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7f9c900bf5b0 con 0x7f9c9c100ea0 2024-04-21T09:06:08.974 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:08.974 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9c880a3c30 msgr2=0x7f9c880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9c880a3c30 0x7f9c880a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06af00 tx=0x7f9c98005e70 comp rx=0 tx=0).stop 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 msgr2=0x7f9c9c06b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c06b190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000c00 tx=0x7f9c900042f0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 shutdown_connections 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9c880a3c30 0x7f9c880a60f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9c1029b0 0x7f9c9c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9c101ff0 0x7f9c9c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 --2- 172.21.15.66:0/4102504400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c9c100ea0 0x7f9c9c06b190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:08.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 >> 172.21.15.66:0/4102504400 conn(0x7f9c9c0fc450 msgr2=0x7f9c9c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:08.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 shutdown_connections 2024-04-21T09:06:08.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:08.975+0000 7f9ca3e11640 1 -- 172.21.15.66:0/4102504400 wait complete. 2024-04-21T09:06:08.987 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:06:09.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 -- 172.21.15.66:0/3803879293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 msgr2=0x7f915c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 --2- 172.21.15.66:0/3803879293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1023c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9150009900 tx=0x7f915002bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:09.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 -- 172.21.15.66:0/3803879293 shutdown_connections 2024-04-21T09:06:09.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 --2- 172.21.15.66:0/3803879293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f915c103b50 0x7f915c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 --2- 172.21.15.66:0/3803879293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f915c103190 0x7f915c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 --2- 172.21.15.66:0/3803879293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1023c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 -- 172.21.15.66:0/3803879293 >> 172.21.15.66:0/3803879293 conn(0x7f915c0fd530 msgr2=0x7f915c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 -- 172.21.15.66:0/3803879293 shutdown_connections 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 -- 172.21.15.66:0/3803879293 wait complete. 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.090+0000 7f91615ce640 1 Processor -- start 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f91615ce640 1 -- start start 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f91615ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f91615ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f915c103190 0x7f915c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f91615ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f915c103b50 0x7f915c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f91615ce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f915c10daa0 con 0x7f915c101fa0 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f91615ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f915c10d920 con 0x7f915c103190 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f91615ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f915c10dc20 con 0x7f915c103b50 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42992/0 (socket says 172.21.15.66:42992) 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 -- 172.21.15.66:0/1098361080 learned_addr learned my addr 172.21.15.66:0/1098361080 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:09.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915b7fe640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f915c103b50 0x7f915c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915a7fc640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f915c103190 0x7f915c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 -- 172.21.15.66:0/1098361080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f915c103b50 msgr2=0x7f915c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f915c103b50 0x7f915c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 -- 172.21.15.66:0/1098361080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f915c103190 msgr2=0x7f915c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f915c103190 0x7f915c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 -- 172.21.15.66:0/1098361080 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915c19fdf0 con 0x7f915c101fa0 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915a7fc640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f915c103190 0x7f915c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.091+0000 7f915affd640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1981d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f9150004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:09.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.092+0000 7f9143fff640 1 -- 172.21.15.66:0/1098361080 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150043020 con 0x7f915c101fa0 2024-04-21T09:06:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.092+0000 7f9143fff640 1 -- 172.21.15.66:0/1098361080 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9150004aa0 con 0x7f915c101fa0 2024-04-21T09:06:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.092+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915c1a0080 con 0x7f915c101fa0 2024-04-21T09:06:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.092+0000 7f9143fff640 1 -- 172.21.15.66:0/1098361080 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150003d00 con 0x7f915c101fa0 2024-04-21T09:06:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.092+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915c1a0620 con 0x7f915c101fa0 2024-04-21T09:06:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.093+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915c109330 con 0x7f915c101fa0 2024-04-21T09:06:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.094+0000 7f9143fff640 1 -- 172.21.15.66:0/1098361080 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9150046020 con 0x7f915c101fa0 2024-04-21T09:06:09.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.094+0000 7f9143fff640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91440a3c30 0x7f91440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.094+0000 7f915a7fc640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91440a3c30 0x7f91440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.094+0000 7f9143fff640 1 -- 172.21.15.66:0/1098361080 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f91500f2c60 con 0x7f915c101fa0 2024-04-21T09:06:09.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.095+0000 7f915a7fc640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91440a3c30 0x7f91440a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f915c1996e0 tx=0x7f914c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:09.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.099+0000 7f9143fff640 1 -- 172.21.15.66:0/1098361080 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f915004d020 con 0x7f915c101fa0 2024-04-21T09:06:09.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.286+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f915c062b50 con 0x7f915c101fa0 2024-04-21T09:06:09.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.288+0000 7f9143fff640 1 -- 172.21.15.66:0/1098361080 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7f91500bf400 con 0x7f915c101fa0 2024-04-21T09:06:09.289 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:09.289 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91440a3c30 msgr2=0x7f91440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91440a3c30 0x7f91440a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f915c1996e0 tx=0x7f914c005e50 comp rx=0 tx=0).stop 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 msgr2=0x7f915c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1981d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f9150004270 comp rx=0 tx=0).stop 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 shutdown_connections 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f91440a3c30 0x7f91440a60f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f915c103b50 0x7f915c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f915c103190 0x7f915c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 --2- 172.21.15.66:0/1098361080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f915c101fa0 0x7f915c1981d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 >> 172.21.15.66:0/1098361080 conn(0x7f915c0fd530 msgr2=0x7f915c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 shutdown_connections 2024-04-21T09:06:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.290+0000 7f91615ce640 1 -- 172.21.15.66:0/1098361080 wait complete. 2024-04-21T09:06:09.301 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:09 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4102504400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:09.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-21T09:06:09.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:09 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4102504400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:09.364 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:09 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4102504400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 -- 172.21.15.66:0/1584641617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde281031a0 msgr2=0x7fde28103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/1584641617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde281031a0 0x7fde28103620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde10009900 tx=0x7fde1002bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 -- 172.21.15.66:0/1584641617 shutdown_connections 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/1584641617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde28103b60 0x7fde2810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/1584641617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde281031a0 0x7fde28103620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/1584641617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde28101fb0 0x7fde281023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 -- 172.21.15.66:0/1584641617 >> 172.21.15.66:0/1584641617 conn(0x7fde280fd520 msgr2=0x7fde280ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 -- 172.21.15.66:0/1584641617 shutdown_connections 2024-04-21T09:06:09.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.406+0000 7fde2c8d2640 1 -- 172.21.15.66:0/1584641617 wait complete. 2024-04-21T09:06:09.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 Processor -- start 2024-04-21T09:06:09.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 -- start start 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28101fb0 0x7fde28198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde281031a0 0x7fde28198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde28103b60 0x7fde2819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde2810daf0 con 0x7fde28101fb0 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde2810d970 con 0x7fde28103b60 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2c8d2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fde2810dc70 con 0x7fde281031a0 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28101fb0 0x7fde28198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28101fb0 0x7fde28198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43016/0 (socket says 172.21.15.66:43016) 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 -- 172.21.15.66:0/2669029287 learned_addr learned my addr 172.21.15.66:0/2669029287 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde26d9d640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde28103b60 0x7fde2819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde25d9b640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde281031a0 0x7fde28198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 -- 172.21.15.66:0/2669029287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde281031a0 msgr2=0x7fde28198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde281031a0 0x7fde28198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 -- 172.21.15.66:0/2669029287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde28103b60 msgr2=0x7fde2819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde28103b60 0x7fde2819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.407+0000 7fde2659c640 1 -- 172.21.15.66:0/2669029287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2819fd70 con 0x7fde28101fb0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.408+0000 7fde2659c640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28101fb0 0x7fde28198150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c970 tx=0x7fde1800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.408+0000 7fde1f7fe640 1 -- 172.21.15.66:0/2669029287 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18007c10 con 0x7fde28101fb0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.408+0000 7fde1f7fe640 1 -- 172.21.15.66:0/2669029287 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde18007db0 con 0x7fde28101fb0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.408+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde281a0060 con 0x7fde28101fb0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.408+0000 7fde1f7fe640 1 -- 172.21.15.66:0/2669029287 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800f870 con 0x7fde28101fb0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.408+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281a05b0 con 0x7fde28101fb0 2024-04-21T09:06:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.409+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde281092d0 con 0x7fde28101fb0 2024-04-21T09:06:09.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.410+0000 7fde1f7fe640 1 -- 172.21.15.66:0/2669029287 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde1800fa10 con 0x7fde28101fb0 2024-04-21T09:06:09.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.411+0000 7fde1f7fe640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fde140a3bb0 0x7fde140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:09.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.411+0000 7fde25d9b640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fde140a3bb0 0x7fde140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:09.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.411+0000 7fde1f7fe640 1 -- 172.21.15.66:0/2669029287 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fde180d0c30 con 0x7fde28101fb0 2024-04-21T09:06:09.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.412+0000 7fde25d9b640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fde140a3bb0 0x7fde140a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fde28199660 tx=0x7fde10005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:09.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.415+0000 7fde1f7fe640 1 -- 172.21.15.66:0/2669029287 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde1809d4f0 con 0x7fde28101fb0 2024-04-21T09:06:09.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:09.643+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fde2802d5c0 con 0x7fde28101fb0 2024-04-21T09:06:10.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.057+0000 7fde1f7fe640 1 -- 172.21.15.66:0/2669029287 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fde1809cdf0 con 0x7fde28101fb0 2024-04-21T09:06:10.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fde140a3bb0 msgr2=0x7fde140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:10.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fde140a3bb0 0x7fde140a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fde28199660 tx=0x7fde10005be0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28101fb0 msgr2=0x7fde28198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28101fb0 0x7fde28198150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c970 tx=0x7fde1800ce30 comp rx=0 tx=0).stop 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 shutdown_connections 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fde140a3bb0 0x7fde140a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde28103b60 0x7fde2819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde281031a0 0x7fde28198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 --2- 172.21.15.66:0/2669029287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28101fb0 0x7fde28198150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 >> 172.21.15.66:0/2669029287 conn(0x7fde280fd520 msgr2=0x7fde281064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 shutdown_connections 2024-04-21T09:06:10.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.059+0000 7fde2c8d2640 1 -- 172.21.15.66:0/2669029287 wait complete. 2024-04-21T09:06:10.071 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-21T09:06:10.071 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-21T09:06:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 -- 172.21.15.66:0/3710278378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c102a30 msgr2=0x7f3e8c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 --2- 172.21.15.66:0/3710278378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c102a30 0x7f3e8c104e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 -- 172.21.15.66:0/3710278378 shutdown_connections 2024-04-21T09:06:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 --2- 172.21.15.66:0/3710278378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8c107c90 0x7f3e8c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 --2- 172.21.15.66:0/3710278378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e8c105360 0x7f3e8c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 --2- 172.21.15.66:0/3710278378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c102a30 0x7f3e8c104e20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 -- 172.21.15.66:0/3710278378 >> 172.21.15.66:0/3710278378 conn(0x7f3e8c0fc440 msgr2=0x7f3e8c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 -- 172.21.15.66:0/3710278378 shutdown_connections 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.176+0000 7f3e945e6640 1 -- 172.21.15.66:0/3710278378 wait complete. 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 Processor -- start 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 -- start start 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8c102a30 0x7f3e8c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e8c105360 0x7f3e8c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c107c90 0x7f3e8c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c10daf0 con 0x7f3e8c107c90 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c10d970 con 0x7f3e8c102a30 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e945e6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c10dc70 con 0x7f3e8c105360 2024-04-21T09:06:10.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c107c90 0x7f3e8c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c107c90 0x7f3e8c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43038/0 (socket says 172.21.15.66:43038) 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 -- 172.21.15.66:0/4145280685 learned_addr learned my addr 172.21.15.66:0/4145280685 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92382640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8c102a30 0x7f3e8c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e91b81640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e8c105360 0x7f3e8c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 -- 172.21.15.66:0/4145280685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e8c105360 msgr2=0x7f3e8c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e8c105360 0x7f3e8c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 -- 172.21.15.66:0/4145280685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8c102a30 msgr2=0x7f3e8c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8c102a30 0x7f3e8c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92b83640 1 -- 172.21.15.66:0/4145280685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8c19fd30 con 0x7f3e8c107c90 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.177+0000 7f3e92382640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8c102a30 0x7f3e8c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.178+0000 7f3e92b83640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c107c90 0x7f3e8c19f6c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400cce0 tx=0x7f3e8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.178+0000 7f3e837fe640 1 -- 172.21.15.66:0/4145280685 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400f450 con 0x7f3e8c107c90 2024-04-21T09:06:10.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.178+0000 7f3e837fe640 1 -- 172.21.15.66:0/4145280685 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e8400faa0 con 0x7f3e8c107c90 2024-04-21T09:06:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.178+0000 7f3e837fe640 1 -- 172.21.15.66:0/4145280685 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84018b70 con 0x7f3e8c107c90 2024-04-21T09:06:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.178+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8c19ffc0 con 0x7f3e8c107c90 2024-04-21T09:06:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.178+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8c1a04e0 con 0x7f3e8c107c90 2024-04-21T09:06:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.179+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8c0fdf30 con 0x7f3e8c107c90 2024-04-21T09:06:10.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.180+0000 7f3e837fe640 1 -- 172.21.15.66:0/4145280685 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3e84018d10 con 0x7f3e8c107c90 2024-04-21T09:06:10.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.181+0000 7f3e837fe640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e780a3c30 0x7f3e780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:10.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.181+0000 7f3e92382640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e780a3c30 0x7f3e780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:10.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.181+0000 7f3e837fe640 1 -- 172.21.15.66:0/4145280685 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f3e840d0b30 con 0x7f3e8c107c90 2024-04-21T09:06:10.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.181+0000 7f3e92382640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e780a3c30 0x7f3e780a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c000c00 tx=0x7f3e7c0095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:10.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.185+0000 7f3e837fe640 1 -- 172.21.15.66:0/4145280685 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e84014030 con 0x7f3e8c107c90 2024-04-21T09:06:10.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:10 smithi186 ceph-mon[31505]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:10.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1098361080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:10.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2669029287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T09:06:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:10 smithi083 ceph-mon[31413]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1098361080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:10.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2669029287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T09:06:10.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:10 smithi066 ceph-mon[27953]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:10.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1098361080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:10.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2669029287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T09:06:10.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:10.414+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f3e8c062b50 con 0x7f3e8c107c90 2024-04-21T09:06:11.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.071+0000 7f3e837fe640 1 -- 172.21.15.66:0/4145280685 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f3e8409d3e0 con 0x7f3e8c107c90 2024-04-21T09:06:11.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e780a3c30 msgr2=0x7f3e780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:11.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e780a3c30 0x7f3e780a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c000c00 tx=0x7f3e7c0095f0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c107c90 msgr2=0x7f3e8c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:11.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c107c90 0x7f3e8c19f6c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400cce0 tx=0x7f3e8400a620 comp rx=0 tx=0).stop 2024-04-21T09:06:11.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 shutdown_connections 2024-04-21T09:06:11.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3e780a3c30 0x7f3e780a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e8c107c90 0x7f3e8c19f6c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e8c105360 0x7f3e8c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 --2- 172.21.15.66:0/4145280685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8c102a30 0x7f3e8c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 >> 172.21.15.66:0/4145280685 conn(0x7f3e8c0fc440 msgr2=0x7f3e8c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:11.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 shutdown_connections 2024-04-21T09:06:11.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.074+0000 7f3e945e6640 1 -- 172.21.15.66:0/4145280685 wait complete. 2024-04-21T09:06:11.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-21T09:06:11.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.198+0000 7f7034732640 1 -- 172.21.15.66:0/3191369391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c105400 msgr2=0x7f702c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.198+0000 7f7034732640 1 --2- 172.21.15.66:0/3191369391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c105400 0x7f702c1077f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7020009900 tx=0x7f702002bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.198+0000 7f7034732640 1 -- 172.21.15.66:0/3191369391 shutdown_connections 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 --2- 172.21.15.66:0/3191369391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f702c107d30 0x7f702c10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 --2- 172.21.15.66:0/3191369391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c105400 0x7f702c1077f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 --2- 172.21.15.66:0/3191369391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f702c102ad0 0x7f702c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 -- 172.21.15.66:0/3191369391 >> 172.21.15.66:0/3191369391 conn(0x7f702c0fc400 msgr2=0x7f702c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 -- 172.21.15.66:0/3191369391 shutdown_connections 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 -- 172.21.15.66:0/3191369391 wait complete. 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 Processor -- start 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 -- start start 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f702c102ad0 0x7f702c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:11.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f702c105400 0x7f702c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c107d30 0x7f702c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f702c10db20 con 0x7f702c107d30 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f702c10d9a0 con 0x7f702c105400 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7034732640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f702c10dca0 con 0x7f702c102ad0 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7032ccf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c107d30 0x7f702c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.199+0000 7f7032ccf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c107d30 0x7f702c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43056/0 (socket says 172.21.15.66:43056) 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7032ccf640 1 -- 172.21.15.66:0/3523343604 learned_addr learned my addr 172.21.15.66:0/3523343604 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f70324ce640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f702c102ad0 0x7f702c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7031ccd640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f702c105400 0x7f702c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7032ccf640 1 -- 172.21.15.66:0/3523343604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f702c102ad0 msgr2=0x7f702c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7032ccf640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f702c102ad0 0x7f702c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7032ccf640 1 -- 172.21.15.66:0/3523343604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f702c105400 msgr2=0x7f702c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:11.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7032ccf640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f702c105400 0x7f702c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7032ccf640 1 -- 172.21.15.66:0/3523343604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c19fe40 con 0x7f702c107d30 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f70324ce640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f702c102ad0 0x7f702c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7032ccf640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c107d30 0x7f702c19f7d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f701c00b970 tx=0x7f701c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f702b7fe640 1 -- 172.21.15.66:0/3523343604 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c019070 con 0x7f702c107d30 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f702b7fe640 1 -- 172.21.15.66:0/3523343604 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c014440 con 0x7f702c107d30 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c1a0130 con 0x7f702c107d30 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f702b7fe640 1 -- 172.21.15.66:0/3523343604 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0075d0 con 0x7f702c107d30 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.200+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c1a06d0 con 0x7f702c107d30 2024-04-21T09:06:11.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.201+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702c0fdef0 con 0x7f702c107d30 2024-04-21T09:06:11.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.202+0000 7f702b7fe640 1 -- 172.21.15.66:0/3523343604 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f701c01c070 con 0x7f702c107d30 2024-04-21T09:06:11.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.203+0000 7f702b7fe640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70140a3c30 0x7f70140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:11.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.203+0000 7f70324ce640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70140a3c30 0x7f70140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:11.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.203+0000 7f702b7fe640 1 -- 172.21.15.66:0/3523343604 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f701c0cfc20 con 0x7f702c107d30 2024-04-21T09:06:11.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.203+0000 7f70324ce640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70140a3c30 0x7f70140a60f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f702400baa0 tx=0x7f7024005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:11.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.207+0000 7f702b7fe640 1 -- 172.21.15.66:0/3523343604 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f701c09c3c0 con 0x7f702c107d30 2024-04-21T09:06:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2669029287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T09:06:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:11 smithi186 ceph-mon[31505]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:06:11.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4145280685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-21T09:06:11.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2669029287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T09:06:11.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:11 smithi083 ceph-mon[31413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:06:11.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4145280685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-21T09:06:11.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2669029287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T09:06:11.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:11 smithi066 ceph-mon[27953]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:06:11.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4145280685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-21T09:06:11.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:11.435+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f702c062b50 con 0x7f702c107d30 2024-04-21T09:06:12.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.081+0000 7f702b7fe640 1 -- 172.21.15.66:0/3523343604 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f701c09bcc0 con 0x7f702c107d30 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70140a3c30 msgr2=0x7f70140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70140a3c30 0x7f70140a60f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f702400baa0 tx=0x7f7024005d90 comp rx=0 tx=0).stop 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c107d30 msgr2=0x7f702c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c107d30 0x7f702c19f7d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f701c00b970 tx=0x7f701c00be30 comp rx=0 tx=0).stop 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 shutdown_connections 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f70140a3c30 0x7f70140a60f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f702c107d30 0x7f702c19f7d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f702c105400 0x7f702c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 --2- 172.21.15.66:0/3523343604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f702c102ad0 0x7f702c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 >> 172.21.15.66:0/3523343604 conn(0x7f702c0fc400 msgr2=0x7f702c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 shutdown_connections 2024-04-21T09:06:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.084+0000 7f7034732640 1 -- 172.21.15.66:0/3523343604 wait complete. 2024-04-21T09:06:12.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-21T09:06:12.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 -- 172.21.15.66:0/4068878934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 msgr2=0x7f0d54074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 --2- 172.21.15.66:0/4068878934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d54074e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009930 tx=0x7f0d5002bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 -- 172.21.15.66:0/4068878934 shutdown_connections 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 --2- 172.21.15.66:0/4068878934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d5410e580 0x7f0d54110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 --2- 172.21.15.66:0/4068878934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d54074e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 --2- 172.21.15.66:0/4068878934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d54075f90 0x7f0d540743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 -- 172.21.15.66:0/4068878934 >> 172.21.15.66:0/4068878934 conn(0x7f0d540fd520 msgr2=0x7f0d540ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 -- 172.21.15.66:0/4068878934 shutdown_connections 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 -- 172.21.15.66:0/4068878934 wait complete. 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 Processor -- start 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.212+0000 7f0d5c031640 1 -- start start 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5c031640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d5419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5c031640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d54075f90 0x7f0d5419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:12.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5c031640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5410e580 0x7f0d541a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5c031640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54114130 con 0x7f0d540749b0 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5c031640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54113fb0 con 0x7f0d5410e580 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5c031640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541142b0 con 0x7f0d54075f90 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d5419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d5419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43074/0 (socket says 172.21.15.66:43074) 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 -- 172.21.15.66:0/3357748547 learned_addr learned my addr 172.21.15.66:0/3357748547 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5a5ce640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5410e580 0x7f0d541a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d595cc640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d54075f90 0x7f0d5419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 -- 172.21.15.66:0/3357748547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d54075f90 msgr2=0x7f0d5419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d54075f90 0x7f0d5419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 -- 172.21.15.66:0/3357748547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5410e580 msgr2=0x7f0d541a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5410e580 0x7f0d541a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 -- 172.21.15.66:0/3357748547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d541a6490 con 0x7f0d540749b0 2024-04-21T09:06:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d59dcd640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d5419e870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400e9f0 tx=0x7f0d4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d4affd640 1 -- 172.21.15.66:0/3357748547 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400cd80 con 0x7f0d540749b0 2024-04-21T09:06:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d541a6780 con 0x7f0d540749b0 2024-04-21T09:06:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d4affd640 1 -- 172.21.15.66:0/3357748547 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d440108e0 con 0x7f0d540749b0 2024-04-21T09:06:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.213+0000 7f0d4affd640 1 -- 172.21.15.66:0/3357748547 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44018980 con 0x7f0d540749b0 2024-04-21T09:06:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.214+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d541a6d20 con 0x7f0d540749b0 2024-04-21T09:06:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.214+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d54104750 con 0x7f0d540749b0 2024-04-21T09:06:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.216+0000 7f0d4affd640 1 -- 172.21.15.66:0/3357748547 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d44010430 con 0x7f0d540749b0 2024-04-21T09:06:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.216+0000 7f0d4affd640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0d400a3bb0 0x7f0d400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.216+0000 7f0d595cc640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0d400a3bb0 0x7f0d400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.216+0000 7f0d4affd640 1 -- 172.21.15.66:0/3357748547 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f0d440d0ec0 con 0x7f0d540749b0 2024-04-21T09:06:12.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.217+0000 7f0d595cc640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0d400a3bb0 0x7f0d400a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006010 tx=0x7f0d5002c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:12.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.220+0000 7f0d4affd640 1 -- 172.21.15.66:0/3357748547 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d4409d660 con 0x7f0d540749b0 2024-04-21T09:06:12.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T09:06:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4145280685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-21T09:06:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T09:06:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:06:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T09:06:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T09:06:12.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3523343604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T09:06:12.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:12.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T09:06:12.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4145280685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-21T09:06:12.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T09:06:12.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:06:12.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T09:06:12.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T09:06:12.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3523343604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T09:06:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T09:06:12.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T09:06:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4145280685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-21T09:06:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T09:06:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T09:06:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T09:06:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T09:06:12.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3523343604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T09:06:12.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:12.448+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f0d5402d5c0 con 0x7f0d540749b0 2024-04-21T09:06:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.092+0000 7f0d4affd640 1 -- 172.21.15.66:0/3357748547 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f0d4409cf60 con 0x7f0d540749b0 2024-04-21T09:06:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0d400a3bb0 msgr2=0x7f0d400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0d400a3bb0 0x7f0d400a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006010 tx=0x7f0d5002c800 comp rx=0 tx=0).stop 2024-04-21T09:06:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 msgr2=0x7f0d5419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d5419e870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400e9f0 tx=0x7f0d4400eeb0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 shutdown_connections 2024-04-21T09:06:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0d400a3bb0 0x7f0d400a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5410e580 0x7f0d541a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d54075f90 0x7f0d5419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 --2- 172.21.15.66:0/3357748547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d540749b0 0x7f0d5419e870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.095+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 >> 172.21.15.66:0/3357748547 conn(0x7f0d540fd520 msgr2=0x7f0d54101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.096+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 shutdown_connections 2024-04-21T09:06:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.096+0000 7f0d5c031640 1 -- 172.21.15.66:0/3357748547 wait complete. 2024-04-21T09:06:13.111 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-21T09:06:13.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-21T09:06:13.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.222+0000 7f75e3140640 1 -- 172.21.15.66:0/2205426086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 msgr2=0x7f75dc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.222+0000 7f75e3140640 1 --2- 172.21.15.66:0/2205426086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc1092c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc02bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:13.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.222+0000 7f75e3140640 1 -- 172.21.15.66:0/2205426086 shutdown_connections 2024-04-21T09:06:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.222+0000 7f75e3140640 1 --2- 172.21.15.66:0/2205426086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc1092c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.222+0000 7f75e3140640 1 --2- 172.21.15.66:0/2205426086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc101fe0 0x7f75dc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.222+0000 7f75e3140640 1 --2- 172.21.15.66:0/2205426086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc100e90 0x7f75dc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.222+0000 7f75e3140640 1 -- 172.21.15.66:0/2205426086 >> 172.21.15.66:0/2205426086 conn(0x7f75dc0fc400 msgr2=0x7f75dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 -- 172.21.15.66:0/2205426086 shutdown_connections 2024-04-21T09:06:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 -- 172.21.15.66:0/2205426086 wait complete. 2024-04-21T09:06:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 Processor -- start 2024-04-21T09:06:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 -- start start 2024-04-21T09:06:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc100e90 0x7f75dc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc101fe0 0x7f75dc198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10db70 con 0x7f75dc1029a0 2024-04-21T09:06:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10d9f0 con 0x7f75dc101fe0 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e3140640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10dcf0 con 0x7f75dc100e90 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e16dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e16dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43088/0 (socket says 172.21.15.66:43088) 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.223+0000 7f75e16dd640 1 -- 172.21.15.66:0/2051450813 learned_addr learned my addr 172.21.15.66:0/2051450813 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75dbfff640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc101fe0 0x7f75dc198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e0edc640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc100e90 0x7f75dc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e16dd640 1 -- 172.21.15.66:0/2051450813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc100e90 msgr2=0x7f75dc198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e16dd640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc100e90 0x7f75dc198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e16dd640 1 -- 172.21.15.66:0/2051450813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc101fe0 msgr2=0x7f75dc198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e16dd640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc101fe0 0x7f75dc198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e16dd640 1 -- 172.21.15.66:0/2051450813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc19fe60 con 0x7f75dc1029a0 2024-04-21T09:06:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e16dd640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc19f7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009230 tx=0x7f75cc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75d9ffb640 1 -- 172.21.15.66:0/2051450813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc043020 con 0x7f75dc1029a0 2024-04-21T09:06:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75d9ffb640 1 -- 172.21.15.66:0/2051450813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75cc0044c0 con 0x7f75dc1029a0 2024-04-21T09:06:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc1a00f0 con 0x7f75dc1029a0 2024-04-21T09:06:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75d9ffb640 1 -- 172.21.15.66:0/2051450813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003b00 con 0x7f75dc1029a0 2024-04-21T09:06:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.224+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc1a06c0 con 0x7f75dc1029a0 2024-04-21T09:06:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.225+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc108220 con 0x7f75dc1029a0 2024-04-21T09:06:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.226+0000 7f75d9ffb640 1 -- 172.21.15.66:0/2051450813 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f75cc046020 con 0x7f75dc1029a0 2024-04-21T09:06:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.227+0000 7f75d9ffb640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75c40a3bb0 0x7f75c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.227+0000 7f75e0edc640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75c40a3bb0 0x7f75c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.228+0000 7f75d9ffb640 1 -- 172.21.15.66:0/2051450813 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f75cc0f2890 con 0x7f75dc1029a0 2024-04-21T09:06:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.228+0000 7f75e0edc640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75c40a3bb0 0x7f75c40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005fd0 tx=0x7f75d4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.232+0000 7f75d9ffb640 1 -- 172.21.15.66:0/2051450813 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75cc04d020 con 0x7f75dc1029a0 2024-04-21T09:06:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3523343604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T09:06:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] up:active 2024-04-21T09:06:13.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T09:06:13.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T09:06:13.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: Cluster is now healthy 2024-04-21T09:06:13.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T09:06:13.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T09:06:13.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T09:06:13.339 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3357748547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T09:06:13.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3523343604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T09:06:13.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] up:active 2024-04-21T09:06:13.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T09:06:13.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T09:06:13.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: Cluster is now healthy 2024-04-21T09:06:13.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T09:06:13.361 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T09:06:13.361 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T09:06:13.361 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3357748547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T09:06:13.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3523343604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T09:06:13.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] up:active 2024-04-21T09:06:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T09:06:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T09:06:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: Cluster is now healthy 2024-04-21T09:06:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T09:06:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T09:06:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T09:06:13.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3357748547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T09:06:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.446+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f75dc062390 con 0x7f75c40a3bb0 2024-04-21T09:06:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.466+0000 7f75d9ffb640 1 -- 172.21.15.66:0/2051450813 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f75dc062390 con 0x7f75c40a3bb0 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75c40a3bb0 msgr2=0x7f75c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75c40a3bb0 0x7f75c40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005fd0 tx=0x7f75d4009500 comp rx=0 tx=0).stop 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 msgr2=0x7f75dc19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc19f7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009230 tx=0x7f75cc0040c0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 shutdown_connections 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f75c40a3bb0 0x7f75c40a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc1029a0 0x7f75dc19f7f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc101fe0 0x7f75dc198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 --2- 172.21.15.66:0/2051450813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc100e90 0x7f75dc198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.468+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 >> 172.21.15.66:0/2051450813 conn(0x7f75dc0fc400 msgr2=0x7f75dc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.469+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 shutdown_connections 2024-04-21T09:06:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.469+0000 7f75e3140640 1 -- 172.21.15.66:0/2051450813 wait complete. 2024-04-21T09:06:13.480 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-21T09:06:13.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 -- 172.21.15.66:0/188800209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124101fb0 msgr2=0x7ff1241023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 --2- 172.21.15.66:0/188800209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124101fb0 0x7ff1241023d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff114009900 tx=0x7ff11402bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 -- 172.21.15.66:0/188800209 shutdown_connections 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 --2- 172.21.15.66:0/188800209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124103b60 0x7ff12410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 --2- 172.21.15.66:0/188800209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1241031a0 0x7ff124103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 --2- 172.21.15.66:0/188800209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124101fb0 0x7ff1241023d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 -- 172.21.15.66:0/188800209 >> 172.21.15.66:0/188800209 conn(0x7ff1240fd520 msgr2=0x7ff1240ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 -- 172.21.15.66:0/188800209 shutdown_connections 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.586+0000 7ff128cc1640 1 -- 172.21.15.66:0/188800209 wait complete. 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 Processor -- start 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 -- start start 2024-04-21T09:06:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff124101fb0 0x7ff124075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1241031a0 0x7ff124074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124103b60 0x7ff124074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff12410daf0 con 0x7ff124103b60 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff12410d970 con 0x7ff1241031a0 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff128cc1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff12410dc70 con 0x7ff124101fb0 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff122d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124103b60 0x7ff124074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff122d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124103b60 0x7ff124074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43118/0 (socket says 172.21.15.66:43118) 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.587+0000 7ff122d9d640 1 -- 172.21.15.66:0/2568135735 learned_addr learned my addr 172.21.15.66:0/2568135735 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff11bfff640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1241031a0 0x7ff124074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff12259c640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff124101fb0 0x7ff124075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff122d9d640 1 -- 172.21.15.66:0/2568135735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff124101fb0 msgr2=0x7ff124075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff122d9d640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff124101fb0 0x7ff124075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff122d9d640 1 -- 172.21.15.66:0/2568135735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1241031a0 msgr2=0x7ff124074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff122d9d640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1241031a0 0x7ff124074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff122d9d640 1 -- 172.21.15.66:0/2568135735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff124075110 con 0x7ff124103b60 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff122d9d640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124103b60 0x7ff124074880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00efd0 tx=0x7ff11c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff11b7fe640 1 -- 172.21.15.66:0/2568135735 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c003b70 con 0x7ff124103b60 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff11b7fe640 1 -- 172.21.15.66:0/2568135735 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11c003d10 con 0x7ff124103b60 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1241a2370 con 0x7ff124103b60 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff11b7fe640 1 -- 172.21.15.66:0/2568135735 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0108a0 con 0x7ff124103b60 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.588+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1241a2870 con 0x7ff124103b60 2024-04-21T09:06:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.589+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1241093f0 con 0x7ff124103b60 2024-04-21T09:06:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.590+0000 7ff11b7fe640 1 -- 172.21.15.66:0/2568135735 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff11c004020 con 0x7ff124103b60 2024-04-21T09:06:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.591+0000 7ff11b7fe640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1100a3bb0 0x7ff1100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.591+0000 7ff12259c640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1100a3bb0 0x7ff1100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.591+0000 7ff11b7fe640 1 -- 172.21.15.66:0/2568135735 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7ff11c0d0ac0 con 0x7ff124103b60 2024-04-21T09:06:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.591+0000 7ff12259c640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1100a3bb0 0x7ff1100a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff114005ab0 tx=0x7ff114005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.595+0000 7ff11b7fe640 1 -- 172.21.15.66:0/2568135735 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff11c09d260 con 0x7ff124103b60 2024-04-21T09:06:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.811+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff124062420 con 0x7ff1100a3bb0 2024-04-21T09:06:13.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.834+0000 7ff11b7fe640 1 -- 172.21.15.66:0/2568135735 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff124062420 con 0x7ff1100a3bb0 2024-04-21T09:06:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1100a3bb0 msgr2=0x7ff1100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1100a3bb0 0x7ff1100a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff114005ab0 tx=0x7ff114005fd0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124103b60 msgr2=0x7ff124074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124103b60 0x7ff124074880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00efd0 tx=0x7ff11c00c6a0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 shutdown_connections 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1100a3bb0 0x7ff1100a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124103b60 0x7ff124074880 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1241031a0 0x7ff124074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 --2- 172.21.15.66:0/2568135735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff124101fb0 0x7ff124075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 >> 172.21.15.66:0/2568135735 conn(0x7ff1240fd520 msgr2=0x7ff1241065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 shutdown_connections 2024-04-21T09:06:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.836+0000 7ff128cc1640 1 -- 172.21.15.66:0/2568135735 wait complete. 2024-04-21T09:06:13.846 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:06:13.451+0000 7f431c874700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:06:13.846 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:06:13.451+0000 7f431c874700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:06:13.847 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:06:13.451+0000 7f431c874700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:06:13.847 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:06:13.451+0000 7f431c874700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:06:13.847 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:06:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:06:13.451+0000 7f431c874700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:06:13.848 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-21T09:06:13.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 -- 172.21.15.66:0/2142534077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 msgr2=0x7f44001023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 --2- 172.21.15.66:0/2142534077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f44001023c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009900 tx=0x7f43fc02bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 -- 172.21.15.66:0/2142534077 shutdown_connections 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 --2- 172.21.15.66:0/2142534077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4400103b50 0x7f440010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 --2- 172.21.15.66:0/2142534077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4400103190 0x7f4400103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 --2- 172.21.15.66:0/2142534077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f44001023c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 -- 172.21.15.66:0/2142534077 >> 172.21.15.66:0/2142534077 conn(0x7f44000fd530 msgr2=0x7f44000ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 -- 172.21.15.66:0/2142534077 shutdown_connections 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 -- 172.21.15.66:0/2142534077 wait complete. 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.954+0000 7f440805c640 1 Processor -- start 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f440805c640 1 -- start start 2024-04-21T09:06:13.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f440805c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f4400198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f440805c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4400103190 0x7f4400198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f440805c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4400103b50 0x7f440019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f440805c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f440010db30 con 0x7f4400101fa0 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f440805c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f440010d9b0 con 0x7f4400103b50 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f440805c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f440010dcb0 con 0x7f4400103190 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f4400198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f4400198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43138/0 (socket says 172.21.15.66:43138) 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 -- 172.21.15.66:0/2788737653 learned_addr learned my addr 172.21.15.66:0/2788737653 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f44055f7640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4400103190 0x7f4400198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f44065f9640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4400103b50 0x7f440019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 -- 172.21.15.66:0/2788737653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4400103190 msgr2=0x7f4400198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4400103190 0x7f4400198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 -- 172.21.15.66:0/2788737653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4400103b50 msgr2=0x7f440019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4400103b50 0x7f440019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 -- 172.21.15.66:0/2788737653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440019fe10 con 0x7f4400101fa0 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f4405df8640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f4400198240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f43fc000c00 tx=0x7f43fc005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.955+0000 7f43f6ffd640 1 -- 172.21.15.66:0/2788737653 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc043020 con 0x7f4400101fa0 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.956+0000 7f43f6ffd640 1 -- 172.21.15.66:0/2788737653 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43fc0043d0 con 0x7f4400101fa0 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.956+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44001a00a0 con 0x7f4400101fa0 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.956+0000 7f43f6ffd640 1 -- 172.21.15.66:0/2788737653 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc003b20 con 0x7f4400101fa0 2024-04-21T09:06:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.956+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44001a05f0 con 0x7f4400101fa0 2024-04-21T09:06:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.957+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4400109310 con 0x7f4400101fa0 2024-04-21T09:06:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.958+0000 7f43f6ffd640 1 -- 172.21.15.66:0/2788737653 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f43fc046020 con 0x7f4400101fa0 2024-04-21T09:06:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.958+0000 7f43f6ffd640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f43ec0a3bb0 0x7f43ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.959+0000 7f44055f7640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f43ec0a3bb0 0x7f43ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.959+0000 7f43f6ffd640 1 -- 172.21.15.66:0/2788737653 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f43fc0f2b70 con 0x7f4400101fa0 2024-04-21T09:06:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.959+0000 7f44055f7640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f43ec0a3bb0 0x7f43ec0a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4400199700 tx=0x7f43f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:13.962+0000 7f43f6ffd640 1 -- 172.21.15.66:0/2788737653 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43fc04d020 con 0x7f4400101fa0 2024-04-21T09:06:14.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:14 smithi066 ceph-mon[27953]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-04-21T09:06:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:14 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3357748547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T09:06:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:14 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] up:active 2024-04-21T09:06:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:14 smithi066 ceph-mon[27953]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-21T09:06:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:14 smithi066 ceph-mon[27953]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T09:06:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.180+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4400062420 con 0x7f43ec0a3bb0 2024-04-21T09:06:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.185+0000 7f43f6ffd640 1 -- 172.21.15.66:0/2788737653 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4400062420 con 0x7f43ec0a3bb0 2024-04-21T09:06:14.185 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/qa/sv_0/8eb6a6f6-b5c1-4622-877f-9d06cda82824 2024-04-21T09:06:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f43ec0a3bb0 msgr2=0x7f43ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f43ec0a3bb0 0x7f43ec0a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4400199700 tx=0x7f43f0005f90 comp rx=0 tx=0).stop 2024-04-21T09:06:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 msgr2=0x7f4400198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f4400198240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f43fc000c00 tx=0x7f43fc005e60 comp rx=0 tx=0).stop 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 shutdown_connections 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f43ec0a3bb0 0x7f43ec0a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4400103b50 0x7f440019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4400103190 0x7f4400198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 --2- 172.21.15.66:0/2788737653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4400101fa0 0x7f4400198240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 >> 172.21.15.66:0/2788737653 conn(0x7f44000fd530 msgr2=0x7f44001064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 shutdown_connections 2024-04-21T09:06:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.187+0000 7f440805c640 1 -- 172.21.15.66:0/2788737653 wait complete. 2024-04-21T09:06:14.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 -- 172.21.15.66:0/866434885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a81031a0 msgr2=0x7f06a8103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 --2- 172.21.15.66:0/866434885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a81031a0 0x7f06a8103620 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0694009900 tx=0x7f069402bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 -- 172.21.15.66:0/866434885 shutdown_connections 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 --2- 172.21.15.66:0/866434885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a8103b60 0x7f06a810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 --2- 172.21.15.66:0/866434885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a81031a0 0x7f06a8103620 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 --2- 172.21.15.66:0/866434885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06a8101fb0 0x7f06a81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 -- 172.21.15.66:0/866434885 >> 172.21.15.66:0/866434885 conn(0x7f06a80fd520 msgr2=0x7f06a80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 -- 172.21.15.66:0/866434885 shutdown_connections 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.305+0000 7f06aef15640 1 -- 172.21.15.66:0/866434885 wait complete. 2024-04-21T09:06:14.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 Processor -- start 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 -- start start 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8101fb0 0x7f06a81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06a81031a0 0x7f06a8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a8103b60 0x7f06a819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810daf0 con 0x7f06a8101fb0 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810d970 con 0x7f06a81031a0 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06aef15640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810dc70 con 0x7f06a8103b60 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8101fb0 0x7f06a81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8101fb0 0x7f06a81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43162/0 (socket says 172.21.15.66:43162) 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 -- 172.21.15.66:0/2474339912 learned_addr learned my addr 172.21.15.66:0/2474339912 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06a7fff640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06a81031a0 0x7f06a8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06ad4b2640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a8103b60 0x7f06a819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 -- 172.21.15.66:0/2474339912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a8103b60 msgr2=0x7f06a819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a8103b60 0x7f06a819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 -- 172.21.15.66:0/2474339912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06a81031a0 msgr2=0x7f06a8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06a81031a0 0x7f06a8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.306+0000 7f06accb1640 1 -- 172.21.15.66:0/2474339912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a819fde0 con 0x7f06a8101fb0 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.307+0000 7f06accb1640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8101fb0 0x7f06a81981c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f069c00e9c0 tx=0x7f069c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.307+0000 7f06a5ffb640 1 -- 172.21.15.66:0/2474339912 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c00cd80 con 0x7f06a8101fb0 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.307+0000 7f06a5ffb640 1 -- 172.21.15.66:0/2474339912 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c0035f0 con 0x7f06a8101fb0 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.307+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81a00d0 con 0x7f06a8101fb0 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.307+0000 7f06a5ffb640 1 -- 172.21.15.66:0/2474339912 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c010690 con 0x7f06a8101fb0 2024-04-21T09:06:14.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.307+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81a05f0 con 0x7f06a8101fb0 2024-04-21T09:06:14.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.308+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a81093a0 con 0x7f06a8101fb0 2024-04-21T09:06:14.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.309+0000 7f06a5ffb640 1 -- 172.21.15.66:0/2474339912 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f069c0037a0 con 0x7f06a8101fb0 2024-04-21T09:06:14.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.309+0000 7f06a5ffb640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f06980a3bb0 0x7f06980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.310+0000 7f06a7fff640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f06980a3bb0 0x7f06980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.310+0000 7f06a5ffb640 1 -- 172.21.15.66:0/2474339912 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f069c0d0a50 con 0x7f06a8101fb0 2024-04-21T09:06:14.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.310+0000 7f06a7fff640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f06980a3bb0 0x7f06980a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0694009230 tx=0x7f0694005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:14.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.314+0000 7f06a5ffb640 1 -- 172.21.15.66:0/2474339912 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f069c09d1f0 con 0x7f06a8101fb0 2024-04-21T09:06:14.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:14 smithi083 ceph-mon[31413]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-04-21T09:06:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:14 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3357748547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T09:06:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:14 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] up:active 2024-04-21T09:06:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:14 smithi083 ceph-mon[31413]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-21T09:06:14.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:14 smithi083 ceph-mon[31413]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T09:06:14.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.533+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f06a8062390 con 0x7f06980a3bb0 2024-04-21T09:06:14.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.557+0000 7f06a5ffb640 1 -- 172.21.15.66:0/2474339912 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f06a8062390 con 0x7f06980a3bb0 2024-04-21T09:06:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f06980a3bb0 msgr2=0x7f06980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f06980a3bb0 0x7f06980a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0694009230 tx=0x7f0694005fd0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8101fb0 msgr2=0x7f06a81981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8101fb0 0x7f06a81981c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f069c00e9c0 tx=0x7f069c00ee80 comp rx=0 tx=0).stop 2024-04-21T09:06:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 shutdown_connections 2024-04-21T09:06:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f06980a3bb0 0x7f06980a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06a8103b60 0x7f06a819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06a81031a0 0x7f06a8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 --2- 172.21.15.66:0/2474339912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8101fb0 0x7f06a81981c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 >> 172.21.15.66:0/2474339912 conn(0x7f06a80fd520 msgr2=0x7f06a8106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 shutdown_connections 2024-04-21T09:06:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.559+0000 7f06aef15640 1 -- 172.21.15.66:0/2474339912 wait complete. 2024-04-21T09:06:14.571 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-21T09:06:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:14 smithi186 ceph-mon[31505]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-04-21T09:06:14.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:14 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3357748547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T09:06:14.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:14 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] up:active 2024-04-21T09:06:14.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:14 smithi186 ceph-mon[31505]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-21T09:06:14.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:14 smithi186 ceph-mon[31505]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T09:06:14.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/679616847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 msgr2=0x7ff2b4104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/679616847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4104ea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009900 tx=0x7ff2a402bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:14.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/679616847 shutdown_connections 2024-04-21T09:06:14.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/679616847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b4107d10 0x7ff2b410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/679616847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b41053e0 0x7ff2b41077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/679616847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4104ea0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/679616847 >> 172.21.15.66:0/679616847 conn(0x7ff2b40fc3e0 msgr2=0x7ff2b40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/679616847 shutdown_connections 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/679616847 wait complete. 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 Processor -- start 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.679+0000 7ff2bb7ef640 1 -- start start 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2bb7ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2bb7ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b41053e0 0x7ff2b4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2bb7ef640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b4107d10 0x7ff2b419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2bb7ef640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410dac0 con 0x7ff2b4102ab0 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2bb7ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410d940 con 0x7ff2b41053e0 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2bb7ef640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410dc40 con 0x7ff2b4107d10 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50330/0 (socket says 172.21.15.66:50330) 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 -- 172.21.15.66:0/194464063 learned_addr learned my addr 172.21.15.66:0/194464063 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b9d8c640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b4107d10 0x7ff2b419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 -- 172.21.15.66:0/194464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b4107d10 msgr2=0x7ff2b419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b4107d10 0x7ff2b419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 -- 172.21.15.66:0/194464063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b41053e0 msgr2=0x7ff2b4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b8d8a640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b41053e0 0x7ff2b4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b41053e0 0x7ff2b4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 -- 172.21.15.66:0/194464063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b419fe20 con 0x7ff2b4102ab0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2b958b640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4198200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4006010 tx=0x7ff2a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2aa7fc640 1 -- 172.21.15.66:0/194464063 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4043020 con 0x7ff2b4102ab0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2aa7fc640 1 -- 172.21.15.66:0/194464063 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a4004aa0 con 0x7ff2b4102ab0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.680+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41a00b0 con 0x7ff2b4102ab0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.681+0000 7ff2aa7fc640 1 -- 172.21.15.66:0/194464063 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4003c70 con 0x7ff2b4102ab0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.681+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41a0600 con 0x7ff2b4102ab0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.682+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b40fded0 con 0x7ff2b4102ab0 2024-04-21T09:06:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.683+0000 7ff2aa7fc640 1 -- 172.21.15.66:0/194464063 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2a4046020 con 0x7ff2b4102ab0 2024-04-21T09:06:14.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.683+0000 7ff2aa7fc640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:14.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.683+0000 7ff2b8d8a640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:14.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.683+0000 7ff2aa7fc640 1 -- 172.21.15.66:0/194464063 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7ff2a40f2ac0 con 0x7ff2b4102ab0 2024-04-21T09:06:14.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.684+0000 7ff2b8d8a640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005fd0 tx=0x7ff2b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.688+0000 7ff2aa7fc640 1 -- 172.21.15.66:0/194464063 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2a404d020 con 0x7ff2b4102ab0 2024-04-21T09:06:14.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.902+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff2b4062420 con 0x7ff2a00a3bb0 2024-04-21T09:06:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.907+0000 7ff2aa7fc640 1 -- 172.21.15.66:0/194464063 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff2b4062420 con 0x7ff2a00a3bb0 2024-04-21T09:06:14.908 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/qa/sv_1/1215737e-19f8-4b71-ae4b-bbe3123bfec4 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff2a00a3bb0 msgr2=0x7ff2a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005fd0 tx=0x7ff2b0005850 comp rx=0 tx=0).stop 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 msgr2=0x7ff2b4198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4198200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4006010 tx=0x7ff2a40042f0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 shutdown_connections 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b4107d10 0x7ff2b419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b41053e0 0x7ff2b4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 --2- 172.21.15.66:0/194464063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102ab0 0x7ff2b4198200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 >> 172.21.15.66:0/194464063 conn(0x7ff2b40fc3e0 msgr2=0x7ff2b4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 shutdown_connections 2024-04-21T09:06:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:14.909+0000 7ff2bb7ef640 1 -- 172.21.15.66:0/194464063 wait complete. 2024-04-21T09:06:14.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:06:15.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 -- 172.21.15.66:0/1975954107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 msgr2=0x7f9d7010a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 --2- 172.21.15.66:0/1975954107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d7010a0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009ca0 tx=0x7f9d6802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 -- 172.21.15.66:0/1975954107 shutdown_connections 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 --2- 172.21.15.66:0/1975954107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d7010a0a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 --2- 172.21.15.66:0/1975954107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70105320 0x7f9d70107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 --2- 172.21.15.66:0/1975954107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d701029f0 0x7f9d70104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 -- 172.21.15.66:0/1975954107 >> 172.21.15.66:0/1975954107 conn(0x7f9d700fc3e0 msgr2=0x7f9d700fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.024+0000 7f9d75e55640 1 -- 172.21.15.66:0/1975954107 shutdown_connections 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 -- 172.21.15.66:0/1975954107 wait complete. 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 Processor -- start 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 -- start start 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d701029f0 0x7f9d70102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70105320 0x7f9d701009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d70100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010dab0 con 0x7f9d70107c50 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010d930 con 0x7f9d701029f0 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.025+0000 7f9d75e55640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010dc30 con 0x7f9d70105320 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d70100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d701029f0 0x7f9d70102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d70100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50346/0 (socket says 172.21.15.66:50346) 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6effd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70105320 0x7f9d701009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6ffff640 1 -- 172.21.15.66:0/2472018933 learned_addr learned my addr 172.21.15.66:0/2472018933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6f7fe640 1 -- 172.21.15.66:0/2472018933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70105320 msgr2=0x7f9d701009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6f7fe640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70105320 0x7f9d701009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6f7fe640 1 -- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 msgr2=0x7f9d70100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6f7fe640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d70100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6f7fe640 1 -- 172.21.15.66:0/2472018933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d70101770 con 0x7f9d701029f0 2024-04-21T09:06:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6ffff640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:06:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6f7fe640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d701029f0 0x7f9d70102360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e9f0 tx=0x7f9d6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000ccc0 con 0x7f9d701029f0 2024-04-21T09:06:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701a2270 con 0x7f9d701029f0 2024-04-21T09:06:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d600108e0 con 0x7f9d701029f0 2024-04-21T09:06:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60018880 con 0x7f9d701029f0 2024-04-21T09:06:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.026+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701a2810 con 0x7f9d701029f0 2024-04-21T09:06:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.027+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d700fded0 con 0x7f9d701029f0 2024-04-21T09:06:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.029+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d60010430 con 0x7f9d701029f0 2024-04-21T09:06:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.029+0000 7f9d6cff9640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d580a3bb0 0x7f9d580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.029+0000 7f9d6effd640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d580a3bb0 0x7f9d580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.029+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f9d600d0dc0 con 0x7f9d701029f0 2024-04-21T09:06:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.030+0000 7f9d6effd640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d580a3bb0 0x7f9d580a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9d701020d0 tx=0x7f9d64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.033+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d6009cf10 con 0x7f9d701029f0 2024-04-21T09:06:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.111+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9d60099580 con 0x7f9d701029f0 2024-04-21T09:06:15.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.264+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d7002d5c0 con 0x7f9d701029f0 2024-04-21T09:06:15.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.266+0000 7f9d6cff9640 1 -- 172.21.15.66:0/2472018933 <== mon.2 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11867 (secure 0 0 0) 0x7f9d600ce0c0 con 0x7f9d701029f0 2024-04-21T09:06:15.266 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:15.267 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:06:13.097844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24476,24490]},"id":1}]} 2024-04-21T09:06:15.267 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-04-21T09:06:15.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d580a3bb0 msgr2=0x7f9d580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d580a3bb0 0x7f9d580a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9d701020d0 tx=0x7f9d64005f90 comp rx=0 tx=0).stop 2024-04-21T09:06:15.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d701029f0 msgr2=0x7f9d70102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d701029f0 0x7f9d70102360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e9f0 tx=0x7f9d6000eeb0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 shutdown_connections 2024-04-21T09:06:15.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d580a3bb0 0x7f9d580a6070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70107c50 0x7f9d70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70105320 0x7f9d701009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.267+0000 7f9d75e55640 1 --2- 172.21.15.66:0/2472018933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d701029f0 0x7f9d70102360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.268+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 >> 172.21.15.66:0/2472018933 conn(0x7f9d700fc3e0 msgr2=0x7f9d701034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:15.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.268+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 shutdown_connections 2024-04-21T09:06:15.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.268+0000 7f9d75e55640 1 -- 172.21.15.66:0/2472018933 wait complete. 2024-04-21T09:06:15.278 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:15 smithi066 ceph-mon[27953]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:15.278 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:15 smithi066 ceph-mon[27953]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:15.279 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:06:15.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:15 smithi083 ceph-mon[31413]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:15.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:15 smithi083 ceph-mon[31413]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 -- 172.21.15.66:0/836539399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 msgr2=0x7f3df0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 --2- 172.21.15.66:0/836539399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df0103610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 -- 172.21.15.66:0/836539399 shutdown_connections 2024-04-21T09:06:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 --2- 172.21.15.66:0/836539399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df0103b50 0x7f3df010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 --2- 172.21.15.66:0/836539399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df0103610 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 --2- 172.21.15.66:0/836539399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0101fa0 0x7f3df01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 -- 172.21.15.66:0/836539399 >> 172.21.15.66:0/836539399 conn(0x7f3df00fd530 msgr2=0x7f3df00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.381+0000 7f3df657e640 1 -- 172.21.15.66:0/836539399 shutdown_connections 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 -- 172.21.15.66:0/836539399 wait complete. 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 Processor -- start 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 -- start start 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0101fa0 0x7f3df0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df0103b50 0x7f3df019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010dae0 con 0x7f3df0103190 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010d960 con 0x7f3df0101fa0 2024-04-21T09:06:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.382+0000 7f3df657e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010dc60 con 0x7f3df0103b50 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df01986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50352/0 (socket says 172.21.15.66:50352) 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 -- 172.21.15.66:0/617112219 learned_addr learned my addr 172.21.15.66:0/617112219 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3deffff640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0101fa0 0x7f3df0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 -- 172.21.15.66:0/617112219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df0103b50 msgr2=0x7f3df019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df0103b50 0x7f3df019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 -- 172.21.15.66:0/617112219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0101fa0 msgr2=0x7f3df0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0101fa0 0x7f3df0198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 -- 172.21.15.66:0/617112219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df019fd90 con 0x7f3df0103190 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3def7fe640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df01986b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3de0006010 tx=0x7f3de0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3ded7fa640 1 -- 172.21.15.66:0/617112219 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0043020 con 0x7f3df0103190 2024-04-21T09:06:15.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3ded7fa640 1 -- 172.21.15.66:0/617112219 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0004440 con 0x7f3df0103190 2024-04-21T09:06:15.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df01a0020 con 0x7f3df0103190 2024-04-21T09:06:15.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3ded7fa640 1 -- 172.21.15.66:0/617112219 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0003bf0 con 0x7f3df0103190 2024-04-21T09:06:15.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.383+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a05c0 con 0x7f3df0103190 2024-04-21T09:06:15.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.384+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df01092c0 con 0x7f3df0103190 2024-04-21T09:06:15.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.385+0000 7f3ded7fa640 1 -- 172.21.15.66:0/617112219 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3de0046020 con 0x7f3df0103190 2024-04-21T09:06:15.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.386+0000 7f3ded7fa640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3dd80a3cc0 0x7f3dd80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.386+0000 7f3ded7fa640 1 -- 172.21.15.66:0/617112219 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f3de00f3060 con 0x7f3df0103190 2024-04-21T09:06:15.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.386+0000 7f3deffff640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3dd80a3cc0 0x7f3dd80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.387+0000 7f3deffff640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3dd80a3cc0 0x7f3dd80a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3df0102ff0 tx=0x7f3de4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:15.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.390+0000 7f3ded7fa640 1 -- 172.21.15.66:0/617112219 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3de004d020 con 0x7f3df0103190 2024-04-21T09:06:15.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.577+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3df002d7c0 con 0x7f3df0103190 2024-04-21T09:06:15.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.579+0000 7f3ded7fa640 1 -- 172.21.15.66:0/617112219 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7f3de00bf7b0 con 0x7f3df0103190 2024-04-21T09:06:15.580 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:15.580 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3dd80a3cc0 msgr2=0x7f3dd80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3dd80a3cc0 0x7f3dd80a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3df0102ff0 tx=0x7f3de4005f90 comp rx=0 tx=0).stop 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 msgr2=0x7f3df01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df01986b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3de0006010 tx=0x7f3de0004040 comp rx=0 tx=0).stop 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 shutdown_connections 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3dd80a3cc0 0x7f3dd80a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df0103b50 0x7f3df019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0103190 0x7f3df01986b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 --2- 172.21.15.66:0/617112219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0101fa0 0x7f3df0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 >> 172.21.15.66:0/617112219 conn(0x7f3df00fd530 msgr2=0x7f3df0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:15.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 shutdown_connections 2024-04-21T09:06:15.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.581+0000 7f3df657e640 1 -- 172.21.15.66:0/617112219 wait complete. 2024-04-21T09:06:15.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:15 smithi186 ceph-mon[31505]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:15 smithi186 ceph-mon[31505]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:15.593 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-21T09:06:15.593 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:06:15.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 -- 172.21.15.66:0/3131794338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 msgr2=0x7fa26810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/3131794338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26810a3e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa260009ca0 tx=0x7fa26002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 -- 172.21.15.66:0/3131794338 shutdown_connections 2024-04-21T09:06:15.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/3131794338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26810a3e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/3131794338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa268103190 0x7fa268103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/3131794338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa268101fa0 0x7fa2681023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 -- 172.21.15.66:0/3131794338 >> 172.21.15.66:0/3131794338 conn(0x7fa2680fd530 msgr2=0x7fa2680ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 -- 172.21.15.66:0/3131794338 shutdown_connections 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 -- 172.21.15.66:0/3131794338 wait complete. 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.695+0000 7fa26e4c0640 1 Processor -- start 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26e4c0640 1 -- start start 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26e4c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa268101fa0 0x7fa268198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26e4c0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa268103190 0x7fa268198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26e4c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26e4c0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa26810dae0 con 0x7fa268103b50 2024-04-21T09:06:15.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26e4c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa26810d960 con 0x7fa268101fa0 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26e4c0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa26810dc60 con 0x7fa268103190 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50372/0 (socket says 172.21.15.66:50372) 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 -- 172.21.15.66:0/1649757543 learned_addr learned my addr 172.21.15.66:0/1649757543 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa267fff640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa268101fa0 0x7fa268198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa2677fe640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa268103190 0x7fa268198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 -- 172.21.15.66:0/1649757543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa268103190 msgr2=0x7fa268198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa268103190 0x7fa268198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 -- 172.21.15.66:0/1649757543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa268101fa0 msgr2=0x7fa268198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa268101fa0 0x7fa268198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 -- 172.21.15.66:0/1649757543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26819fd20 con 0x7fa268103b50 2024-04-21T09:06:15.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa2677fe640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa268103190 0x7fa268198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa267fff640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa268101fa0 0x7fa268198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.696+0000 7fa26ca5d640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26819f6b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa26002c700 tx=0x7fa260034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.697+0000 7fa2657fa640 1 -- 172.21.15.66:0/1649757543 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260039040 con 0x7fa268103b50 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.697+0000 7fa2657fa640 1 -- 172.21.15.66:0/1649757543 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa260044070 con 0x7fa268103b50 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.697+0000 7fa2657fa640 1 -- 172.21.15.66:0/1649757543 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600042a0 con 0x7fa268103b50 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.697+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26819ffb0 con 0x7fa268103b50 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.697+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2681a0470 con 0x7fa268103b50 2024-04-21T09:06:15.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.698+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2681092c0 con 0x7fa268103b50 2024-04-21T09:06:15.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.699+0000 7fa2657fa640 1 -- 172.21.15.66:0/1649757543 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa260003970 con 0x7fa268103b50 2024-04-21T09:06:15.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.699+0000 7fa2657fa640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2500a3cc0 0x7fa2500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:15.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.699+0000 7fa267fff640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2500a3cc0 0x7fa2500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:15.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.699+0000 7fa2657fa640 1 -- 172.21.15.66:0/1649757543 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fa2600f2aa0 con 0x7fa268103b50 2024-04-21T09:06:15.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.700+0000 7fa267fff640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2500a3cc0 0x7fa2500a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0058c0 tx=0x7fa25c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:15.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.704+0000 7fa2657fa640 1 -- 172.21.15.66:0/1649757543 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2600bf1f0 con 0x7fa268103b50 2024-04-21T09:06:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.886+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa268062390 con 0x7fa2500a3cc0 2024-04-21T09:06:15.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.896+0000 7fa2657fa640 1 -- 172.21.15.66:0/1649757543 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7fa268062390 con 0x7fa2500a3cc0 2024-04-21T09:06:15.901 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:15.903 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:06:15.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2500a3cc0 msgr2=0x7fa2500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2500a3cc0 0x7fa2500a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0058c0 tx=0x7fa25c005850 comp rx=0 tx=0).stop 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 msgr2=0x7fa26819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26819f6b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa26002c700 tx=0x7fa260034760 comp rx=0 tx=0).stop 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 shutdown_connections 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2500a3cc0 0x7fa2500a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa268103b50 0x7fa26819f6b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa268103190 0x7fa268198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 --2- 172.21.15.66:0/1649757543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa268101fa0 0x7fa268198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 >> 172.21.15.66:0/1649757543 conn(0x7fa2680fd530 msgr2=0x7fa268106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 shutdown_connections 2024-04-21T09:06:15.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:15.900+0000 7fa26e4c0640 1 -- 172.21.15.66:0/1649757543 wait complete. 2024-04-21T09:06:15.911 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-04-21T09:06:14.301800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596436,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":141,"num_read_kb":124,"num_write":272,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356824,"kb_used_data":11812,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585000,"statfs":{"total":1151940427776,"available":1151575040000,"internally_reserved":0,"allocated":12095488,"data_stored":7664924,"data_compressed":317901,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1569,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":44,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003662"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007602+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:05:53.007602+0000","last_peered":"2024-04-21T09:05:53.007602+0000","last_clean":"2024-04-21T09:05:53.007602+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:05:53.007602+0000","last_undegraded":"2024-04-21T09:05:53.007602+0000","last_fullsized":"2024-04-21T09:05:53.007602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006546+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:05:53.006546+0000","last_peered":"2024-04-21T09:05:53.006546+0000","last_clean":"2024-04-21T09:05:53.006546+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:05:53.006546+0000","last_undegraded":"2024-04-21T09:05:53.006546+0000","last_fullsized":"2024-04-21T09:05:53.006546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007483+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:05:53.007483+0000","last_peered":"2024-04-21T09:05:53.007483+0000","last_clean":"2024-04-21T09:05:53.007483+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:05:53.007483+0000","last_undegraded":"2024-04-21T09:05:53.007483+0000","last_fullsized":"2024-04-21T09:05:53.007483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998866+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:05:52.998866+0000","last_peered":"2024-04-21T09:05:52.998866+0000","last_clean":"2024-04-21T09:05:52.998866+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:05:52.998866+0000","last_undegraded":"2024-04-21T09:05:52.998866+0000","last_fullsized":"2024-04-21T09:05:52.998866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006461+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:05:53.006461+0000","last_peered":"2024-04-21T09:05:53.006461+0000","last_clean":"2024-04-21T09:05:53.006461+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:05:53.006461+0000","last_undegraded":"2024-04-21T09:05:53.006461+0000","last_fullsized":"2024-04-21T09:05:53.006461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007439+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:05:53.007439+0000","last_peered":"2024-04-21T09:05:53.007439+0000","last_clean":"2024-04-21T09:05:53.007439+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:05:53.007439+0000","last_undegraded":"2024-04-21T09:05:53.007439+0000","last_fullsized":"2024-04-21T09:05:53.007439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006512+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:05:53.006512+0000","last_peered":"2024-04-21T09:05:53.006512+0000","last_clean":"2024-04-21T09:05:53.006512+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:05:53.006512+0000","last_undegraded":"2024-04-21T09:05:53.006512+0000","last_fullsized":"2024-04-21T09:05:53.006512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.005940+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:05:53.005940+0000","last_peered":"2024-04-21T09:05:53.005940+0000","last_clean":"2024-04-21T09:05:53.005940+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:05:53.005940+0000","last_undegraded":"2024-04-21T09:05:53.005940+0000","last_fullsized":"2024-04-21T09:05:53.005940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007060+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:05:53.007060+0000","last_peered":"2024-04-21T09:05:53.007060+0000","last_clean":"2024-04-21T09:05:53.007060+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:05:53.007060+0000","last_undegraded":"2024-04-21T09:05:53.007060+0000","last_fullsized":"2024-04-21T09:05:53.007060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007411+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:05:53.007411+0000","last_peered":"2024-04-21T09:05:53.007411+0000","last_clean":"2024-04-21T09:05:53.007411+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:05:53.007411+0000","last_undegraded":"2024-04-21T09:05:53.007411+0000","last_fullsized":"2024-04-21T09:05:53.007411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.008007+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:05:53.008007+0000","last_peered":"2024-04-21T09:05:53.008007+0000","last_clean":"2024-04-21T09:05:53.008007+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:05:53.008007+0000","last_undegraded":"2024-04-21T09:05:53.008007+0000","last_fullsized":"2024-04-21T09:05:53.008007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998723+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:05:52.998723+0000","last_peered":"2024-04-21T09:05:52.998723+0000","last_clean":"2024-04-21T09:05:52.998723+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:05:52.998723+0000","last_undegraded":"2024-04-21T09:05:52.998723+0000","last_fullsized":"2024-04-21T09:05:52.998723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007431+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:05:53.007431+0000","last_peered":"2024-04-21T09:05:53.007431+0000","last_clean":"2024-04-21T09:05:53.007431+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:05:53.007431+0000","last_undegraded":"2024-04-21T09:05:53.007431+0000","last_fullsized":"2024-04-21T09:05:53.007431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006834+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:05:53.006834+0000","last_peered":"2024-04-21T09:05:53.006834+0000","last_clean":"2024-04-21T09:05:53.006834+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:05:53.006834+0000","last_undegraded":"2024-04-21T09:05:53.006834+0000","last_fullsized":"2024-04-21T09:05:53.006834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007249+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:05:53.007249+0000","last_peered":"2024-04-21T09:05:53.007249+0000","last_clean":"2024-04-21T09:05:53.007249+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:05:53.007249+0000","last_undegraded":"2024-04-21T09:05:53.007249+0000","last_fullsized":"2024-04-21T09:05:53.007249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.005990+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:05:53.005990+0000","last_peered":"2024-04-21T09:05:53.005990+0000","last_clean":"2024-04-21T09:05:53.005990+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:05:53.005990+0000","last_undegraded":"2024-04-21T09:05:53.005990+0000","last_fullsized":"2024-04-21T09:05:53.005990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007514+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:05:53.007514+0000","last_peered":"2024-04-21T09:05:53.007514+0000","last_clean":"2024-04-21T09:05:53.007514+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:05:53.007514+0000","last_undegraded":"2024-04-21T09:05:53.007514+0000","last_fullsized":"2024-04-21T09:05:53.007514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998384+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:05:52.998384+0000","last_peered":"2024-04-21T09:05:52.998384+0000","last_clean":"2024-04-21T09:05:52.998384+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:05:52.998384+0000","last_undegraded":"2024-04-21T09:05:52.998384+0000","last_fullsized":"2024-04-21T09:05:52.998384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998768+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:05:52.998768+0000","last_peered":"2024-04-21T09:05:52.998768+0000","last_clean":"2024-04-21T09:05:52.998768+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:05:52.998768+0000","last_undegraded":"2024-04-21T09:05:52.998768+0000","last_fullsized":"2024-04-21T09:05:52.998768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998379+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:05:52.998379+0000","last_peered":"2024-04-21T09:05:52.998379+0000","last_clean":"2024-04-21T09:05:52.998379+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:05:52.998379+0000","last_undegraded":"2024-04-21T09:05:52.998379+0000","last_fullsized":"2024-04-21T09:05:52.998379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998364+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:05:52.998364+0000","last_peered":"2024-04-21T09:05:52.998364+0000","last_clean":"2024-04-21T09:05:52.998364+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:05:52.998364+0000","last_undegraded":"2024-04-21T09:05:52.998364+0000","last_fullsized":"2024-04-21T09:05:52.998364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007732+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:05:53.007732+0000","last_peered":"2024-04-21T09:05:53.007732+0000","last_clean":"2024-04-21T09:05:53.007732+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:05:53.007732+0000","last_undegraded":"2024-04-21T09:05:53.007732+0000","last_fullsized":"2024-04-21T09:05:53.007732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998354+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:05:52.998354+0000","last_peered":"2024-04-21T09:05:52.998354+0000","last_clean":"2024-04-21T09:05:52.998354+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:05:52.998354+0000","last_undegraded":"2024-04-21T09:05:52.998354+0000","last_fullsized":"2024-04-21T09:05:52.998354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007958+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:05:53.007958+0000","last_peered":"2024-04-21T09:05:53.007958+0000","last_clean":"2024-04-21T09:05:53.007958+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:05:53.007958+0000","last_undegraded":"2024-04-21T09:05:53.007958+0000","last_fullsized":"2024-04-21T09:05:53.007958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.005980+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:05:53.005980+0000","last_peered":"2024-04-21T09:05:53.005980+0000","last_clean":"2024-04-21T09:05:53.005980+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:05:53.005980+0000","last_undegraded":"2024-04-21T09:05:53.005980+0000","last_fullsized":"2024-04-21T09:05:53.005980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998507+0000","last_change":"2024-04-21T09:05:51.013392+0000","last_active":"2024-04-21T09:05:52.998507+0000","last_peered":"2024-04-21T09:05:52.998507+0000","last_clean":"2024-04-21T09:05:52.998507+0000","last_became_active":"2024-04-21T09:05:51.012918+0000","last_became_peered":"2024-04-21T09:05:51.012918+0000","last_unstale":"2024-04-21T09:05:52.998507+0000","last_undegraded":"2024-04-21T09:05:52.998507+0000","last_fullsized":"2024-04-21T09:05:52.998507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:31.142591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.100086+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:06:12.100086+0000","last_peered":"2024-04-21T09:06:12.100086+0000","last_clean":"2024-04-21T09:06:12.100086+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:06:12.100086+0000","last_undegraded":"2024-04-21T09:06:12.100086+0000","last_fullsized":"2024-04-21T09:06:12.100086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007255+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:05:53.007255+0000","last_peered":"2024-04-21T09:05:53.007255+0000","last_clean":"2024-04-21T09:05:53.007255+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:05:53.007255+0000","last_undegraded":"2024-04-21T09:05:53.007255+0000","last_fullsized":"2024-04-21T09:05:53.007255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998464+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:05:52.998464+0000","last_peered":"2024-04-21T09:05:52.998464+0000","last_clean":"2024-04-21T09:05:52.998464+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:05:52.998464+0000","last_undegraded":"2024-04-21T09:05:52.998464+0000","last_fullsized":"2024-04-21T09:05:52.998464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007335+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:05:53.007335+0000","last_peered":"2024-04-21T09:05:53.007335+0000","last_clean":"2024-04-21T09:05:53.007335+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:05:53.007335+0000","last_undegraded":"2024-04-21T09:05:53.007335+0000","last_fullsized":"2024-04-21T09:05:53.007335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007048+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:05:53.007048+0000","last_peered":"2024-04-21T09:05:53.007048+0000","last_clean":"2024-04-21T09:05:53.007048+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:05:53.007048+0000","last_undegraded":"2024-04-21T09:05:53.007048+0000","last_fullsized":"2024-04-21T09:05:53.007048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998560+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:05:52.998560+0000","last_peered":"2024-04-21T09:05:52.998560+0000","last_clean":"2024-04-21T09:05:52.998560+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:05:52.998560+0000","last_undegraded":"2024-04-21T09:05:52.998560+0000","last_fullsized":"2024-04-21T09:05:52.998560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091692+0000","last_change":"2024-04-21T09:05:45.965239+0000","last_active":"2024-04-21T09:06:11.091692+0000","last_peered":"2024-04-21T09:06:11.091692+0000","last_clean":"2024-04-21T09:06:11.091692+0000","last_became_active":"2024-04-21T09:05:45.964879+0000","last_became_peered":"2024-04-21T09:05:45.964879+0000","last_unstale":"2024-04-21T09:06:11.091692+0000","last_undegraded":"2024-04-21T09:06:11.091692+0000","last_fullsized":"2024-04-21T09:06:11.091692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:26.985732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007611+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:05:53.007611+0000","last_peered":"2024-04-21T09:05:53.007611+0000","last_clean":"2024-04-21T09:05:53.007611+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:05:53.007611+0000","last_undegraded":"2024-04-21T09:05:53.007611+0000","last_fullsized":"2024-04-21T09:05:53.007611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998430+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:05:52.998430+0000","last_peered":"2024-04-21T09:05:52.998430+0000","last_clean":"2024-04-21T09:05:52.998430+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:05:52.998430+0000","last_undegraded":"2024-04-21T09:05:52.998430+0000","last_fullsized":"2024-04-21T09:05:52.998430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007340+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:05:53.007340+0000","last_peered":"2024-04-21T09:05:53.007340+0000","last_clean":"2024-04-21T09:05:53.007340+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:05:53.007340+0000","last_undegraded":"2024-04-21T09:05:53.007340+0000","last_fullsized":"2024-04-21T09:05:53.007340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006847+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:05:53.006847+0000","last_peered":"2024-04-21T09:05:53.006847+0000","last_clean":"2024-04-21T09:05:53.006847+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:05:53.006847+0000","last_undegraded":"2024-04-21T09:05:53.006847+0000","last_fullsized":"2024-04-21T09:05:53.006847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007375+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:05:53.007375+0000","last_peered":"2024-04-21T09:05:53.007375+0000","last_clean":"2024-04-21T09:05:53.007375+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:05:53.007375+0000","last_undegraded":"2024-04-21T09:05:53.007375+0000","last_fullsized":"2024-04-21T09:05:53.007375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.099512+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:06:12.099512+0000","last_peered":"2024-04-21T09:06:12.099512+0000","last_clean":"2024-04-21T09:06:12.099512+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:06:12.099512+0000","last_undegraded":"2024-04-21T09:06:12.099512+0000","last_fullsized":"2024-04-21T09:06:12.099512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006231+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:05:53.006231+0000","last_peered":"2024-04-21T09:05:53.006231+0000","last_clean":"2024-04-21T09:05:53.006231+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:05:53.006231+0000","last_undegraded":"2024-04-21T09:05:53.006231+0000","last_fullsized":"2024-04-21T09:05:53.006231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006317+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:05:53.006317+0000","last_peered":"2024-04-21T09:05:53.006317+0000","last_clean":"2024-04-21T09:05:53.006317+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:05:53.006317+0000","last_undegraded":"2024-04-21T09:05:53.006317+0000","last_fullsized":"2024-04-21T09:05:53.006317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007398+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:05:53.007398+0000","last_peered":"2024-04-21T09:05:53.007398+0000","last_clean":"2024-04-21T09:05:53.007398+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:05:53.007398+0000","last_undegraded":"2024-04-21T09:05:53.007398+0000","last_fullsized":"2024-04-21T09:05:53.007398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.997886+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:05:52.997886+0000","last_peered":"2024-04-21T09:05:52.997886+0000","last_clean":"2024-04-21T09:05:52.997886+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:05:52.997886+0000","last_undegraded":"2024-04-21T09:05:52.997886+0000","last_fullsized":"2024-04-21T09:05:52.997886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006204+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:05:53.006204+0000","last_peered":"2024-04-21T09:05:53.006204+0000","last_clean":"2024-04-21T09:05:53.006204+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:05:53.006204+0000","last_undegraded":"2024-04-21T09:05:53.006204+0000","last_fullsized":"2024-04-21T09:05:53.006204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006837+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:05:53.006837+0000","last_peered":"2024-04-21T09:05:53.006837+0000","last_clean":"2024-04-21T09:05:53.006837+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:05:53.006837+0000","last_undegraded":"2024-04-21T09:05:53.006837+0000","last_fullsized":"2024-04-21T09:05:53.006837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006837+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:05:53.006837+0000","last_peered":"2024-04-21T09:05:53.006837+0000","last_clean":"2024-04-21T09:05:53.006837+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:05:53.006837+0000","last_undegraded":"2024-04-21T09:05:53.006837+0000","last_fullsized":"2024-04-21T09:05:53.006837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998011+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:05:52.998011+0000","last_peered":"2024-04-21T09:05:52.998011+0000","last_clean":"2024-04-21T09:05:52.998011+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:05:52.998011+0000","last_undegraded":"2024-04-21T09:05:52.998011+0000","last_fullsized":"2024-04-21T09:05:52.998011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.090177+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:06:11.090177+0000","last_peered":"2024-04-21T09:06:11.090177+0000","last_clean":"2024-04-21T09:06:11.090177+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:06:11.090177+0000","last_undegraded":"2024-04-21T09:06:11.090177+0000","last_fullsized":"2024-04-21T09:06:11.090177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006524+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:05:53.006524+0000","last_peered":"2024-04-21T09:05:53.006524+0000","last_clean":"2024-04-21T09:05:53.006524+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:05:53.006524+0000","last_undegraded":"2024-04-21T09:05:53.006524+0000","last_fullsized":"2024-04-21T09:05:53.006524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998728+0000","last_change":"2024-04-21T09:05:51.012297+0000","last_active":"2024-04-21T09:05:52.998728+0000","last_peered":"2024-04-21T09:05:52.998728+0000","last_clean":"2024-04-21T09:05:52.998728+0000","last_became_active":"2024-04-21T09:05:51.011936+0000","last_became_peered":"2024-04-21T09:05:51.011936+0000","last_unstale":"2024-04-21T09:05:52.998728+0000","last_undegraded":"2024-04-21T09:05:52.998728+0000","last_fullsized":"2024-04-21T09:05:52.998728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:11.567022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006994+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:05:53.006994+0000","last_peered":"2024-04-21T09:05:53.006994+0000","last_clean":"2024-04-21T09:05:53.006994+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:05:53.006994+0000","last_undegraded":"2024-04-21T09:05:53.006994+0000","last_fullsized":"2024-04-21T09:05:53.006994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007145+0000","last_change":"2024-04-21T09:05:46.971142+0000","last_active":"2024-04-21T09:05:53.007145+0000","last_peered":"2024-04-21T09:05:53.007145+0000","last_clean":"2024-04-21T09:05:53.007145+0000","last_became_active":"2024-04-21T09:05:46.970565+0000","last_became_peered":"2024-04-21T09:05:46.970565+0000","last_unstale":"2024-04-21T09:05:53.007145+0000","last_undegraded":"2024-04-21T09:05:53.007145+0000","last_fullsized":"2024-04-21T09:05:53.007145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:01.543316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998015+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:05:52.998015+0000","last_peered":"2024-04-21T09:05:52.998015+0000","last_clean":"2024-04-21T09:05:52.998015+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:05:52.998015+0000","last_undegraded":"2024-04-21T09:05:52.998015+0000","last_fullsized":"2024-04-21T09:05:52.998015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:02.490911+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:06:02.490911+0000","last_peered":"2024-04-21T09:06:02.490911+0000","last_clean":"2024-04-21T09:06:02.490911+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:06:02.490911+0000","last_undegraded":"2024-04-21T09:06:02.490911+0000","last_fullsized":"2024-04-21T09:06:02.490911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007883+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:05:53.007883+0000","last_peered":"2024-04-21T09:05:53.007883+0000","last_clean":"2024-04-21T09:05:53.007883+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:05:53.007883+0000","last_undegraded":"2024-04-21T09:05:53.007883+0000","last_fullsized":"2024-04-21T09:05:53.007883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007463+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:05:53.007463+0000","last_peered":"2024-04-21T09:05:53.007463+0000","last_clean":"2024-04-21T09:05:53.007463+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:05:53.007463+0000","last_undegraded":"2024-04-21T09:05:53.007463+0000","last_fullsized":"2024-04-21T09:05:53.007463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998369+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:05:52.998369+0000","last_peered":"2024-04-21T09:05:52.998369+0000","last_clean":"2024-04-21T09:05:52.998369+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:05:52.998369+0000","last_undegraded":"2024-04-21T09:05:52.998369+0000","last_fullsized":"2024-04-21T09:05:52.998369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006750+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:05:53.006750+0000","last_peered":"2024-04-21T09:05:53.006750+0000","last_clean":"2024-04-21T09:05:53.006750+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:05:53.006750+0000","last_undegraded":"2024-04-21T09:05:53.006750+0000","last_fullsized":"2024-04-21T09:05:53.006750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007163+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:05:53.007163+0000","last_peered":"2024-04-21T09:05:53.007163+0000","last_clean":"2024-04-21T09:05:53.007163+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:05:53.007163+0000","last_undegraded":"2024-04-21T09:05:53.007163+0000","last_fullsized":"2024-04-21T09:05:53.007163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.090554+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:06:11.090554+0000","last_peered":"2024-04-21T09:06:11.090554+0000","last_clean":"2024-04-21T09:06:11.090554+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:06:11.090554+0000","last_undegraded":"2024-04-21T09:06:11.090554+0000","last_fullsized":"2024-04-21T09:06:11.090554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007218+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:05:53.007218+0000","last_peered":"2024-04-21T09:05:53.007218+0000","last_clean":"2024-04-21T09:05:53.007218+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:05:53.007218+0000","last_undegraded":"2024-04-21T09:05:53.007218+0000","last_fullsized":"2024-04-21T09:05:53.007218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998302+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:05:52.998302+0000","last_peered":"2024-04-21T09:05:52.998302+0000","last_clean":"2024-04-21T09:05:52.998302+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:05:52.998302+0000","last_undegraded":"2024-04-21T09:05:52.998302+0000","last_fullsized":"2024-04-21T09:05:52.998302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998237+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:05:52.998237+0000","last_peered":"2024-04-21T09:05:52.998237+0000","last_clean":"2024-04-21T09:05:52.998237+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:05:52.998237+0000","last_undegraded":"2024-04-21T09:05:52.998237+0000","last_fullsized":"2024-04-21T09:05:52.998237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006597+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:05:53.006597+0000","last_peered":"2024-04-21T09:05:53.006597+0000","last_clean":"2024-04-21T09:05:53.006597+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:05:53.006597+0000","last_undegraded":"2024-04-21T09:05:53.006597+0000","last_fullsized":"2024-04-21T09:05:53.006597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007925+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:05:53.007925+0000","last_peered":"2024-04-21T09:05:53.007925+0000","last_clean":"2024-04-21T09:05:53.007925+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:05:53.007925+0000","last_undegraded":"2024-04-21T09:05:53.007925+0000","last_fullsized":"2024-04-21T09:05:53.007925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.102035+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:06:12.102035+0000","last_peered":"2024-04-21T09:06:12.102035+0000","last_clean":"2024-04-21T09:06:12.102035+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:06:12.102035+0000","last_undegraded":"2024-04-21T09:06:12.102035+0000","last_fullsized":"2024-04-21T09:06:12.102035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006472+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:05:53.006472+0000","last_peered":"2024-04-21T09:05:53.006472+0000","last_clean":"2024-04-21T09:05:53.006472+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:05:53.006472+0000","last_undegraded":"2024-04-21T09:05:53.006472+0000","last_fullsized":"2024-04-21T09:05:53.006472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006313+0000","last_change":"2024-04-21T09:05:51.013349+0000","last_active":"2024-04-21T09:05:53.006313+0000","last_peered":"2024-04-21T09:05:53.006313+0000","last_clean":"2024-04-21T09:05:53.006313+0000","last_became_active":"2024-04-21T09:05:51.012980+0000","last_became_peered":"2024-04-21T09:05:51.012980+0000","last_unstale":"2024-04-21T09:05:53.006313+0000","last_undegraded":"2024-04-21T09:05:53.006313+0000","last_fullsized":"2024-04-21T09:05:53.006313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:39:00.153892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007095+0000","last_change":"2024-04-21T09:05:45.963133+0000","last_active":"2024-04-21T09:05:53.007095+0000","last_peered":"2024-04-21T09:05:53.007095+0000","last_clean":"2024-04-21T09:05:53.007095+0000","last_became_active":"2024-04-21T09:05:45.962764+0000","last_became_peered":"2024-04-21T09:05:45.962764+0000","last_unstale":"2024-04-21T09:05:53.007095+0000","last_undegraded":"2024-04-21T09:05:53.007095+0000","last_fullsized":"2024-04-21T09:05:53.007095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:58.238118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006653+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:05:53.006653+0000","last_peered":"2024-04-21T09:05:53.006653+0000","last_clean":"2024-04-21T09:05:53.006653+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:05:53.006653+0000","last_undegraded":"2024-04-21T09:05:53.006653+0000","last_fullsized":"2024-04-21T09:05:53.006653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007554+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:05:53.007554+0000","last_peered":"2024-04-21T09:05:53.007554+0000","last_clean":"2024-04-21T09:05:53.007554+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:05:53.007554+0000","last_undegraded":"2024-04-21T09:05:53.007554+0000","last_fullsized":"2024-04-21T09:05:53.007554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998176+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:05:52.998176+0000","last_peered":"2024-04-21T09:05:52.998176+0000","last_clean":"2024-04-21T09:05:52.998176+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:05:52.998176+0000","last_undegraded":"2024-04-21T09:05:52.998176+0000","last_fullsized":"2024-04-21T09:05:52.998176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007086+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:05:53.007086+0000","last_peered":"2024-04-21T09:05:53.007086+0000","last_clean":"2024-04-21T09:05:53.007086+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:05:53.007086+0000","last_undegraded":"2024-04-21T09:05:53.007086+0000","last_fullsized":"2024-04-21T09:05:53.007086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007764+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:05:53.007764+0000","last_peered":"2024-04-21T09:05:53.007764+0000","last_clean":"2024-04-21T09:05:53.007764+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:05:53.007764+0000","last_undegraded":"2024-04-21T09:05:53.007764+0000","last_fullsized":"2024-04-21T09:05:53.007764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007791+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:05:53.007791+0000","last_peered":"2024-04-21T09:05:53.007791+0000","last_clean":"2024-04-21T09:05:53.007791+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:05:53.007791+0000","last_undegraded":"2024-04-21T09:05:53.007791+0000","last_fullsized":"2024-04-21T09:05:53.007791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006941+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:05:53.006941+0000","last_peered":"2024-04-21T09:05:53.006941+0000","last_clean":"2024-04-21T09:05:53.006941+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:05:53.006941+0000","last_undegraded":"2024-04-21T09:05:53.006941+0000","last_fullsized":"2024-04-21T09:05:53.006941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006093+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:05:53.006093+0000","last_peered":"2024-04-21T09:05:53.006093+0000","last_clean":"2024-04-21T09:05:53.006093+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:05:53.006093+0000","last_undegraded":"2024-04-21T09:05:53.006093+0000","last_fullsized":"2024-04-21T09:05:53.006093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007498+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:05:53.007498+0000","last_peered":"2024-04-21T09:05:53.007498+0000","last_clean":"2024-04-21T09:05:53.007498+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:05:53.007498+0000","last_undegraded":"2024-04-21T09:05:53.007498+0000","last_fullsized":"2024-04-21T09:05:53.007498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.101673+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:06:12.101673+0000","last_peered":"2024-04-21T09:06:12.101673+0000","last_clean":"2024-04-21T09:06:12.101673+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:06:12.101673+0000","last_undegraded":"2024-04-21T09:06:12.101673+0000","last_fullsized":"2024-04-21T09:06:12.101673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.008018+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:05:53.008018+0000","last_peered":"2024-04-21T09:05:53.008018+0000","last_clean":"2024-04-21T09:05:53.008018+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:05:53.008018+0000","last_undegraded":"2024-04-21T09:05:53.008018+0000","last_fullsized":"2024-04-21T09:05:53.008018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006726+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:05:53.006726+0000","last_peered":"2024-04-21T09:05:53.006726+0000","last_clean":"2024-04-21T09:05:53.006726+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:05:53.006726+0000","last_undegraded":"2024-04-21T09:05:53.006726+0000","last_fullsized":"2024-04-21T09:05:53.006726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091689+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:06:11.091689+0000","last_peered":"2024-04-21T09:06:11.091689+0000","last_clean":"2024-04-21T09:06:11.091689+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:06:11.091689+0000","last_undegraded":"2024-04-21T09:06:11.091689+0000","last_fullsized":"2024-04-21T09:06:11.091689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006299+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:05:53.006299+0000","last_peered":"2024-04-21T09:05:53.006299+0000","last_clean":"2024-04-21T09:05:53.006299+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:05:53.006299+0000","last_undegraded":"2024-04-21T09:05:53.006299+0000","last_fullsized":"2024-04-21T09:05:53.006299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006855+0000","last_change":"2024-04-21T09:05:51.415003+0000","last_active":"2024-04-21T09:05:53.006855+0000","last_peered":"2024-04-21T09:05:53.006855+0000","last_clean":"2024-04-21T09:05:53.006855+0000","last_became_active":"2024-04-21T09:05:51.414596+0000","last_became_peered":"2024-04-21T09:05:51.414596+0000","last_unstale":"2024-04-21T09:05:53.006855+0000","last_undegraded":"2024-04-21T09:05:53.006855+0000","last_fullsized":"2024-04-21T09:05:53.006855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:29.389711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006187+0000","last_change":"2024-04-21T09:05:46.970236+0000","last_active":"2024-04-21T09:05:53.006187+0000","last_peered":"2024-04-21T09:05:53.006187+0000","last_clean":"2024-04-21T09:05:53.006187+0000","last_became_active":"2024-04-21T09:05:46.969854+0000","last_became_peered":"2024-04-21T09:05:46.969854+0000","last_unstale":"2024-04-21T09:05:53.006187+0000","last_undegraded":"2024-04-21T09:05:53.006187+0000","last_fullsized":"2024-04-21T09:05:53.006187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:30.990859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.099427+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:06:12.099427+0000","last_peered":"2024-04-21T09:06:12.099427+0000","last_clean":"2024-04-21T09:06:12.099427+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:06:12.099427+0000","last_undegraded":"2024-04-21T09:06:12.099427+0000","last_fullsized":"2024-04-21T09:06:12.099427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006466+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:05:53.006466+0000","last_peered":"2024-04-21T09:05:53.006466+0000","last_clean":"2024-04-21T09:05:53.006466+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:05:53.006466+0000","last_undegraded":"2024-04-21T09:05:53.006466+0000","last_fullsized":"2024-04-21T09:05:53.006466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091457+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:06:11.091457+0000","last_peered":"2024-04-21T09:06:11.091457+0000","last_clean":"2024-04-21T09:06:11.091457+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:06:11.091457+0000","last_undegraded":"2024-04-21T09:06:11.091457+0000","last_fullsized":"2024-04-21T09:06:11.091457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007789+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:05:53.007789+0000","last_peered":"2024-04-21T09:05:53.007789+0000","last_clean":"2024-04-21T09:05:53.007789+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:05:53.007789+0000","last_undegraded":"2024-04-21T09:05:53.007789+0000","last_fullsized":"2024-04-21T09:05:53.007789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006110+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:05:53.006110+0000","last_peered":"2024-04-21T09:05:53.006110+0000","last_clean":"2024-04-21T09:05:53.006110+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:05:53.006110+0000","last_undegraded":"2024-04-21T09:05:53.006110+0000","last_fullsized":"2024-04-21T09:05:53.006110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998381+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:05:52.998381+0000","last_peered":"2024-04-21T09:05:52.998381+0000","last_clean":"2024-04-21T09:05:52.998381+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:05:52.998381+0000","last_undegraded":"2024-04-21T09:05:52.998381+0000","last_fullsized":"2024-04-21T09:05:52.998381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006412+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:05:53.006412+0000","last_peered":"2024-04-21T09:05:53.006412+0000","last_clean":"2024-04-21T09:05:53.006412+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:05:53.006412+0000","last_undegraded":"2024-04-21T09:05:53.006412+0000","last_fullsized":"2024-04-21T09:05:53.006412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998057+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:05:52.998057+0000","last_peered":"2024-04-21T09:05:52.998057+0000","last_clean":"2024-04-21T09:05:52.998057+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:05:52.998057+0000","last_undegraded":"2024-04-21T09:05:52.998057+0000","last_fullsized":"2024-04-21T09:05:52.998057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.090992+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:06:11.090992+0000","last_peered":"2024-04-21T09:06:11.090992+0000","last_clean":"2024-04-21T09:06:11.090992+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:06:11.090992+0000","last_undegraded":"2024-04-21T09:06:11.090992+0000","last_fullsized":"2024-04-21T09:06:11.090992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006438+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:05:53.006438+0000","last_peered":"2024-04-21T09:05:53.006438+0000","last_clean":"2024-04-21T09:05:53.006438+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:05:53.006438+0000","last_undegraded":"2024-04-21T09:05:53.006438+0000","last_fullsized":"2024-04-21T09:05:53.006438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998943+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:05:52.998943+0000","last_peered":"2024-04-21T09:05:52.998943+0000","last_clean":"2024-04-21T09:05:52.998943+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:05:52.998943+0000","last_undegraded":"2024-04-21T09:05:52.998943+0000","last_fullsized":"2024-04-21T09:05:52.998943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091096+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:06:11.091096+0000","last_peered":"2024-04-21T09:06:11.091096+0000","last_clean":"2024-04-21T09:06:11.091096+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:06:11.091096+0000","last_undegraded":"2024-04-21T09:06:11.091096+0000","last_fullsized":"2024-04-21T09:06:11.091096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006420+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:05:53.006420+0000","last_peered":"2024-04-21T09:05:53.006420+0000","last_clean":"2024-04-21T09:05:53.006420+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:05:53.006420+0000","last_undegraded":"2024-04-21T09:05:53.006420+0000","last_fullsized":"2024-04-21T09:05:53.006420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998385+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:05:52.998385+0000","last_peered":"2024-04-21T09:05:52.998385+0000","last_clean":"2024-04-21T09:05:52.998385+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:05:52.998385+0000","last_undegraded":"2024-04-21T09:05:52.998385+0000","last_fullsized":"2024-04-21T09:05:52.998385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006382+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:05:53.006382+0000","last_peered":"2024-04-21T09:05:53.006382+0000","last_clean":"2024-04-21T09:05:53.006382+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:05:53.006382+0000","last_undegraded":"2024-04-21T09:05:53.006382+0000","last_fullsized":"2024-04-21T09:05:53.006382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006834+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:05:53.006834+0000","last_peered":"2024-04-21T09:05:53.006834+0000","last_clean":"2024-04-21T09:05:53.006834+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:05:53.006834+0000","last_undegraded":"2024-04-21T09:05:53.006834+0000","last_fullsized":"2024-04-21T09:05:53.006834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998659+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:05:52.998659+0000","last_peered":"2024-04-21T09:05:52.998659+0000","last_clean":"2024-04-21T09:05:52.998659+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:05:52.998659+0000","last_undegraded":"2024-04-21T09:05:52.998659+0000","last_fullsized":"2024-04-21T09:05:52.998659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998139+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:05:52.998139+0000","last_peered":"2024-04-21T09:05:52.998139+0000","last_clean":"2024-04-21T09:05:52.998139+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:05:52.998139+0000","last_undegraded":"2024-04-21T09:05:52.998139+0000","last_fullsized":"2024-04-21T09:05:52.998139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007207+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:05:53.007207+0000","last_peered":"2024-04-21T09:05:53.007207+0000","last_clean":"2024-04-21T09:05:53.007207+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:05:53.007207+0000","last_undegraded":"2024-04-21T09:05:53.007207+0000","last_fullsized":"2024-04-21T09:05:53.007207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998600+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:05:52.998600+0000","last_peered":"2024-04-21T09:05:52.998600+0000","last_clean":"2024-04-21T09:05:52.998600+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:05:52.998600+0000","last_undegraded":"2024-04-21T09:05:52.998600+0000","last_fullsized":"2024-04-21T09:05:52.998600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091888+0000","last_change":"2024-04-21T09:05:45.967431+0000","last_active":"2024-04-21T09:06:11.091888+0000","last_peered":"2024-04-21T09:06:11.091888+0000","last_clean":"2024-04-21T09:06:11.091888+0000","last_became_active":"2024-04-21T09:05:45.967106+0000","last_became_peered":"2024-04-21T09:05:45.967106+0000","last_unstale":"2024-04-21T09:06:11.091888+0000","last_undegraded":"2024-04-21T09:06:11.091888+0000","last_fullsized":"2024-04-21T09:06:11.091888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:00.877072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007853+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:05:53.007853+0000","last_peered":"2024-04-21T09:05:53.007853+0000","last_clean":"2024-04-21T09:05:53.007853+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:05:53.007853+0000","last_undegraded":"2024-04-21T09:05:53.007853+0000","last_fullsized":"2024-04-21T09:05:53.007853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006388+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:05:53.006388+0000","last_peered":"2024-04-21T09:05:53.006388+0000","last_clean":"2024-04-21T09:05:53.006388+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:05:53.006388+0000","last_undegraded":"2024-04-21T09:05:53.006388+0000","last_fullsized":"2024-04-21T09:05:53.006388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998571+0000","last_change":"2024-04-21T09:05:45.976029+0000","last_active":"2024-04-21T09:05:52.998571+0000","last_peered":"2024-04-21T09:05:52.998571+0000","last_clean":"2024-04-21T09:05:52.998571+0000","last_became_active":"2024-04-21T09:05:45.975647+0000","last_became_peered":"2024-04-21T09:05:45.975647+0000","last_unstale":"2024-04-21T09:05:52.998571+0000","last_undegraded":"2024-04-21T09:05:52.998571+0000","last_fullsized":"2024-04-21T09:05:52.998571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:50:46.781522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.008080+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:05:53.008080+0000","last_peered":"2024-04-21T09:05:53.008080+0000","last_clean":"2024-04-21T09:05:53.008080+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:05:53.008080+0000","last_undegraded":"2024-04-21T09:05:53.008080+0000","last_fullsized":"2024-04-21T09:05:53.008080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006897+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:05:53.006897+0000","last_peered":"2024-04-21T09:05:53.006897+0000","last_clean":"2024-04-21T09:05:53.006897+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:05:53.006897+0000","last_undegraded":"2024-04-21T09:05:53.006897+0000","last_fullsized":"2024-04-21T09:05:53.006897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007146+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:05:53.007146+0000","last_peered":"2024-04-21T09:05:53.007146+0000","last_clean":"2024-04-21T09:05:53.007146+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:05:53.007146+0000","last_undegraded":"2024-04-21T09:05:53.007146+0000","last_fullsized":"2024-04-21T09:05:53.007146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007677+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:05:53.007677+0000","last_peered":"2024-04-21T09:05:53.007677+0000","last_clean":"2024-04-21T09:05:53.007677+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:05:53.007677+0000","last_undegraded":"2024-04-21T09:05:53.007677+0000","last_fullsized":"2024-04-21T09:05:53.007677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007082+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:05:53.007082+0000","last_peered":"2024-04-21T09:05:53.007082+0000","last_clean":"2024-04-21T09:05:53.007082+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:05:53.007082+0000","last_undegraded":"2024-04-21T09:05:53.007082+0000","last_fullsized":"2024-04-21T09:05:53.007082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007793+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:05:53.007793+0000","last_peered":"2024-04-21T09:05:53.007793+0000","last_clean":"2024-04-21T09:05:53.007793+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:05:53.007793+0000","last_undegraded":"2024-04-21T09:05:53.007793+0000","last_fullsized":"2024-04-21T09:05:53.007793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007449+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:05:53.007449+0000","last_peered":"2024-04-21T09:05:53.007449+0000","last_clean":"2024-04-21T09:05:53.007449+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:05:53.007449+0000","last_undegraded":"2024-04-21T09:05:53.007449+0000","last_fullsized":"2024-04-21T09:05:53.007449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006946+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:05:53.006946+0000","last_peered":"2024-04-21T09:05:53.006946+0000","last_clean":"2024-04-21T09:05:53.006946+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:05:53.006946+0000","last_undegraded":"2024-04-21T09:05:53.006946+0000","last_fullsized":"2024-04-21T09:05:53.006946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998140+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:05:52.998140+0000","last_peered":"2024-04-21T09:05:52.998140+0000","last_clean":"2024-04-21T09:05:52.998140+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:05:52.998140+0000","last_undegraded":"2024-04-21T09:05:52.998140+0000","last_fullsized":"2024-04-21T09:05:52.998140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998051+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:05:52.998051+0000","last_peered":"2024-04-21T09:05:52.998051+0000","last_clean":"2024-04-21T09:05:52.998051+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:05:52.998051+0000","last_undegraded":"2024-04-21T09:05:52.998051+0000","last_fullsized":"2024-04-21T09:05:52.998051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007211+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:05:53.007211+0000","last_peered":"2024-04-21T09:05:53.007211+0000","last_clean":"2024-04-21T09:05:53.007211+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:05:53.007211+0000","last_undegraded":"2024-04-21T09:05:53.007211+0000","last_fullsized":"2024-04-21T09:05:53.007211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007874+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:05:53.007874+0000","last_peered":"2024-04-21T09:05:53.007874+0000","last_clean":"2024-04-21T09:05:53.007874+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:05:53.007874+0000","last_undegraded":"2024-04-21T09:05:53.007874+0000","last_fullsized":"2024-04-21T09:05:53.007874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007181+0000","last_change":"2024-04-21T09:05:46.972797+0000","last_active":"2024-04-21T09:05:53.007181+0000","last_peered":"2024-04-21T09:05:53.007181+0000","last_clean":"2024-04-21T09:05:53.007181+0000","last_became_active":"2024-04-21T09:05:46.972465+0000","last_became_peered":"2024-04-21T09:05:46.972465+0000","last_unstale":"2024-04-21T09:05:53.007181+0000","last_undegraded":"2024-04-21T09:05:53.007181+0000","last_fullsized":"2024-04-21T09:05:53.007181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:29:45.732617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998739+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:05:52.998739+0000","last_peered":"2024-04-21T09:05:52.998739+0000","last_clean":"2024-04-21T09:05:52.998739+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:05:52.998739+0000","last_undegraded":"2024-04-21T09:05:52.998739+0000","last_fullsized":"2024-04-21T09:05:52.998739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006593+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:05:53.006593+0000","last_peered":"2024-04-21T09:05:53.006593+0000","last_clean":"2024-04-21T09:05:53.006593+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:05:53.006593+0000","last_undegraded":"2024-04-21T09:05:53.006593+0000","last_fullsized":"2024-04-21T09:05:53.006593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007457+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:05:53.007457+0000","last_peered":"2024-04-21T09:05:53.007457+0000","last_clean":"2024-04-21T09:05:53.007457+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:05:53.007457+0000","last_undegraded":"2024-04-21T09:05:53.007457+0000","last_fullsized":"2024-04-21T09:05:53.007457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006485+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:05:53.006485+0000","last_peered":"2024-04-21T09:05:53.006485+0000","last_clean":"2024-04-21T09:05:53.006485+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:05:53.006485+0000","last_undegraded":"2024-04-21T09:05:53.006485+0000","last_fullsized":"2024-04-21T09:05:53.006485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006810+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:05:53.006810+0000","last_peered":"2024-04-21T09:05:53.006810+0000","last_clean":"2024-04-21T09:05:53.006810+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:05:53.006810+0000","last_undegraded":"2024-04-21T09:05:53.006810+0000","last_fullsized":"2024-04-21T09:05:53.006810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007532+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:05:53.007532+0000","last_peered":"2024-04-21T09:05:53.007532+0000","last_clean":"2024-04-21T09:05:53.007532+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:05:53.007532+0000","last_undegraded":"2024-04-21T09:05:53.007532+0000","last_fullsized":"2024-04-21T09:05:53.007532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007374+0000","last_change":"2024-04-21T09:05:46.988031+0000","last_active":"2024-04-21T09:05:53.007374+0000","last_peered":"2024-04-21T09:05:53.007374+0000","last_clean":"2024-04-21T09:05:53.007374+0000","last_became_active":"2024-04-21T09:05:46.987629+0000","last_became_peered":"2024-04-21T09:05:46.987629+0000","last_unstale":"2024-04-21T09:05:53.007374+0000","last_undegraded":"2024-04-21T09:05:53.007374+0000","last_fullsized":"2024-04-21T09:05:53.007374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:33.326914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210748,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6068,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":91,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":17834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":490255,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612630,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1084077,"data_compressed":53245,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":490255,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":490411,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1081833,"data_compressed":53245,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":489673,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":493515,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":490367,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493549,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489777,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":489695,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1081517,"data_compressed":53245,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:06:15.918 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:06:16.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 -- 172.21.15.66:0/2564110758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 msgr2=0x7fedd4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:16.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 --2- 172.21.15.66:0/2564110758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd4101290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009900 tx=0x7fedc802bc30 comp rx=0 tx=0).stop 2024-04-21T09:06:16.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 -- 172.21.15.66:0/2564110758 shutdown_connections 2024-04-21T09:06:16.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 --2- 172.21.15.66:0/2564110758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd4102a10 0x7fedd4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 --2- 172.21.15.66:0/2564110758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd4102050 0x7fedd41024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 --2- 172.21.15.66:0/2564110758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd4101290 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 -- 172.21.15.66:0/2564110758 >> 172.21.15.66:0/2564110758 conn(0x7fedd40fc470 msgr2=0x7fedd40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 -- 172.21.15.66:0/2564110758 shutdown_connections 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.053+0000 7feddacfd640 1 -- 172.21.15.66:0/2564110758 wait complete. 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 Processor -- start 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 -- start start 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd4102050 0x7fedd4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd4102a10 0x7fedd419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedd410ca00 con 0x7fedd4100e70 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedd410c880 con 0x7fedd4102a10 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7feddacfd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fedd410cb80 con 0x7fedd4102050 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7fedd8a99640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7fedd8a99640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd41981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50378/0 (socket says 172.21.15.66:50378) 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7fedd8a99640 1 -- 172.21.15.66:0/232488816 learned_addr learned my addr 172.21.15.66:0/232488816 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:16.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7fedd929a640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd4102a10 0x7fedd419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.054+0000 7fedd3fff640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd4102050 0x7fedd4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd8a99640 1 -- 172.21.15.66:0/232488816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd4102050 msgr2=0x7fedd4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd8a99640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd4102050 0x7fedd4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd8a99640 1 -- 172.21.15.66:0/232488816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd4102a10 msgr2=0x7fedd419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd8a99640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd4102a10 0x7fedd419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd8a99640 1 -- 172.21.15.66:0/232488816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd419fdf0 con 0x7fedd4100e70 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd8a99640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd41981d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fedc8000c00 tx=0x7fedc8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd1ffb640 1 -- 172.21.15.66:0/232488816 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8043020 con 0x7fedd4100e70 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd1ffb640 1 -- 172.21.15.66:0/232488816 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedc8004440 con 0x7fedd4100e70 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd41a0080 con 0x7fedd4100e70 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7fedd1ffb640 1 -- 172.21.15.66:0/232488816 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8003c90 con 0x7fedd4100e70 2024-04-21T09:06:16.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.055+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41a05d0 con 0x7fedd4100e70 2024-04-21T09:06:16.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.056+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd41a07d0 con 0x7fedd4100e70 2024-04-21T09:06:16.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.057+0000 7fedd1ffb640 1 -- 172.21.15.66:0/232488816 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fedc8046020 con 0x7fedd4100e70 2024-04-21T09:06:16.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.058+0000 7fedd1ffb640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedc40a3c40 0x7fedc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:16.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.058+0000 7fedd3fff640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedc40a3c40 0x7fedc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:16.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.058+0000 7fedd1ffb640 1 -- 172.21.15.66:0/232488816 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fedc80f28e0 con 0x7fedd4100e70 2024-04-21T09:06:16.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.059+0000 7fedd3fff640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedc40a3c40 0x7fedc40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005fd0 tx=0x7fedc0005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:16.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.063+0000 7fedd1ffb640 1 -- 172.21.15.66:0/232488816 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedc804d020 con 0x7fedd4100e70 2024-04-21T09:06:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.246+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedd40fe620 con 0x7fedc40a3c40 2024-04-21T09:06:16.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.256+0000 7fedd1ffb640 1 -- 172.21.15.66:0/232488816 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7fedd40fe620 con 0x7fedc40a3c40 2024-04-21T09:06:16.260 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:16.262 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:06:16.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedc40a3c40 msgr2=0x7fedc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:16.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedc40a3c40 0x7fedc40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005fd0 tx=0x7fedc0005e20 comp rx=0 tx=0).stop 2024-04-21T09:06:16.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 msgr2=0x7fedd41981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:16.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd41981d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fedc8000c00 tx=0x7fedc8004040 comp rx=0 tx=0).stop 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 shutdown_connections 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedc40a3c40 0x7fedc40a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd4102a10 0x7fedd419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd4102050 0x7fedd4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 --2- 172.21.15.66:0/232488816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4100e70 0x7fedd41981d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 >> 172.21.15.66:0/232488816 conn(0x7fedd40fc470 msgr2=0x7fedd40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 shutdown_connections 2024-04-21T09:06:16.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:16.259+0000 7feddacfd640 1 -- 172.21.15.66:0/232488816 wait complete. 2024-04-21T09:06:16.270 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:16 smithi066 ceph-mon[27953]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.270 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:16 smithi066 ceph-mon[27953]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-21T09:06:16.270 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:16 smithi066 ceph-mon[27953]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.270 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:16 smithi066 ceph-mon[27953]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.270 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:16 smithi066 ceph-mon[27953]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T09:06:16.270 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2472018933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:06:16.270 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/617112219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:16.271 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-04-21T09:06:14.301800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596436,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":141,"num_read_kb":124,"num_write":272,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356824,"kb_used_data":11812,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585000,"statfs":{"total":1151940427776,"available":1151575040000,"internally_reserved":0,"allocated":12095488,"data_stored":7664924,"data_compressed":317901,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1569,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":44,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003662"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007602+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:05:53.007602+0000","last_peered":"2024-04-21T09:05:53.007602+0000","last_clean":"2024-04-21T09:05:53.007602+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:05:53.007602+0000","last_undegraded":"2024-04-21T09:05:53.007602+0000","last_fullsized":"2024-04-21T09:05:53.007602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006546+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:05:53.006546+0000","last_peered":"2024-04-21T09:05:53.006546+0000","last_clean":"2024-04-21T09:05:53.006546+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:05:53.006546+0000","last_undegraded":"2024-04-21T09:05:53.006546+0000","last_fullsized":"2024-04-21T09:05:53.006546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007483+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:05:53.007483+0000","last_peered":"2024-04-21T09:05:53.007483+0000","last_clean":"2024-04-21T09:05:53.007483+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:05:53.007483+0000","last_undegraded":"2024-04-21T09:05:53.007483+0000","last_fullsized":"2024-04-21T09:05:53.007483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998866+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:05:52.998866+0000","last_peered":"2024-04-21T09:05:52.998866+0000","last_clean":"2024-04-21T09:05:52.998866+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:05:52.998866+0000","last_undegraded":"2024-04-21T09:05:52.998866+0000","last_fullsized":"2024-04-21T09:05:52.998866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006461+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:05:53.006461+0000","last_peered":"2024-04-21T09:05:53.006461+0000","last_clean":"2024-04-21T09:05:53.006461+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:05:53.006461+0000","last_undegraded":"2024-04-21T09:05:53.006461+0000","last_fullsized":"2024-04-21T09:05:53.006461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007439+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:05:53.007439+0000","last_peered":"2024-04-21T09:05:53.007439+0000","last_clean":"2024-04-21T09:05:53.007439+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:05:53.007439+0000","last_undegraded":"2024-04-21T09:05:53.007439+0000","last_fullsized":"2024-04-21T09:05:53.007439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006512+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:05:53.006512+0000","last_peered":"2024-04-21T09:05:53.006512+0000","last_clean":"2024-04-21T09:05:53.006512+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:05:53.006512+0000","last_undegraded":"2024-04-21T09:05:53.006512+0000","last_fullsized":"2024-04-21T09:05:53.006512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.005940+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:05:53.005940+0000","last_peered":"2024-04-21T09:05:53.005940+0000","last_clean":"2024-04-21T09:05:53.005940+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:05:53.005940+0000","last_undegraded":"2024-04-21T09:05:53.005940+0000","last_fullsized":"2024-04-21T09:05:53.005940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007060+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:05:53.007060+0000","last_peered":"2024-04-21T09:05:53.007060+0000","last_clean":"2024-04-21T09:05:53.007060+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:05:53.007060+0000","last_undegraded":"2024-04-21T09:05:53.007060+0000","last_fullsized":"2024-04-21T09:05:53.007060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007411+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:05:53.007411+0000","last_peered":"2024-04-21T09:05:53.007411+0000","last_clean":"2024-04-21T09:05:53.007411+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:05:53.007411+0000","last_undegraded":"2024-04-21T09:05:53.007411+0000","last_fullsized":"2024-04-21T09:05:53.007411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.008007+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:05:53.008007+0000","last_peered":"2024-04-21T09:05:53.008007+0000","last_clean":"2024-04-21T09:05:53.008007+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:05:53.008007+0000","last_undegraded":"2024-04-21T09:05:53.008007+0000","last_fullsized":"2024-04-21T09:05:53.008007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998723+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:05:52.998723+0000","last_peered":"2024-04-21T09:05:52.998723+0000","last_clean":"2024-04-21T09:05:52.998723+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:05:52.998723+0000","last_undegraded":"2024-04-21T09:05:52.998723+0000","last_fullsized":"2024-04-21T09:05:52.998723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007431+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:05:53.007431+0000","last_peered":"2024-04-21T09:05:53.007431+0000","last_clean":"2024-04-21T09:05:53.007431+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:05:53.007431+0000","last_undegraded":"2024-04-21T09:05:53.007431+0000","last_fullsized":"2024-04-21T09:05:53.007431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006834+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:05:53.006834+0000","last_peered":"2024-04-21T09:05:53.006834+0000","last_clean":"2024-04-21T09:05:53.006834+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:05:53.006834+0000","last_undegraded":"2024-04-21T09:05:53.006834+0000","last_fullsized":"2024-04-21T09:05:53.006834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007249+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:05:53.007249+0000","last_peered":"2024-04-21T09:05:53.007249+0000","last_clean":"2024-04-21T09:05:53.007249+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:05:53.007249+0000","last_undegraded":"2024-04-21T09:05:53.007249+0000","last_fullsized":"2024-04-21T09:05:53.007249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.005990+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:05:53.005990+0000","last_peered":"2024-04-21T09:05:53.005990+0000","last_clean":"2024-04-21T09:05:53.005990+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:05:53.005990+0000","last_undegraded":"2024-04-21T09:05:53.005990+0000","last_fullsized":"2024-04-21T09:05:53.005990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007514+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:05:53.007514+0000","last_peered":"2024-04-21T09:05:53.007514+0000","last_clean":"2024-04-21T09:05:53.007514+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:05:53.007514+0000","last_undegraded":"2024-04-21T09:05:53.007514+0000","last_fullsized":"2024-04-21T09:05:53.007514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998384+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:05:52.998384+0000","last_peered":"2024-04-21T09:05:52.998384+0000","last_clean":"2024-04-21T09:05:52.998384+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:05:52.998384+0000","last_undegraded":"2024-04-21T09:05:52.998384+0000","last_fullsized":"2024-04-21T09:05:52.998384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998768+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:05:52.998768+0000","last_peered":"2024-04-21T09:05:52.998768+0000","last_clean":"2024-04-21T09:05:52.998768+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:05:52.998768+0000","last_undegraded":"2024-04-21T09:05:52.998768+0000","last_fullsized":"2024-04-21T09:05:52.998768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998379+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:05:52.998379+0000","last_peered":"2024-04-21T09:05:52.998379+0000","last_clean":"2024-04-21T09:05:52.998379+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:05:52.998379+0000","last_undegraded":"2024-04-21T09:05:52.998379+0000","last_fullsized":"2024-04-21T09:05:52.998379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998364+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:05:52.998364+0000","last_peered":"2024-04-21T09:05:52.998364+0000","last_clean":"2024-04-21T09:05:52.998364+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:05:52.998364+0000","last_undegraded":"2024-04-21T09:05:52.998364+0000","last_fullsized":"2024-04-21T09:05:52.998364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007732+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:05:53.007732+0000","last_peered":"2024-04-21T09:05:53.007732+0000","last_clean":"2024-04-21T09:05:53.007732+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:05:53.007732+0000","last_undegraded":"2024-04-21T09:05:53.007732+0000","last_fullsized":"2024-04-21T09:05:53.007732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998354+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:05:52.998354+0000","last_peered":"2024-04-21T09:05:52.998354+0000","last_clean":"2024-04-21T09:05:52.998354+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:05:52.998354+0000","last_undegraded":"2024-04-21T09:05:52.998354+0000","last_fullsized":"2024-04-21T09:05:52.998354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007958+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:05:53.007958+0000","last_peered":"2024-04-21T09:05:53.007958+0000","last_clean":"2024-04-21T09:05:53.007958+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:05:53.007958+0000","last_undegraded":"2024-04-21T09:05:53.007958+0000","last_fullsized":"2024-04-21T09:05:53.007958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.005980+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:05:53.005980+0000","last_peered":"2024-04-21T09:05:53.005980+0000","last_clean":"2024-04-21T09:05:53.005980+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:05:53.005980+0000","last_undegraded":"2024-04-21T09:05:53.005980+0000","last_fullsized":"2024-04-21T09:05:53.005980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998507+0000","last_change":"2024-04-21T09:05:51.013392+0000","last_active":"2024-04-21T09:05:52.998507+0000","last_peered":"2024-04-21T09:05:52.998507+0000","last_clean":"2024-04-21T09:05:52.998507+0000","last_became_active":"2024-04-21T09:05:51.012918+0000","last_became_peered":"2024-04-21T09:05:51.012918+0000","last_unstale":"2024-04-21T09:05:52.998507+0000","last_undegraded":"2024-04-21T09:05:52.998507+0000","last_fullsized":"2024-04-21T09:05:52.998507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:31.142591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.100086+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:06:12.100086+0000","last_peered":"2024-04-21T09:06:12.100086+0000","last_clean":"2024-04-21T09:06:12.100086+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:06:12.100086+0000","last_undegraded":"2024-04-21T09:06:12.100086+0000","last_fullsized":"2024-04-21T09:06:12.100086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007255+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:05:53.007255+0000","last_peered":"2024-04-21T09:05:53.007255+0000","last_clean":"2024-04-21T09:05:53.007255+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:05:53.007255+0000","last_undegraded":"2024-04-21T09:05:53.007255+0000","last_fullsized":"2024-04-21T09:05:53.007255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998464+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:05:52.998464+0000","last_peered":"2024-04-21T09:05:52.998464+0000","last_clean":"2024-04-21T09:05:52.998464+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:05:52.998464+0000","last_undegraded":"2024-04-21T09:05:52.998464+0000","last_fullsized":"2024-04-21T09:05:52.998464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007335+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:05:53.007335+0000","last_peered":"2024-04-21T09:05:53.007335+0000","last_clean":"2024-04-21T09:05:53.007335+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:05:53.007335+0000","last_undegraded":"2024-04-21T09:05:53.007335+0000","last_fullsized":"2024-04-21T09:05:53.007335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007048+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:05:53.007048+0000","last_peered":"2024-04-21T09:05:53.007048+0000","last_clean":"2024-04-21T09:05:53.007048+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:05:53.007048+0000","last_undegraded":"2024-04-21T09:05:53.007048+0000","last_fullsized":"2024-04-21T09:05:53.007048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998560+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:05:52.998560+0000","last_peered":"2024-04-21T09:05:52.998560+0000","last_clean":"2024-04-21T09:05:52.998560+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:05:52.998560+0000","last_undegraded":"2024-04-21T09:05:52.998560+0000","last_fullsized":"2024-04-21T09:05:52.998560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091692+0000","last_change":"2024-04-21T09:05:45.965239+0000","last_active":"2024-04-21T09:06:11.091692+0000","last_peered":"2024-04-21T09:06:11.091692+0000","last_clean":"2024-04-21T09:06:11.091692+0000","last_became_active":"2024-04-21T09:05:45.964879+0000","last_became_peered":"2024-04-21T09:05:45.964879+0000","last_unstale":"2024-04-21T09:06:11.091692+0000","last_undegraded":"2024-04-21T09:06:11.091692+0000","last_fullsized":"2024-04-21T09:06:11.091692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:26.985732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007611+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:05:53.007611+0000","last_peered":"2024-04-21T09:05:53.007611+0000","last_clean":"2024-04-21T09:05:53.007611+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:05:53.007611+0000","last_undegraded":"2024-04-21T09:05:53.007611+0000","last_fullsized":"2024-04-21T09:05:53.007611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998430+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:05:52.998430+0000","last_peered":"2024-04-21T09:05:52.998430+0000","last_clean":"2024-04-21T09:05:52.998430+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:05:52.998430+0000","last_undegraded":"2024-04-21T09:05:52.998430+0000","last_fullsized":"2024-04-21T09:05:52.998430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007340+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:05:53.007340+0000","last_peered":"2024-04-21T09:05:53.007340+0000","last_clean":"2024-04-21T09:05:53.007340+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:05:53.007340+0000","last_undegraded":"2024-04-21T09:05:53.007340+0000","last_fullsized":"2024-04-21T09:05:53.007340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006847+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:05:53.006847+0000","last_peered":"2024-04-21T09:05:53.006847+0000","last_clean":"2024-04-21T09:05:53.006847+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:05:53.006847+0000","last_undegraded":"2024-04-21T09:05:53.006847+0000","last_fullsized":"2024-04-21T09:05:53.006847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007375+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:05:53.007375+0000","last_peered":"2024-04-21T09:05:53.007375+0000","last_clean":"2024-04-21T09:05:53.007375+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:05:53.007375+0000","last_undegraded":"2024-04-21T09:05:53.007375+0000","last_fullsized":"2024-04-21T09:05:53.007375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.099512+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:06:12.099512+0000","last_peered":"2024-04-21T09:06:12.099512+0000","last_clean":"2024-04-21T09:06:12.099512+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:06:12.099512+0000","last_undegraded":"2024-04-21T09:06:12.099512+0000","last_fullsized":"2024-04-21T09:06:12.099512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006231+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:05:53.006231+0000","last_peered":"2024-04-21T09:05:53.006231+0000","last_clean":"2024-04-21T09:05:53.006231+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:05:53.006231+0000","last_undegraded":"2024-04-21T09:05:53.006231+0000","last_fullsized":"2024-04-21T09:05:53.006231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006317+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:05:53.006317+0000","last_peered":"2024-04-21T09:05:53.006317+0000","last_clean":"2024-04-21T09:05:53.006317+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:05:53.006317+0000","last_undegraded":"2024-04-21T09:05:53.006317+0000","last_fullsized":"2024-04-21T09:05:53.006317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007398+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:05:53.007398+0000","last_peered":"2024-04-21T09:05:53.007398+0000","last_clean":"2024-04-21T09:05:53.007398+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:05:53.007398+0000","last_undegraded":"2024-04-21T09:05:53.007398+0000","last_fullsized":"2024-04-21T09:05:53.007398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.997886+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:05:52.997886+0000","last_peered":"2024-04-21T09:05:52.997886+0000","last_clean":"2024-04-21T09:05:52.997886+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:05:52.997886+0000","last_undegraded":"2024-04-21T09:05:52.997886+0000","last_fullsized":"2024-04-21T09:05:52.997886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006204+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:05:53.006204+0000","last_peered":"2024-04-21T09:05:53.006204+0000","last_clean":"2024-04-21T09:05:53.006204+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:05:53.006204+0000","last_undegraded":"2024-04-21T09:05:53.006204+0000","last_fullsized":"2024-04-21T09:05:53.006204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006837+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:05:53.006837+0000","last_peered":"2024-04-21T09:05:53.006837+0000","last_clean":"2024-04-21T09:05:53.006837+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:05:53.006837+0000","last_undegraded":"2024-04-21T09:05:53.006837+0000","last_fullsized":"2024-04-21T09:05:53.006837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006837+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:05:53.006837+0000","last_peered":"2024-04-21T09:05:53.006837+0000","last_clean":"2024-04-21T09:05:53.006837+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:05:53.006837+0000","last_undegraded":"2024-04-21T09:05:53.006837+0000","last_fullsized":"2024-04-21T09:05:53.006837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998011+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:05:52.998011+0000","last_peered":"2024-04-21T09:05:52.998011+0000","last_clean":"2024-04-21T09:05:52.998011+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:05:52.998011+0000","last_undegraded":"2024-04-21T09:05:52.998011+0000","last_fullsized":"2024-04-21T09:05:52.998011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.090177+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:06:11.090177+0000","last_peered":"2024-04-21T09:06:11.090177+0000","last_clean":"2024-04-21T09:06:11.090177+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:06:11.090177+0000","last_undegraded":"2024-04-21T09:06:11.090177+0000","last_fullsized":"2024-04-21T09:06:11.090177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006524+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:05:53.006524+0000","last_peered":"2024-04-21T09:05:53.006524+0000","last_clean":"2024-04-21T09:05:53.006524+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:05:53.006524+0000","last_undegraded":"2024-04-21T09:05:53.006524+0000","last_fullsized":"2024-04-21T09:05:53.006524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998728+0000","last_change":"2024-04-21T09:05:51.012297+0000","last_active":"2024-04-21T09:05:52.998728+0000","last_peered":"2024-04-21T09:05:52.998728+0000","last_clean":"2024-04-21T09:05:52.998728+0000","last_became_active":"2024-04-21T09:05:51.011936+0000","last_became_peered":"2024-04-21T09:05:51.011936+0000","last_unstale":"2024-04-21T09:05:52.998728+0000","last_undegraded":"2024-04-21T09:05:52.998728+0000","last_fullsized":"2024-04-21T09:05:52.998728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:11.567022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006994+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:05:53.006994+0000","last_peered":"2024-04-21T09:05:53.006994+0000","last_clean":"2024-04-21T09:05:53.006994+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:05:53.006994+0000","last_undegraded":"2024-04-21T09:05:53.006994+0000","last_fullsized":"2024-04-21T09:05:53.006994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007145+0000","last_change":"2024-04-21T09:05:46.971142+0000","last_active":"2024-04-21T09:05:53.007145+0000","last_peered":"2024-04-21T09:05:53.007145+0000","last_clean":"2024-04-21T09:05:53.007145+0000","last_became_active":"2024-04-21T09:05:46.970565+0000","last_became_peered":"2024-04-21T09:05:46.970565+0000","last_unstale":"2024-04-21T09:05:53.007145+0000","last_undegraded":"2024-04-21T09:05:53.007145+0000","last_fullsized":"2024-04-21T09:05:53.007145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:01.543316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998015+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:05:52.998015+0000","last_peered":"2024-04-21T09:05:52.998015+0000","last_clean":"2024-04-21T09:05:52.998015+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:05:52.998015+0000","last_undegraded":"2024-04-21T09:05:52.998015+0000","last_fullsized":"2024-04-21T09:05:52.998015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:02.490911+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:06:02.490911+0000","last_peered":"2024-04-21T09:06:02.490911+0000","last_clean":"2024-04-21T09:06:02.490911+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:06:02.490911+0000","last_undegraded":"2024-04-21T09:06:02.490911+0000","last_fullsized":"2024-04-21T09:06:02.490911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007883+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:05:53.007883+0000","last_peered":"2024-04-21T09:05:53.007883+0000","last_clean":"2024-04-21T09:05:53.007883+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:05:53.007883+0000","last_undegraded":"2024-04-21T09:05:53.007883+0000","last_fullsized":"2024-04-21T09:05:53.007883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007463+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:05:53.007463+0000","last_peered":"2024-04-21T09:05:53.007463+0000","last_clean":"2024-04-21T09:05:53.007463+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:05:53.007463+0000","last_undegraded":"2024-04-21T09:05:53.007463+0000","last_fullsized":"2024-04-21T09:05:53.007463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998369+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:05:52.998369+0000","last_peered":"2024-04-21T09:05:52.998369+0000","last_clean":"2024-04-21T09:05:52.998369+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:05:52.998369+0000","last_undegraded":"2024-04-21T09:05:52.998369+0000","last_fullsized":"2024-04-21T09:05:52.998369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006750+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:05:53.006750+0000","last_peered":"2024-04-21T09:05:53.006750+0000","last_clean":"2024-04-21T09:05:53.006750+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:05:53.006750+0000","last_undegraded":"2024-04-21T09:05:53.006750+0000","last_fullsized":"2024-04-21T09:05:53.006750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007163+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:05:53.007163+0000","last_peered":"2024-04-21T09:05:53.007163+0000","last_clean":"2024-04-21T09:05:53.007163+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:05:53.007163+0000","last_undegraded":"2024-04-21T09:05:53.007163+0000","last_fullsized":"2024-04-21T09:05:53.007163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.090554+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:06:11.090554+0000","last_peered":"2024-04-21T09:06:11.090554+0000","last_clean":"2024-04-21T09:06:11.090554+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:06:11.090554+0000","last_undegraded":"2024-04-21T09:06:11.090554+0000","last_fullsized":"2024-04-21T09:06:11.090554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007218+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:05:53.007218+0000","last_peered":"2024-04-21T09:05:53.007218+0000","last_clean":"2024-04-21T09:05:53.007218+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:05:53.007218+0000","last_undegraded":"2024-04-21T09:05:53.007218+0000","last_fullsized":"2024-04-21T09:05:53.007218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998302+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:05:52.998302+0000","last_peered":"2024-04-21T09:05:52.998302+0000","last_clean":"2024-04-21T09:05:52.998302+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:05:52.998302+0000","last_undegraded":"2024-04-21T09:05:52.998302+0000","last_fullsized":"2024-04-21T09:05:52.998302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998237+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:05:52.998237+0000","last_peered":"2024-04-21T09:05:52.998237+0000","last_clean":"2024-04-21T09:05:52.998237+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:05:52.998237+0000","last_undegraded":"2024-04-21T09:05:52.998237+0000","last_fullsized":"2024-04-21T09:05:52.998237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006597+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:05:53.006597+0000","last_peered":"2024-04-21T09:05:53.006597+0000","last_clean":"2024-04-21T09:05:53.006597+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:05:53.006597+0000","last_undegraded":"2024-04-21T09:05:53.006597+0000","last_fullsized":"2024-04-21T09:05:53.006597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007925+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:05:53.007925+0000","last_peered":"2024-04-21T09:05:53.007925+0000","last_clean":"2024-04-21T09:05:53.007925+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:05:53.007925+0000","last_undegraded":"2024-04-21T09:05:53.007925+0000","last_fullsized":"2024-04-21T09:05:53.007925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.102035+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:06:12.102035+0000","last_peered":"2024-04-21T09:06:12.102035+0000","last_clean":"2024-04-21T09:06:12.102035+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:06:12.102035+0000","last_undegraded":"2024-04-21T09:06:12.102035+0000","last_fullsized":"2024-04-21T09:06:12.102035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006472+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:05:53.006472+0000","last_peered":"2024-04-21T09:05:53.006472+0000","last_clean":"2024-04-21T09:05:53.006472+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:05:53.006472+0000","last_undegraded":"2024-04-21T09:05:53.006472+0000","last_fullsized":"2024-04-21T09:05:53.006472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006313+0000","last_change":"2024-04-21T09:05:51.013349+0000","last_active":"2024-04-21T09:05:53.006313+0000","last_peered":"2024-04-21T09:05:53.006313+0000","last_clean":"2024-04-21T09:05:53.006313+0000","last_became_active":"2024-04-21T09:05:51.012980+0000","last_became_peered":"2024-04-21T09:05:51.012980+0000","last_unstale":"2024-04-21T09:05:53.006313+0000","last_undegraded":"2024-04-21T09:05:53.006313+0000","last_fullsized":"2024-04-21T09:05:53.006313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:39:00.153892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007095+0000","last_change":"2024-04-21T09:05:45.963133+0000","last_active":"2024-04-21T09:05:53.007095+0000","last_peered":"2024-04-21T09:05:53.007095+0000","last_clean":"2024-04-21T09:05:53.007095+0000","last_became_active":"2024-04-21T09:05:45.962764+0000","last_became_peered":"2024-04-21T09:05:45.962764+0000","last_unstale":"2024-04-21T09:05:53.007095+0000","last_undegraded":"2024-04-21T09:05:53.007095+0000","last_fullsized":"2024-04-21T09:05:53.007095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:58.238118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006653+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:05:53.006653+0000","last_peered":"2024-04-21T09:05:53.006653+0000","last_clean":"2024-04-21T09:05:53.006653+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:05:53.006653+0000","last_undegraded":"2024-04-21T09:05:53.006653+0000","last_fullsized":"2024-04-21T09:05:53.006653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007554+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:05:53.007554+0000","last_peered":"2024-04-21T09:05:53.007554+0000","last_clean":"2024-04-21T09:05:53.007554+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:05:53.007554+0000","last_undegraded":"2024-04-21T09:05:53.007554+0000","last_fullsized":"2024-04-21T09:05:53.007554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998176+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:05:52.998176+0000","last_peered":"2024-04-21T09:05:52.998176+0000","last_clean":"2024-04-21T09:05:52.998176+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:05:52.998176+0000","last_undegraded":"2024-04-21T09:05:52.998176+0000","last_fullsized":"2024-04-21T09:05:52.998176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007086+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:05:53.007086+0000","last_peered":"2024-04-21T09:05:53.007086+0000","last_clean":"2024-04-21T09:05:53.007086+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:05:53.007086+0000","last_undegraded":"2024-04-21T09:05:53.007086+0000","last_fullsized":"2024-04-21T09:05:53.007086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007764+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:05:53.007764+0000","last_peered":"2024-04-21T09:05:53.007764+0000","last_clean":"2024-04-21T09:05:53.007764+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:05:53.007764+0000","last_undegraded":"2024-04-21T09:05:53.007764+0000","last_fullsized":"2024-04-21T09:05:53.007764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007791+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:05:53.007791+0000","last_peered":"2024-04-21T09:05:53.007791+0000","last_clean":"2024-04-21T09:05:53.007791+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:05:53.007791+0000","last_undegraded":"2024-04-21T09:05:53.007791+0000","last_fullsized":"2024-04-21T09:05:53.007791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006941+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:05:53.006941+0000","last_peered":"2024-04-21T09:05:53.006941+0000","last_clean":"2024-04-21T09:05:53.006941+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:05:53.006941+0000","last_undegraded":"2024-04-21T09:05:53.006941+0000","last_fullsized":"2024-04-21T09:05:53.006941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006093+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:05:53.006093+0000","last_peered":"2024-04-21T09:05:53.006093+0000","last_clean":"2024-04-21T09:05:53.006093+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:05:53.006093+0000","last_undegraded":"2024-04-21T09:05:53.006093+0000","last_fullsized":"2024-04-21T09:05:53.006093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007498+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:05:53.007498+0000","last_peered":"2024-04-21T09:05:53.007498+0000","last_clean":"2024-04-21T09:05:53.007498+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:05:53.007498+0000","last_undegraded":"2024-04-21T09:05:53.007498+0000","last_fullsized":"2024-04-21T09:05:53.007498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.101673+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:06:12.101673+0000","last_peered":"2024-04-21T09:06:12.101673+0000","last_clean":"2024-04-21T09:06:12.101673+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:06:12.101673+0000","last_undegraded":"2024-04-21T09:06:12.101673+0000","last_fullsized":"2024-04-21T09:06:12.101673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.008018+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:05:53.008018+0000","last_peered":"2024-04-21T09:05:53.008018+0000","last_clean":"2024-04-21T09:05:53.008018+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:05:53.008018+0000","last_undegraded":"2024-04-21T09:05:53.008018+0000","last_fullsized":"2024-04-21T09:05:53.008018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006726+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:05:53.006726+0000","last_peered":"2024-04-21T09:05:53.006726+0000","last_clean":"2024-04-21T09:05:53.006726+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:05:53.006726+0000","last_undegraded":"2024-04-21T09:05:53.006726+0000","last_fullsized":"2024-04-21T09:05:53.006726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091689+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:06:11.091689+0000","last_peered":"2024-04-21T09:06:11.091689+0000","last_clean":"2024-04-21T09:06:11.091689+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:06:11.091689+0000","last_undegraded":"2024-04-21T09:06:11.091689+0000","last_fullsized":"2024-04-21T09:06:11.091689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006299+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:05:53.006299+0000","last_peered":"2024-04-21T09:05:53.006299+0000","last_clean":"2024-04-21T09:05:53.006299+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:05:53.006299+0000","last_undegraded":"2024-04-21T09:05:53.006299+0000","last_fullsized":"2024-04-21T09:05:53.006299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006855+0000","last_change":"2024-04-21T09:05:51.415003+0000","last_active":"2024-04-21T09:05:53.006855+0000","last_peered":"2024-04-21T09:05:53.006855+0000","last_clean":"2024-04-21T09:05:53.006855+0000","last_became_active":"2024-04-21T09:05:51.414596+0000","last_became_peered":"2024-04-21T09:05:51.414596+0000","last_unstale":"2024-04-21T09:05:53.006855+0000","last_undegraded":"2024-04-21T09:05:53.006855+0000","last_fullsized":"2024-04-21T09:05:53.006855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:29.389711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006187+0000","last_change":"2024-04-21T09:05:46.970236+0000","last_active":"2024-04-21T09:05:53.006187+0000","last_peered":"2024-04-21T09:05:53.006187+0000","last_clean":"2024-04-21T09:05:53.006187+0000","last_became_active":"2024-04-21T09:05:46.969854+0000","last_became_peered":"2024-04-21T09:05:46.969854+0000","last_unstale":"2024-04-21T09:05:53.006187+0000","last_undegraded":"2024-04-21T09:05:53.006187+0000","last_fullsized":"2024-04-21T09:05:53.006187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:30.990859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:12.099427+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:06:12.099427+0000","last_peered":"2024-04-21T09:06:12.099427+0000","last_clean":"2024-04-21T09:06:12.099427+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:06:12.099427+0000","last_undegraded":"2024-04-21T09:06:12.099427+0000","last_fullsized":"2024-04-21T09:06:12.099427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006466+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:05:53.006466+0000","last_peered":"2024-04-21T09:05:53.006466+0000","last_clean":"2024-04-21T09:05:53.006466+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:05:53.006466+0000","last_undegraded":"2024-04-21T09:05:53.006466+0000","last_fullsized":"2024-04-21T09:05:53.006466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091457+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:06:11.091457+0000","last_peered":"2024-04-21T09:06:11.091457+0000","last_clean":"2024-04-21T09:06:11.091457+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:06:11.091457+0000","last_undegraded":"2024-04-21T09:06:11.091457+0000","last_fullsized":"2024-04-21T09:06:11.091457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007789+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:05:53.007789+0000","last_peered":"2024-04-21T09:05:53.007789+0000","last_clean":"2024-04-21T09:05:53.007789+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:05:53.007789+0000","last_undegraded":"2024-04-21T09:05:53.007789+0000","last_fullsized":"2024-04-21T09:05:53.007789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006110+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:05:53.006110+0000","last_peered":"2024-04-21T09:05:53.006110+0000","last_clean":"2024-04-21T09:05:53.006110+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:05:53.006110+0000","last_undegraded":"2024-04-21T09:05:53.006110+0000","last_fullsized":"2024-04-21T09:05:53.006110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998381+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:05:52.998381+0000","last_peered":"2024-04-21T09:05:52.998381+0000","last_clean":"2024-04-21T09:05:52.998381+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:05:52.998381+0000","last_undegraded":"2024-04-21T09:05:52.998381+0000","last_fullsized":"2024-04-21T09:05:52.998381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006412+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:05:53.006412+0000","last_peered":"2024-04-21T09:05:53.006412+0000","last_clean":"2024-04-21T09:05:53.006412+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:05:53.006412+0000","last_undegraded":"2024-04-21T09:05:53.006412+0000","last_fullsized":"2024-04-21T09:05:53.006412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998057+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:05:52.998057+0000","last_peered":"2024-04-21T09:05:52.998057+0000","last_clean":"2024-04-21T09:05:52.998057+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:05:52.998057+0000","last_undegraded":"2024-04-21T09:05:52.998057+0000","last_fullsized":"2024-04-21T09:05:52.998057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.090992+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:06:11.090992+0000","last_peered":"2024-04-21T09:06:11.090992+0000","last_clean":"2024-04-21T09:06:11.090992+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:06:11.090992+0000","last_undegraded":"2024-04-21T09:06:11.090992+0000","last_fullsized":"2024-04-21T09:06:11.090992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006438+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:05:53.006438+0000","last_peered":"2024-04-21T09:05:53.006438+0000","last_clean":"2024-04-21T09:05:53.006438+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:05:53.006438+0000","last_undegraded":"2024-04-21T09:05:53.006438+0000","last_fullsized":"2024-04-21T09:05:53.006438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998943+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:05:52.998943+0000","last_peered":"2024-04-21T09:05:52.998943+0000","last_clean":"2024-04-21T09:05:52.998943+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:05:52.998943+0000","last_undegraded":"2024-04-21T09:05:52.998943+0000","last_fullsized":"2024-04-21T09:05:52.998943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091096+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:06:11.091096+0000","last_peered":"2024-04-21T09:06:11.091096+0000","last_clean":"2024-04-21T09:06:11.091096+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:06:11.091096+0000","last_undegraded":"2024-04-21T09:06:11.091096+0000","last_fullsized":"2024-04-21T09:06:11.091096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006420+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:05:53.006420+0000","last_peered":"2024-04-21T09:05:53.006420+0000","last_clean":"2024-04-21T09:05:53.006420+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:05:53.006420+0000","last_undegraded":"2024-04-21T09:05:53.006420+0000","last_fullsized":"2024-04-21T09:05:53.006420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998385+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:05:52.998385+0000","last_peered":"2024-04-21T09:05:52.998385+0000","last_clean":"2024-04-21T09:05:52.998385+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:05:52.998385+0000","last_undegraded":"2024-04-21T09:05:52.998385+0000","last_fullsized":"2024-04-21T09:05:52.998385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006382+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:05:53.006382+0000","last_peered":"2024-04-21T09:05:53.006382+0000","last_clean":"2024-04-21T09:05:53.006382+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:05:53.006382+0000","last_undegraded":"2024-04-21T09:05:53.006382+0000","last_fullsized":"2024-04-21T09:05:53.006382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006834+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:05:53.006834+0000","last_peered":"2024-04-21T09:05:53.006834+0000","last_clean":"2024-04-21T09:05:53.006834+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:05:53.006834+0000","last_undegraded":"2024-04-21T09:05:53.006834+0000","last_fullsized":"2024-04-21T09:05:53.006834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998659+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:05:52.998659+0000","last_peered":"2024-04-21T09:05:52.998659+0000","last_clean":"2024-04-21T09:05:52.998659+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:05:52.998659+0000","last_undegraded":"2024-04-21T09:05:52.998659+0000","last_fullsized":"2024-04-21T09:05:52.998659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998139+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:05:52.998139+0000","last_peered":"2024-04-21T09:05:52.998139+0000","last_clean":"2024-04-21T09:05:52.998139+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:05:52.998139+0000","last_undegraded":"2024-04-21T09:05:52.998139+0000","last_fullsized":"2024-04-21T09:05:52.998139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007207+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:05:53.007207+0000","last_peered":"2024-04-21T09:05:53.007207+0000","last_clean":"2024-04-21T09:05:53.007207+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:05:53.007207+0000","last_undegraded":"2024-04-21T09:05:53.007207+0000","last_fullsized":"2024-04-21T09:05:53.007207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998600+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:05:52.998600+0000","last_peered":"2024-04-21T09:05:52.998600+0000","last_clean":"2024-04-21T09:05:52.998600+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:05:52.998600+0000","last_undegraded":"2024-04-21T09:05:52.998600+0000","last_fullsized":"2024-04-21T09:05:52.998600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:06:11.091888+0000","last_change":"2024-04-21T09:05:45.967431+0000","last_active":"2024-04-21T09:06:11.091888+0000","last_peered":"2024-04-21T09:06:11.091888+0000","last_clean":"2024-04-21T09:06:11.091888+0000","last_became_active":"2024-04-21T09:05:45.967106+0000","last_became_peered":"2024-04-21T09:05:45.967106+0000","last_unstale":"2024-04-21T09:06:11.091888+0000","last_undegraded":"2024-04-21T09:06:11.091888+0000","last_fullsized":"2024-04-21T09:06:11.091888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:00.877072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007853+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:05:53.007853+0000","last_peered":"2024-04-21T09:05:53.007853+0000","last_clean":"2024-04-21T09:05:53.007853+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:05:53.007853+0000","last_undegraded":"2024-04-21T09:05:53.007853+0000","last_fullsized":"2024-04-21T09:05:53.007853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006388+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:05:53.006388+0000","last_peered":"2024-04-21T09:05:53.006388+0000","last_clean":"2024-04-21T09:05:53.006388+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:05:53.006388+0000","last_undegraded":"2024-04-21T09:05:53.006388+0000","last_fullsized":"2024-04-21T09:05:53.006388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998571+0000","last_change":"2024-04-21T09:05:45.976029+0000","last_active":"2024-04-21T09:05:52.998571+0000","last_peered":"2024-04-21T09:05:52.998571+0000","last_clean":"2024-04-21T09:05:52.998571+0000","last_became_active":"2024-04-21T09:05:45.975647+0000","last_became_peered":"2024-04-21T09:05:45.975647+0000","last_unstale":"2024-04-21T09:05:52.998571+0000","last_undegraded":"2024-04-21T09:05:52.998571+0000","last_fullsized":"2024-04-21T09:05:52.998571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:50:46.781522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.008080+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:05:53.008080+0000","last_peered":"2024-04-21T09:05:53.008080+0000","last_clean":"2024-04-21T09:05:53.008080+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:05:53.008080+0000","last_undegraded":"2024-04-21T09:05:53.008080+0000","last_fullsized":"2024-04-21T09:05:53.008080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006897+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:05:53.006897+0000","last_peered":"2024-04-21T09:05:53.006897+0000","last_clean":"2024-04-21T09:05:53.006897+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:05:53.006897+0000","last_undegraded":"2024-04-21T09:05:53.006897+0000","last_fullsized":"2024-04-21T09:05:53.006897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007146+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:05:53.007146+0000","last_peered":"2024-04-21T09:05:53.007146+0000","last_clean":"2024-04-21T09:05:53.007146+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:05:53.007146+0000","last_undegraded":"2024-04-21T09:05:53.007146+0000","last_fullsized":"2024-04-21T09:05:53.007146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007677+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:05:53.007677+0000","last_peered":"2024-04-21T09:05:53.007677+0000","last_clean":"2024-04-21T09:05:53.007677+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:05:53.007677+0000","last_undegraded":"2024-04-21T09:05:53.007677+0000","last_fullsized":"2024-04-21T09:05:53.007677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007082+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:05:53.007082+0000","last_peered":"2024-04-21T09:05:53.007082+0000","last_clean":"2024-04-21T09:05:53.007082+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:05:53.007082+0000","last_undegraded":"2024-04-21T09:05:53.007082+0000","last_fullsized":"2024-04-21T09:05:53.007082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007793+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:05:53.007793+0000","last_peered":"2024-04-21T09:05:53.007793+0000","last_clean":"2024-04-21T09:05:53.007793+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:05:53.007793+0000","last_undegraded":"2024-04-21T09:05:53.007793+0000","last_fullsized":"2024-04-21T09:05:53.007793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007449+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:05:53.007449+0000","last_peered":"2024-04-21T09:05:53.007449+0000","last_clean":"2024-04-21T09:05:53.007449+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:05:53.007449+0000","last_undegraded":"2024-04-21T09:05:53.007449+0000","last_fullsized":"2024-04-21T09:05:53.007449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006946+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:05:53.006946+0000","last_peered":"2024-04-21T09:05:53.006946+0000","last_clean":"2024-04-21T09:05:53.006946+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:05:53.006946+0000","last_undegraded":"2024-04-21T09:05:53.006946+0000","last_fullsized":"2024-04-21T09:05:53.006946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998140+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:05:52.998140+0000","last_peered":"2024-04-21T09:05:52.998140+0000","last_clean":"2024-04-21T09:05:52.998140+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:05:52.998140+0000","last_undegraded":"2024-04-21T09:05:52.998140+0000","last_fullsized":"2024-04-21T09:05:52.998140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998051+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:05:52.998051+0000","last_peered":"2024-04-21T09:05:52.998051+0000","last_clean":"2024-04-21T09:05:52.998051+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:05:52.998051+0000","last_undegraded":"2024-04-21T09:05:52.998051+0000","last_fullsized":"2024-04-21T09:05:52.998051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007211+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:05:53.007211+0000","last_peered":"2024-04-21T09:05:53.007211+0000","last_clean":"2024-04-21T09:05:53.007211+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:05:53.007211+0000","last_undegraded":"2024-04-21T09:05:53.007211+0000","last_fullsized":"2024-04-21T09:05:53.007211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007874+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:05:53.007874+0000","last_peered":"2024-04-21T09:05:53.007874+0000","last_clean":"2024-04-21T09:05:53.007874+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:05:53.007874+0000","last_undegraded":"2024-04-21T09:05:53.007874+0000","last_fullsized":"2024-04-21T09:05:53.007874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007181+0000","last_change":"2024-04-21T09:05:46.972797+0000","last_active":"2024-04-21T09:05:53.007181+0000","last_peered":"2024-04-21T09:05:53.007181+0000","last_clean":"2024-04-21T09:05:53.007181+0000","last_became_active":"2024-04-21T09:05:46.972465+0000","last_became_peered":"2024-04-21T09:05:46.972465+0000","last_unstale":"2024-04-21T09:05:53.007181+0000","last_undegraded":"2024-04-21T09:05:53.007181+0000","last_fullsized":"2024-04-21T09:05:53.007181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:29:45.732617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:52.998739+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:05:52.998739+0000","last_peered":"2024-04-21T09:05:52.998739+0000","last_clean":"2024-04-21T09:05:52.998739+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:05:52.998739+0000","last_undegraded":"2024-04-21T09:05:52.998739+0000","last_fullsized":"2024-04-21T09:05:52.998739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006593+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:05:53.006593+0000","last_peered":"2024-04-21T09:05:53.006593+0000","last_clean":"2024-04-21T09:05:53.006593+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:05:53.006593+0000","last_undegraded":"2024-04-21T09:05:53.006593+0000","last_fullsized":"2024-04-21T09:05:53.006593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007457+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:05:53.007457+0000","last_peered":"2024-04-21T09:05:53.007457+0000","last_clean":"2024-04-21T09:05:53.007457+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:05:53.007457+0000","last_undegraded":"2024-04-21T09:05:53.007457+0000","last_fullsized":"2024-04-21T09:05:53.007457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006485+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:05:53.006485+0000","last_peered":"2024-04-21T09:05:53.006485+0000","last_clean":"2024-04-21T09:05:53.006485+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:05:53.006485+0000","last_undegraded":"2024-04-21T09:05:53.006485+0000","last_fullsized":"2024-04-21T09:05:53.006485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.006810+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:05:53.006810+0000","last_peered":"2024-04-21T09:05:53.006810+0000","last_clean":"2024-04-21T09:05:53.006810+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:05:53.006810+0000","last_undegraded":"2024-04-21T09:05:53.006810+0000","last_fullsized":"2024-04-21T09:05:53.006810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007532+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:05:53.007532+0000","last_peered":"2024-04-21T09:05:53.007532+0000","last_clean":"2024-04-21T09:05:53.007532+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:05:53.007532+0000","last_undegraded":"2024-04-21T09:05:53.007532+0000","last_fullsized":"2024-04-21T09:05:53.007532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T09:05:53.007374+0000","last_change":"2024-04-21T09:05:46.988031+0000","last_active":"2024-04-21T09:05:53.007374+0000","last_peered":"2024-04-21T09:05:53.007374+0000","last_clean":"2024-04-21T09:05:53.007374+0000","last_became_active":"2024-04-21T09:05:46.987629+0000","last_became_peered":"2024-04-21T09:05:46.987629+0000","last_unstale":"2024-04-21T09:05:53.007374+0000","last_undegraded":"2024-04-21T09:05:53.007374+0000","last_fullsized":"2024-04-21T09:05:53.007374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:33.326914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210748,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6068,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":91,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":17834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":490255,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612630,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1084077,"data_compressed":53245,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":490255,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":490411,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1081833,"data_compressed":53245,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":489673,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":493515,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":490367,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493549,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489777,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":489695,"data_compressed":17574,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1081517,"data_compressed":53245,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:06:16.276 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-21T09:06:16.276 INFO:tasks.cephadm:Setting up client nodes... 2024-04-21T09:06:16.277 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-21T09:06:16.359 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: 2024-04-21T09:06:16.135+0000 7f30a2500700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-21T09:06:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-mon[31413]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-mon[31413]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-21T09:06:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-mon[31413]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-mon[31413]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-mon[31413]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T09:06:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2472018933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:06:16.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/617112219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:16.557 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:16 smithi186 ceph-mon[31505]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:16 smithi186 ceph-mon[31505]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-21T09:06:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:16 smithi186 ceph-mon[31505]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:16 smithi186 ceph-mon[31505]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:06:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:16 smithi186 ceph-mon[31505]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T09:06:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2472018933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:06:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/617112219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:17.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:17 smithi083 ceph-mon[31413]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:06:17.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:17 smithi066 ceph-mon[27953]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:06:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:17 smithi186 ceph-mon[31505]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:06:18.309 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:18 smithi066 ceph-mon[27953]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:06:18.309 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:18 smithi066 ceph-mon[27953]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-21T09:06:18.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:18 smithi083 ceph-mon[31413]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:06:18.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:18 smithi083 ceph-mon[31413]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-21T09:06:18.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.425+0000 7f84c8040700 1 -- 172.21.15.66:0/2135961748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c01019e0 msgr2=0x7f84c0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:18.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.425+0000 7f84c8040700 1 --2- 172.21.15.66:0/2135961748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c01019e0 0x7f84c0101e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0099c0 tx=0x7f84bc009cd0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.425+0000 7f84c8040700 1 -- 172.21.15.66:0/2135961748 shutdown_connections 2024-04-21T09:06:18.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.425+0000 7f84c8040700 1 --2- 172.21.15.66:0/2135961748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84c0103580 0x7f84c0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.425+0000 7f84c8040700 1 --2- 172.21.15.66:0/2135961748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c01019e0 0x7f84c0101e00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.425+0000 7f84c8040700 1 --2- 172.21.15.66:0/2135961748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c0102be0 0x7f84c0103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.425+0000 7f84c8040700 1 -- 172.21.15.66:0/2135961748 >> 172.21.15.66:0/2135961748 conn(0x7f84c00fcfa0 msgr2=0x7f84c00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 -- 172.21.15.66:0/2135961748 shutdown_connections 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 -- 172.21.15.66:0/2135961748 wait complete. 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 Processor -- start 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 -- start start 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c01019e0 0x7f84c0105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84c0102be0 0x7f84c0103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c0103580 0x7f84c0104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010d5a0 con 0x7f84c0103580 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010d420 con 0x7f84c0102be0 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.426+0000 7f84c8040700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010d720 con 0x7f84c01019e0 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c0103580 0x7f84c0104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:18.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c0103580 0x7f84c0104500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50394/0 (socket says 172.21.15.66:50394) 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 -- 172.21.15.66:0/2706204000 learned_addr learned my addr 172.21.15.66:0/2706204000 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c55db700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84c0102be0 0x7f84c0103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c5ddc700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c01019e0 0x7f84c0105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 -- 172.21.15.66:0/2706204000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c01019e0 msgr2=0x7f84c0105970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c01019e0 0x7f84c0105970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 -- 172.21.15.66:0/2706204000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84c0102be0 msgr2=0x7f84c0103fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84c0102be0 0x7f84c0103fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 -- 172.21.15.66:0/2706204000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c0104db0 con 0x7f84c0103580 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c65dd700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c0103580 0x7f84c0104500 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b970 tx=0x7f84b800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84b6ffd700 1 -- 172.21.15.66:0/2706204000 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8019070 con 0x7f84c0103580 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84b6ffd700 1 -- 172.21.15.66:0/2706204000 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b800cde0 con 0x7f84c0103580 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84b6ffd700 1 -- 172.21.15.66:0/2706204000 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8007370 con 0x7f84c0103580 2024-04-21T09:06:18.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.427+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a2da0 con 0x7f84c0103580 2024-04-21T09:06:18.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.428+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a3280 con 0x7f84c0103580 2024-04-21T09:06:18.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.428+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c0050f80 con 0x7f84c0103580 2024-04-21T09:06:18.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.430+0000 7f84b6ffd700 1 -- 172.21.15.66:0/2706204000 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f84b801c070 con 0x7f84c0103580 2024-04-21T09:06:18.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.430+0000 7f84b6ffd700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84ac0a3d80 0x7f84ac0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:18.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.430+0000 7f84c5ddc700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84ac0a3d80 0x7f84ac0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:18.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.430+0000 7f84b6ffd700 1 -- 172.21.15.66:0/2706204000 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f84b80d02f0 con 0x7f84c0103580 2024-04-21T09:06:18.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.431+0000 7f84c5ddc700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84ac0a3d80 0x7f84ac0a6240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00b580 tx=0x7f84bc005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:18.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.434+0000 7f84b6ffd700 1 -- 172.21.15.66:0/2706204000 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84b8023070 con 0x7f84c0103580 2024-04-21T09:06:18.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:18 smithi186 ceph-mon[31505]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:06:18.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:18 smithi186 ceph-mon[31505]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-21T09:06:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.728+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f84c01a3b80 con 0x7f84c0103580 2024-04-21T09:06:18.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.730+0000 7f84b6ffd700 1 -- 172.21.15.66:0/2706204000 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f84b809ca40 con 0x7f84c0103580 2024-04-21T09:06:18.730 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-04-21T09:06:18.730 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAK1yRmgg2DKxAAQttv5+joKOiAKitMaAwAQg== 2024-04-21T09:06:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84ac0a3d80 msgr2=0x7f84ac0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84ac0a3d80 0x7f84ac0a6240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00b580 tx=0x7f84bc005c00 comp rx=0 tx=0).stop 2024-04-21T09:06:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c0103580 msgr2=0x7f84c0104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c0103580 0x7f84c0104500 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b970 tx=0x7f84b800bc80 comp rx=0 tx=0).stop 2024-04-21T09:06:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 shutdown_connections 2024-04-21T09:06:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84ac0a3d80 0x7f84ac0a6240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84c0103580 0x7f84c0104500 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c01019e0 0x7f84c0105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 --2- 172.21.15.66:0/2706204000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84c0102be0 0x7f84c0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 >> 172.21.15.66:0/2706204000 conn(0x7f84c00fcfa0 msgr2=0x7f84c00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 shutdown_connections 2024-04-21T09:06:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:18.732+0000 7f84c8040700 1 -- 172.21.15.66:0/2706204000 wait complete. 2024-04-21T09:06:19.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:19 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2706204000' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T09:06:19.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:19 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2706204000' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T09:06:19.475 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:06:19.475 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-21T09:06:19.475 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-21T09:06:19.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:19 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2706204000' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T09:06:19.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:19 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2706204000' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T09:06:19.512 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-21T09:06:19.512 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-21T09:06:19.512 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-21T09:06:19.529 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:19 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2706204000' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T09:06:19.529 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:19 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2706204000' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T09:06:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:20.299 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:20 smithi066 ceph-mon[27953]: pgmap v311: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-04-21T09:06:20.456 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:20 smithi083 ceph-mon[31413]: pgmap v311: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-04-21T09:06:20.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:20 smithi186 ceph-mon[31505]: pgmap v311: 129 pgs: 129 active+clean; 620 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 -- 172.21.15.66:0/2230499441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0101960 msgr2=0x7f23f0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/2230499441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0101960 0x7f23f0101de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009990 tx=0x7f23e0009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 -- 172.21.15.66:0/2230499441 shutdown_connections 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/2230499441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23f0102320 0x7f23f010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/2230499441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23f0107960 0x7f23f0107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/2230499441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0101960 0x7f23f0101de0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 -- 172.21.15.66:0/2230499441 >> 172.21.15.66:0/2230499441 conn(0x7f23f00789e0 msgr2=0x7f23f0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:21.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 -- 172.21.15.66:0/2230499441 shutdown_connections 2024-04-21T09:06:21.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.974+0000 7f23f7fb4700 1 -- 172.21.15.66:0/2230499441 wait complete. 2024-04-21T09:06:21.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 Processor -- start 2024-04-21T09:06:21.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 -- start start 2024-04-21T09:06:21.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23f0101960 0x7f23f00fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:21.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0102320 0x7f23f00feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23f0107960 0x7f23f00ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0111ba0 con 0x7f23f0102320 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0111a20 con 0x7f23f0107960 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f7fb4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0111d20 con 0x7f23f0101960 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0102320 0x7f23f00feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0102320 0x7f23f00feed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50414/0 (socket says 172.21.15.66:50414) 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 -- 172.21.15.66:0/1038113463 learned_addr learned my addr 172.21.15.66:0/1038113463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f6551700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23f0107960 0x7f23f00ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f5d50700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23f0101960 0x7f23f00fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 -- 172.21.15.66:0/1038113463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23f0101960 msgr2=0x7f23f00fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:21.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23f0101960 0x7f23f00fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 -- 172.21.15.66:0/1038113463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23f0107960 msgr2=0x7f23f00ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23f0107960 0x7f23f00ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f554f700 1 -- 172.21.15.66:0/1038113463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f01a7750 con 0x7f23f0102320 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.975+0000 7f23f5d50700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23f0101960 0x7f23f00fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.976+0000 7f23f554f700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0102320 0x7f23f00feed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f23e00057f0 tx=0x7f23e0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.976+0000 7f23e6ffd700 1 -- 172.21.15.66:0/1038113463 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0027070 con 0x7f23f0102320 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.976+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f01a79e0 con 0x7f23f0102320 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.976+0000 7f23e6ffd700 1 -- 172.21.15.66:0/1038113463 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e00040a0 con 0x7f23f0102320 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.976+0000 7f23e6ffd700 1 -- 172.21.15.66:0/1038113463 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0022dc0 con 0x7f23f0102320 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.976+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f01a7ec0 con 0x7f23f0102320 2024-04-21T09:06:21.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.977+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f0069730 con 0x7f23f0102320 2024-04-21T09:06:21.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.978+0000 7f23e6ffd700 1 -- 172.21.15.66:0/1038113463 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f23e002a070 con 0x7f23f0102320 2024-04-21T09:06:21.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.978+0000 7f23e6ffd700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23dc0a3cd0 0x7f23dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:21.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.979+0000 7f23f5d50700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23dc0a3cd0 0x7f23dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:21.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.979+0000 7f23e6ffd700 1 -- 172.21.15.66:0/1038113463 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f23e00d1c50 con 0x7f23f0102320 2024-04-21T09:06:21.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.979+0000 7f23f5d50700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23dc0a3cd0 0x7f23dc0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005fd0 tx=0x7f23ec005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:21.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:21.982+0000 7f23e6ffd700 1 -- 172.21.15.66:0/1038113463 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23e0031020 con 0x7f23f0102320 2024-04-21T09:06:22.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:22 smithi066 ceph-mon[27953]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:22.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.252+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f23f002df80 con 0x7f23f0102320 2024-04-21T09:06:22.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.257+0000 7f23e6ffd700 1 -- 172.21.15.66:0/1038113463 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7f23e0088090 con 0x7f23f0102320 2024-04-21T09:06:22.261 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.260+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23dc0a3cd0 msgr2=0x7f23dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.260+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23dc0a3cd0 0x7f23dc0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005fd0 tx=0x7f23ec005e30 comp rx=0 tx=0).stop 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.260+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0102320 msgr2=0x7f23f00feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.260+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0102320 0x7f23f00feed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f23e00057f0 tx=0x7f23e0003680 comp rx=0 tx=0).stop 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 shutdown_connections 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23dc0a3cd0 0x7f23dc0a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23f0107960 0x7f23f00ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23f0101960 0x7f23f00fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:22.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 --2- 172.21.15.66:0/1038113463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23f0102320 0x7f23f00feed0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 >> 172.21.15.66:0/1038113463 conn(0x7f23f00789e0 msgr2=0x7f23f006b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 shutdown_connections 2024-04-21T09:06:22.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:22.261+0000 7f23f7fb4700 1 -- 172.21.15.66:0/1038113463 wait complete. 2024-04-21T09:06:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:22 smithi186 ceph-mon[31505]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:22.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:22 smithi083 ceph-mon[31413]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:23.003 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":3032531159},{"type":"v1","addr":"172.21.15.66:6801","nonce":3032531159}]},"active_addr":"172.21.15.66:6801/3032531159","active_change":"2024-04-21T08:58:38.211909+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.66:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":1898303764}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":389506800}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":838597709}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":3217056690}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":1118619881}]}]} 2024-04-21T09:06:23.008 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-21T09:06:23.009 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-21T09:06:23.009 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T09:06:23.262 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:23.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:23 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1038113463' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T09:06:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:23 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1038113463' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T09:06:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:23 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1038113463' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T09:06:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:24 smithi186 ceph-mon[31505]: pgmap v313: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:24.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:24 smithi083 ceph-mon[31413]: pgmap v313: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:24.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:24 smithi066 ceph-mon[27953]: pgmap v313: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1553644636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 msgr2=0x7fad381018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1553644636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad381018f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fad34009990 tx=0x7fad34009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1553644636 shutdown_connections 2024-04-21T09:06:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1553644636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad38108c20 0x7fad38109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1553644636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad38101e30 0x7fad3810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1553644636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad381018f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1553644636 >> 172.21.15.66:0/1553644636 conn(0x7fad38076bb0 msgr2=0x7fad38076fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:25.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1553644636 shutdown_connections 2024-04-21T09:06:25.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.327+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1553644636 wait complete. 2024-04-21T09:06:25.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 Processor -- start 2024-04-21T09:06:25.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 -- start start 2024-04-21T09:06:25.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad38102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad38101e30 0x7fad38102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad38108c20 0x7fad381030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad38112c50 con 0x7fad38101470 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad38112ad0 con 0x7fad38101e30 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3f8a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fad38112dd0 con 0x7fad38108c20 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3d63c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad38102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3d63c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad38102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52334/0 (socket says 172.21.15.66:52334) 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3d63c700 1 -- 172.21.15.66:0/1013886868 learned_addr learned my addr 172.21.15.66:0/1013886868 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3ce3b700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad38101e30 0x7fad38102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.328+0000 7fad3de3d700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad38108c20 0x7fad381030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3d63c700 1 -- 172.21.15.66:0/1013886868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad38108c20 msgr2=0x7fad381030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:25.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3d63c700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad38108c20 0x7fad381030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3d63c700 1 -- 172.21.15.66:0/1013886868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad38101e30 msgr2=0x7fad38102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3d63c700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad38101e30 0x7fad38102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3d63c700 1 -- 172.21.15.66:0/1013886868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad381a7550 con 0x7fad38101470 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3ce3b700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad38101e30 0x7fad38102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3d63c700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad38102650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c920 tx=0x7fad2800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad2e7fc700 1 -- 172.21.15.66:0/1013886868 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28004a30 con 0x7fad38101470 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad2e7fc700 1 -- 172.21.15.66:0/1013886868 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad280054d0 con 0x7fad38101470 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad381a7840 con 0x7fad38101470 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad2e7fc700 1 -- 172.21.15.66:0/1013886868 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28007c70 con 0x7fad38101470 2024-04-21T09:06:25.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.329+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad381a7da0 con 0x7fad38101470 2024-04-21T09:06:25.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.330+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad38069730 con 0x7fad38101470 2024-04-21T09:06:25.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.331+0000 7fad2e7fc700 1 -- 172.21.15.66:0/1013886868 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad28004c60 con 0x7fad38101470 2024-04-21T09:06:25.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.332+0000 7fad2e7fc700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fad240a3d50 0x7fad240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:25.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.332+0000 7fad3ce3b700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fad240a3d50 0x7fad240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:25.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.332+0000 7fad2e7fc700 1 -- 172.21.15.66:0/1013886868 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fad280d0580 con 0x7fad38101470 2024-04-21T09:06:25.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.333+0000 7fad3ce3b700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fad240a3d50 0x7fad240a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fad340098e0 tx=0x7fad34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:25.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.337+0000 7fad2e7fc700 1 -- 172.21.15.66:0/1013886868 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad28014030 con 0x7fad38101470 2024-04-21T09:06:25.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.546+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad3802df80 con 0x7fad38101470 2024-04-21T09:06:25.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.547+0000 7fad2e7fc700 1 -- 172.21.15.66:0/1013886868 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7fad2809cbc0 con 0x7fad38101470 2024-04-21T09:06:25.547 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:06:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.549+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fad240a3d50 msgr2=0x7fad240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.549+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fad240a3d50 0x7fad240a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fad340098e0 tx=0x7fad34005960 comp rx=0 tx=0).stop 2024-04-21T09:06:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.549+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 msgr2=0x7fad38102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.549+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad38102650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c920 tx=0x7fad2800cc30 comp rx=0 tx=0).stop 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 shutdown_connections 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fad240a3d50 0x7fad240a6210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad38108c20 0x7fad381030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad38101470 0x7fad38102650 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 --2- 172.21.15.66:0/1013886868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad38101e30 0x7fad38102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 >> 172.21.15.66:0/1013886868 conn(0x7fad38076bb0 msgr2=0x7fad38100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 shutdown_connections 2024-04-21T09:06:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:25.550+0000 7fad3f8a0700 1 -- 172.21.15.66:0/1013886868 wait complete. 2024-04-21T09:06:26.224 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:26 smithi066 ceph-mon[27953]: pgmap v314: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-21T09:06:26.224 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:26 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1013886868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:26.256 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-21T09:06:26.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T09:06:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:26 smithi186 ceph-mon[31505]: pgmap v314: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-21T09:06:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:26 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1013886868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:26.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:26 smithi083 ceph-mon[31413]: pgmap v314: 129 pgs: 129 active+clean; 625 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-21T09:06:26.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:26 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1013886868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:28.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.291+0000 7fe98c908700 1 -- 172.21.15.66:0/3919043368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 msgr2=0x7fe988101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:28.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.291+0000 7fe98c908700 1 --2- 172.21.15.66:0/3919043368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988101410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe9700099c0 tx=0x7fe970009cd0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.292 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:28 smithi066 ceph-mon[27953]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 -- 172.21.15.66:0/3919043368 shutdown_connections 2024-04-21T09:06:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 --2- 172.21.15.66:0/3919043368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe988101950 0x7fe98810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 --2- 172.21.15.66:0/3919043368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe988107a60 0x7fe988107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 --2- 172.21.15.66:0/3919043368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988101410 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 -- 172.21.15.66:0/3919043368 >> 172.21.15.66:0/3919043368 conn(0x7fe988077f40 msgr2=0x7fe9880ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 -- 172.21.15.66:0/3919043368 shutdown_connections 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 -- 172.21.15.66:0/3919043368 wait complete. 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.292+0000 7fe98c908700 1 Processor -- start 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98c908700 1 -- start start 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98c908700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98c908700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe988101950 0x7fe988199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98c908700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe988107a60 0x7fe98819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98c908700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe988112d70 con 0x7fe988100f90 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98c908700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe988112bf0 con 0x7fe988107a60 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98c908700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe988112ef0 con 0x7fe988101950 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52344/0 (socket says 172.21.15.66:52344) 2024-04-21T09:06:28.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 -- 172.21.15.66:0/1565742350 learned_addr learned my addr 172.21.15.66:0/1565742350 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe986d9d700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe988107a60 0x7fe98819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe985d9b700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe988101950 0x7fe988199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 -- 172.21.15.66:0/1565742350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe988101950 msgr2=0x7fe988199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe988101950 0x7fe988199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 -- 172.21.15.66:0/1565742350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe988107a60 msgr2=0x7fe98819daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe988107a60 0x7fe98819daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe98659c700 1 -- 172.21.15.66:0/1565742350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98819e1e0 con 0x7fe988100f90 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.293+0000 7fe986d9d700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe988107a60 0x7fe98819daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.294+0000 7fe98659c700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988199220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe98000b530 tx=0x7fe98000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.294+0000 7fe97f7fe700 1 -- 172.21.15.66:0/1565742350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980015020 con 0x7fe988100f90 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.294+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98819e4d0 con 0x7fe988100f90 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.294+0000 7fe97f7fe700 1 -- 172.21.15.66:0/1565742350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe980011ce0 con 0x7fe988100f90 2024-04-21T09:06:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.294+0000 7fe97f7fe700 1 -- 172.21.15.66:0/1565742350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980010dd0 con 0x7fe988100f90 2024-04-21T09:06:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.294+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881a5d50 con 0x7fe988100f90 2024-04-21T09:06:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.295+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98810afe0 con 0x7fe988100f90 2024-04-21T09:06:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.296+0000 7fe97f7fe700 1 -- 172.21.15.66:0/1565742350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe980018020 con 0x7fe988100f90 2024-04-21T09:06:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.297+0000 7fe97f7fe700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe9740a3cd0 0x7fe9740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.297+0000 7fe985d9b700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe9740a3cd0 0x7fe9740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.297+0000 7fe97f7fe700 1 -- 172.21.15.66:0/1565742350 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fe9800cfd80 con 0x7fe988100f90 2024-04-21T09:06:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.297+0000 7fe985d9b700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe9740a3cd0 0x7fe9740a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe970009990 tx=0x7fe970005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:28.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.301+0000 7fe97f7fe700 1 -- 172.21.15.66:0/1565742350 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe98001f020 con 0x7fe988100f90 2024-04-21T09:06:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.512+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe98819a530 con 0x7fe988100f90 2024-04-21T09:06:28.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.514+0000 7fe97f7fe700 1 -- 172.21.15.66:0/1565742350 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24090 (secure 0 0 0) 0x7fe98009c4d0 con 0x7fe988100f90 2024-04-21T09:06:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:06:28.514 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:05:52.994229+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe9740a3cd0 msgr2=0x7fe9740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe9740a3cd0 0x7fe9740a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe970009990 tx=0x7fe970005a90 comp rx=0 tx=0).stop 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 msgr2=0x7fe988199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988199220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe98000b530 tx=0x7fe98000b840 comp rx=0 tx=0).stop 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 shutdown_connections 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe9740a3cd0 0x7fe9740a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe988107a60 0x7fe98819daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe988100f90 0x7fe988199220 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 --2- 172.21.15.66:0/1565742350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe988101950 0x7fe988199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:28.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 >> 172.21.15.66:0/1565742350 conn(0x7fe988077f40 msgr2=0x7fe988105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:28.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 shutdown_connections 2024-04-21T09:06:28.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:28.516+0000 7fe98c908700 1 -- 172.21.15.66:0/1565742350 wait complete. 2024-04-21T09:06:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:28 smithi186 ceph-mon[31505]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:28.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:28 smithi083 ceph-mon[31413]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-21T09:06:29.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-21T09:06:29.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-21T09:06:29.303 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-21T09:06:29.303 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-21T09:06:29.303 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-21T09:06:29.304 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-21T09:06:29.304 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-21T09:06:29.305 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-21T09:06:29.305 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-21T09:06:29.306 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-21T09:06:29.306 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-21T09:06:29.306 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-21T09:06:29.401 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:29 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1565742350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:29 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1565742350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:29 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1565742350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:06:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.120 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.121 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.121 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.121 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.124 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.125 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.127 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.128 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.135 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.137 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:30.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:30 smithi083 ceph-mon[31413]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-21T09:06:30.437 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:30 smithi066 ceph-mon[27953]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-21T09:06:30.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:30 smithi186 ceph-mon[31505]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-21T09:06:32.281 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:32 smithi066 ceph-mon[27953]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-21T09:06:32.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:32 smithi186 ceph-mon[31505]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-21T09:06:32.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:32 smithi083 ceph-mon[31413]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-21T09:06:34.498 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:34 smithi066 ceph-mon[27953]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-21T09:06:34.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:34 smithi186 ceph-mon[31505]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-21T09:06:34.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:34 smithi083 ceph-mon[31413]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-21T09:06:36.341 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:36 smithi066 ceph-mon[27953]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-21T09:06:36.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:36 smithi186 ceph-mon[31505]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-21T09:06:36.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.283+0000 7ff0a88d0700 1 -- 172.21.15.66:0/1197780812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4102320 msgr2=0x7ff0a410e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:36.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.283+0000 7ff0a88d0700 1 --2- 172.21.15.66:0/1197780812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4102320 0x7ff0a410e830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00b0a0 tx=0x7ff09c00b3b0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.283+0000 7ff0a88d0700 1 -- 172.21.15.66:0/1197780812 shutdown_connections 2024-04-21T09:06:36.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 --2- 172.21.15.66:0/1197780812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4107960 0x7ff0a4107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 --2- 172.21.15.66:0/1197780812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4102320 0x7ff0a410e830 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 --2- 172.21.15.66:0/1197780812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a4101960 0x7ff0a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 -- 172.21.15.66:0/1197780812 >> 172.21.15.66:0/1197780812 conn(0x7ff0a40789e0 msgr2=0x7ff0a4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 -- 172.21.15.66:0/1197780812 shutdown_connections 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 -- 172.21.15.66:0/1197780812 wait complete. 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 Processor -- start 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.284+0000 7ff0a88d0700 1 -- start start 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4101960 0x7ff0a40fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a4102320 0x7ff0a40feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4107960 0x7ff0a40ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4111ba0 con 0x7ff0a4107960 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4111a20 con 0x7ff0a4101960 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4111d20 con 0x7ff0a4102320 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4107960 0x7ff0a40ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4107960 0x7ff0a40ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50922/0 (socket says 172.21.15.66:50922) 2024-04-21T09:06:36.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 -- 172.21.15.66:0/81478548 learned_addr learned my addr 172.21.15.66:0/81478548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a1d9b700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a4102320 0x7ff0a40feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a259c700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4101960 0x7ff0a40fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 -- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a4102320 msgr2=0x7ff0a40feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a4102320 0x7ff0a40feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 -- 172.21.15.66:0/81478548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4101960 msgr2=0x7ff0a40fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4101960 0x7ff0a40fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 -- 172.21.15.66:0/81478548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a41a76f0 con 0x7ff0a4107960 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a2d9d700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4107960 0x7ff0a40ff410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0062a0 tx=0x7ff09c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c021070 con 0x7ff0a4107960 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a41a7980 con 0x7ff0a4107960 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff09c007d00 con 0x7ff0a4107960 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c01d450 con 0x7ff0a4107960 2024-04-21T09:06:36.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.285+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a41a7e80 con 0x7ff0a4107960 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.286+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0a4063110 con 0x7ff0a4107960 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.288+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff09c004000 con 0x7ff0a4107960 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.288+0000 7ff0937fe700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff08c0a3cd0 0x7ff08c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.288+0000 7ff0a259c700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff08c0a3cd0 0x7ff08c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.288+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7ff09c00f020 con 0x7ff0a4107960 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.289+0000 7ff0937fe700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] conn(0x7ff08c0ae990 0x7ff08c0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.289+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 --> [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff09c003de0 con 0x7ff08c0ae990 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.289+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff09c0d24e0 con 0x7ff0a4107960 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.289+0000 7ff0a259c700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff08c0a3cd0 0x7ff08c0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff098005fd0 tx=0x7ff09800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:36.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.289+0000 7ff0a1d9b700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] conn(0x7ff08c0ae990 0x7ff08c0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.289+0000 7ff0a1d9b700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] conn(0x7ff08c0ae990 0x7ff08c0b0dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.290+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== osd.2 v2:172.21.15.186:6800/993640486 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff09c003de0 con 0x7ff08c0ae990 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.307+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 --> [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff0a4063510 con 0x7ff08c0ae990 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0937fe700 1 -- 172.21.15.66:0/81478548 <== osd.2 v2:172.21.15.186:6800/993640486 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff0a4063510 con 0x7ff08c0ae990 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] conn(0x7ff08c0ae990 msgr2=0x7ff08c0b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] conn(0x7ff08c0ae990 0x7ff08c0b0dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff08c0a3cd0 msgr2=0x7ff08c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff08c0a3cd0 0x7ff08c0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff098005fd0 tx=0x7ff09800a560 comp rx=0 tx=0).stop 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4107960 msgr2=0x7ff0a40ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 --2- 172.21.15.66:0/81478548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0a4107960 0x7ff0a40ff410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0062a0 tx=0x7ff09c004ed0 comp rx=0 tx=0).stop 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a2d9d700 1 -- 172.21.15.66:0/81478548 reap_dead start 2024-04-21T09:06:36.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 shutdown_connections 2024-04-21T09:06:36.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 >> 172.21.15.66:0/81478548 conn(0x7ff0a40789e0 msgr2=0x7ff0a406b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:36.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 shutdown_connections 2024-04-21T09:06:36.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:36.308+0000 7ff0a88d0700 1 -- 172.21.15.66:0/81478548 wait complete. 2024-04-21T09:06:36.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:36 smithi083 ceph-mon[31413]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 -- 172.21.15.66:0/3654137488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4102a90 msgr2=0x7f4ea4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 --2- 172.21.15.66:0/3654137488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4102a90 0x7f4ea4102f10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009990 tx=0x7f4e94009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 -- 172.21.15.66:0/3654137488 shutdown_connections 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 --2- 172.21.15.66:0/3654137488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ea4103450 0x7f4ea410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 --2- 172.21.15.66:0/3654137488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea4108a50 0x7f4ea4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 --2- 172.21.15.66:0/3654137488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4102a90 0x7f4ea4102f10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 -- 172.21.15.66:0/3654137488 >> 172.21.15.66:0/3654137488 conn(0x7f4ea4079ba0 msgr2=0x7f4ea4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 -- 172.21.15.66:0/3654137488 shutdown_connections 2024-04-21T09:06:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.308+0000 7f4eac797700 1 -- 172.21.15.66:0/3654137488 wait complete. 2024-04-21T09:06:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 Processor -- start 2024-04-21T09:06:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 -- start start 2024-04-21T09:06:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ea4102a90 0x7f4ea4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4103450 0x7f4ea4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea4108a50 0x7f4ea40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea4112be0 con 0x7f4ea4103450 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea4112a60 con 0x7f4ea4102a90 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eac797700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea4112d60 con 0x7f4ea4108a50 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4ea9d32700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4103450 0x7f4ea4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4ea9d32700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4103450 0x7f4ea4075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50936/0 (socket says 172.21.15.66:50936) 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4ea9d32700 1 -- 172.21.15.66:0/3599079751 learned_addr learned my addr 172.21.15.66:0/3599079751 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eaad34700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea4108a50 0x7f4ea40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4eaa533700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ea4102a90 0x7f4ea4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4ea9d32700 1 -- 172.21.15.66:0/3599079751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea4108a50 msgr2=0x7f4ea40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.309+0000 7f4ea9d32700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea4108a50 0x7f4ea40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4ea9d32700 1 -- 172.21.15.66:0/3599079751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ea4102a90 msgr2=0x7f4ea4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4ea9d32700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ea4102a90 0x7f4ea4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4ea9d32700 1 -- 172.21.15.66:0/3599079751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea41a7600 con 0x7f4ea4103450 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4ea9d32700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4103450 0x7f4ea4075280 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4e94000c00 tx=0x7f4e940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:38.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94027070 con 0x7f4ea4103450 2024-04-21T09:06:38.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea41a7890 con 0x7f4ea4103450 2024-04-21T09:06:38.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e94003f20 con 0x7f4ea4103450 2024-04-21T09:06:38.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94022ca0 con 0x7f4ea4103450 2024-04-21T09:06:38.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.310+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea41a7d70 con 0x7f4ea4103450 2024-04-21T09:06:38.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.311+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ea40630d0 con 0x7f4ea4103450 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.312+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e9402a070 con 0x7f4ea4103450 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4e9b7fe700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e900a3cd0 0x7f4e900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f4e940d2060 con 0x7f4ea4103450 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4eaa533700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e900a3cd0 0x7f4e900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4e9b7fe700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] conn(0x7f4e900ae990 0x7f4e900b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 --> [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e940095d0 con 0x7f4e900ae990 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4e940d24b0 con 0x7f4ea4103450 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4eaad34700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] conn(0x7f4e900ae990 0x7f4e900b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:38.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.313+0000 7f4eaa533700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e900a3cd0 0x7f4e900a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0005950 tx=0x7f4ea000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:38.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.314+0000 7f4eaad34700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] conn(0x7f4e900ae990 0x7f4e900b0dd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:38.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.314+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== osd.9 v2:172.21.15.66:6826/1483038287 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e940095d0 con 0x7f4e900ae990 2024-04-21T09:06:38.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.331+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 --> [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ea4063550 con 0x7f4e900ae990 2024-04-21T09:06:38.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.331+0000 7f4e9b7fe700 1 -- 172.21.15.66:0/3599079751 <== osd.9 v2:172.21.15.66:6826/1483038287 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4ea4063550 con 0x7f4e900ae990 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.331+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] conn(0x7f4e900ae990 msgr2=0x7f4e900b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.331+0000 7f4eac797700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] conn(0x7f4e900ae990 0x7f4e900b0dd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e900a3cd0 msgr2=0x7f4e900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e900a3cd0 0x7f4e900a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0005950 tx=0x7f4ea000b410 comp rx=0 tx=0).stop 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4103450 msgr2=0x7f4ea4075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 --2- 172.21.15.66:0/3599079751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ea4103450 0x7f4ea4075280 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4e94000c00 tx=0x7f4e940047b0 comp rx=0 tx=0).stop 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eaad34700 1 -- 172.21.15.66:0/3599079751 reap_dead start 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 shutdown_connections 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 >> 172.21.15.66:0/3599079751 conn(0x7f4ea4079ba0 msgr2=0x7f4ea41018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 shutdown_connections 2024-04-21T09:06:38.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.332+0000 7f4eac797700 1 -- 172.21.15.66:0/3599079751 wait complete. 2024-04-21T09:06:38.341 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:38 smithi066 ceph-mon[27953]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T09:06:38.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:38 smithi186 ceph-mon[31505]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T09:06:38.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:38 smithi083 ceph-mon[31413]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T09:06:39.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 -- 172.21.15.66:0/1881755417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 msgr2=0x7f940806b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 --2- 172.21.15.66:0/1881755417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 0x7f940806b5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009990 tx=0x7f93f8009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 -- 172.21.15.66:0/1881755417 shutdown_connections 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 --2- 172.21.15.66:0/1881755417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940806bb10 0x7f940810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 --2- 172.21.15.66:0/1881755417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408108a30 0x7f9408108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 --2- 172.21.15.66:0/1881755417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 0x7f940806b5d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 -- 172.21.15.66:0/1881755417 >> 172.21.15.66:0/1881755417 conn(0x7f94080788f0 msgr2=0x7f9408078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 -- 172.21.15.66:0/1881755417 shutdown_connections 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.946+0000 7f940d652700 1 -- 172.21.15.66:0/1881755417 wait complete. 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 Processor -- start 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 -- start start 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 0x7f9408199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940806bb10 0x7f94081996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408108a30 0x7f940819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9408112bd0 con 0x7f940806b170 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9408112a50 con 0x7f940806bb10 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f940d652700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9408112d50 con 0x7f9408108a30 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f9406ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 0x7f9408199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f9406ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 0x7f9408199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50950/0 (socket says 172.21.15.66:50950) 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.947+0000 7f9406ffd700 1 -- 172.21.15.66:0/3529010271 learned_addr learned my addr 172.21.15.66:0/3529010271 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f94067fc700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940806bb10 0x7f94081996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f94077fe700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408108a30 0x7f940819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f9406ffd700 1 -- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408108a30 msgr2=0x7f940819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f9406ffd700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408108a30 0x7f940819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f9406ffd700 1 -- 172.21.15.66:0/3529010271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940806bb10 msgr2=0x7f94081996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f9406ffd700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f940806bb10 0x7f94081996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f9406ffd700 1 -- 172.21.15.66:0/3529010271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940819e190 con 0x7f940806b170 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f94077fe700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408108a30 0x7f940819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f9406ffd700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 0x7f9408199180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f93f000bfa0 tx=0x7f93f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f000dea0 con 0x7f940806b170 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f0009bb0 con 0x7f940806b170 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940819e480 con 0x7f940806b170 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f001d9b0 con 0x7f940806b170 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.948+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9408101970 con 0x7f940806b170 2024-04-21T09:06:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.949+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94080631d0 con 0x7f940806b170 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.950+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f93f0009700 con 0x7f940806b170 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.951+0000 7f93fffff700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3cd0 0x7f93f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.951+0000 7f94067fc700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3cd0 0x7f93f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.951+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f93f00d4c30 con 0x7f940806b170 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.952+0000 7f93fffff700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] conn(0x7f93f40ae990 0x7f93f40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.952+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 --> [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93f0016ec0 con 0x7f93f40ae990 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.952+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f93f00d5080 con 0x7f940806b170 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.952+0000 7f94067fc700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3cd0 0x7f93f40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006010 tx=0x7f93f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.952+0000 7f94077fe700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] conn(0x7f93f40ae990 0x7f93f40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.952+0000 7f94077fe700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] conn(0x7f93f40ae990 0x7f93f40b0dd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.953+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== osd.11 v2:172.21.15.186:6824/3472608146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93f0016ec0 con 0x7f93f40ae990 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.969+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 --> [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9408063650 con 0x7f93f40ae990 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f93fffff700 1 -- 172.21.15.66:0/3529010271 <== osd.11 v2:172.21.15.186:6824/3472608146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9408063650 con 0x7f93f40ae990 2024-04-21T09:06:39.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] conn(0x7f93f40ae990 msgr2=0x7f93f40b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] conn(0x7f93f40ae990 0x7f93f40b0dd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3cd0 msgr2=0x7f93f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f93f40a3cd0 0x7f93f40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006010 tx=0x7f93f801a040 comp rx=0 tx=0).stop 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 msgr2=0x7f9408199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 --2- 172.21.15.66:0/3529010271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f940806b170 0x7f9408199180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f93f000bfa0 tx=0x7f93f000dc60 comp rx=0 tx=0).stop 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f94077fe700 1 -- 172.21.15.66:0/3529010271 reap_dead start 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 shutdown_connections 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.970+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 >> 172.21.15.66:0/3529010271 conn(0x7f94080788f0 msgr2=0x7f94080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.971+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 shutdown_connections 2024-04-21T09:06:39.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:38.971+0000 7f940d652700 1 -- 172.21.15.66:0/3529010271 wait complete. 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-21T09:06:39.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-21T09:06:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-21T09:06:39.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-21T09:06:39.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 -- 172.21.15.66:0/2634546336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288101940 msgr2=0x7fd288101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 --2- 172.21.15.66:0/2634546336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288101940 0x7fd288101dc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd278009990 tx=0x7fd278009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 -- 172.21.15.66:0/2634546336 shutdown_connections 2024-04-21T09:06:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 --2- 172.21.15.66:0/2634546336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd288102300 0x7fd28810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 --2- 172.21.15.66:0/2634546336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288107990 0x7fd288107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 --2- 172.21.15.66:0/2634546336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288101940 0x7fd288101dc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 -- 172.21.15.66:0/2634546336 >> 172.21.15.66:0/2634546336 conn(0x7fd2880788f0 msgr2=0x7fd288078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 -- 172.21.15.66:0/2634546336 shutdown_connections 2024-04-21T09:06:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 -- 172.21.15.66:0/2634546336 wait complete. 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.814+0000 7fd28fe39700 1 Processor -- start 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28fe39700 1 -- start start 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28fe39700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd288101940 0x7fd2881991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28fe39700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288102300 0x7fd2881996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28fe39700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288107990 0x7fd28819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28fe39700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111b60 con 0x7fd288102300 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28fe39700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2881119e0 con 0x7fd288101940 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28fe39700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111ce0 con 0x7fd288107990 2024-04-21T09:06:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288102300 0x7fd2881996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288102300 0x7fd2881996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50974/0 (socket says 172.21.15.66:50974) 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 -- 172.21.15.66:0/1587775807 learned_addr learned my addr 172.21.15.66:0/1587775807 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28dbd5700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd288101940 0x7fd2881991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28e3d6700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288107990 0x7fd28819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 -- 172.21.15.66:0/1587775807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288107990 msgr2=0x7fd28819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288107990 0x7fd28819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 -- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd288101940 msgr2=0x7fd2881991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd288101940 0x7fd2881991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 -- 172.21.15.66:0/1587775807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28819e170 con 0x7fd288102300 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.815+0000 7fd28d3d4700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288102300 0x7fd2881996f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd2780057c0 tx=0x7fd2780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.816+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278027070 con 0x7fd288102300 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.816+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd278003f20 con 0x7fd288102300 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.816+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28819e400 con 0x7fd288102300 2024-04-21T09:06:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.816+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278022bc0 con 0x7fd288102300 2024-04-21T09:06:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.816+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881a5ce0 con 0x7fd288102300 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.817+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd288063190 con 0x7fd288102300 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.818+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd27802a070 con 0x7fd288102300 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.818+0000 7fd27effd700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd2740a3bb0 0x7fd2740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.819+0000 7fd28dbd5700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd2740a3bb0 0x7fd2740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.819+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd2780d15a0 con 0x7fd288102300 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.819+0000 7fd27effd700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] conn(0x7fd2740aed10 0x7fd2740b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.819+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 --> [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd278003f20 con 0x7fd2740aed10 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.819+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd2780d8050 con 0x7fd288102300 2024-04-21T09:06:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.819+0000 7fd28dbd5700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd2740a3bb0 0x7fd2740a6070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd284009a10 tx=0x7fd284009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.819+0000 7fd28e3d6700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] conn(0x7fd2740aed10 0x7fd2740b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.820+0000 7fd28e3d6700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] conn(0x7fd2740aed10 0x7fd2740b11b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.820+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== osd.4 v2:172.21.15.83:6808/2820280622 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd278003f20 con 0x7fd2740aed10 2024-04-21T09:06:39.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.837+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 --> [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2880635c0 con 0x7fd2740aed10 2024-04-21T09:06:39.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.837+0000 7fd27effd700 1 -- 172.21.15.66:0/1587775807 <== osd.4 v2:172.21.15.83:6808/2820280622 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd2880635c0 con 0x7fd2740aed10 2024-04-21T09:06:39.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] conn(0x7fd2740aed10 msgr2=0x7fd2740b11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] conn(0x7fd2740aed10 0x7fd2740b11b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd2740a3bb0 msgr2=0x7fd2740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd2740a3bb0 0x7fd2740a6070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd284009a10 tx=0x7fd284009190 comp rx=0 tx=0).stop 2024-04-21T09:06:39.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288102300 msgr2=0x7fd2881996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:39.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 --2- 172.21.15.66:0/1587775807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd288102300 0x7fd2881996f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd2780057c0 tx=0x7fd2780057f0 comp rx=0 tx=0).stop 2024-04-21T09:06:39.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28e3d6700 1 -- 172.21.15.66:0/1587775807 reap_dead start 2024-04-21T09:06:39.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 shutdown_connections 2024-04-21T09:06:39.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 >> 172.21.15.66:0/1587775807 conn(0x7fd2880788f0 msgr2=0x7fd28806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:39.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 shutdown_connections 2024-04-21T09:06:39.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:39.838+0000 7fd28fe39700 1 -- 172.21.15.66:0/1587775807 wait complete. 2024-04-21T09:06:40.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T09:06:40.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-21T09:06:40.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-21T09:06:40.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-21T09:06:40.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-21T09:06:40.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-21T09:06:40.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-21T09:06:40.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-21T09:06:40.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]': finished 2024-04-21T09:06:40.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-21T09:06:40.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-21T09:06:40.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:40 smithi066 ceph-mon[27953]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T09:06:40.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-21T09:06:40.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]': finished 2024-04-21T09:06:40.702 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-21T09:06:40.702 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-21T09:06:40.702 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:40 smithi186 ceph-mon[31505]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T09:06:40.702 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T09:06:40.702 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-21T09:06:40.702 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-21T09:06:40.703 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:40 smithi083 ceph-mon[31413]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T09:06:41.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:41 smithi066 ceph-mon[27953]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T09:06:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:41 smithi186 ceph-mon[31505]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T09:06:41.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:41 smithi083 ceph-mon[31413]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T09:06:42.361 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:42 smithi066 ceph-mon[27953]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:06:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 -- 172.21.15.66:0/2200757296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 msgr2=0x7f84b8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 --2- 172.21.15.66:0/2200757296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 0x7f84b8102f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009990 tx=0x7f84a4009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 -- 172.21.15.66:0/2200757296 shutdown_connections 2024-04-21T09:06:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 --2- 172.21.15.66:0/2200757296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b8108a70 0x7f84b8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 --2- 172.21.15.66:0/2200757296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b8103480 0x7f84b810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 --2- 172.21.15.66:0/2200757296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 0x7f84b8102f40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 -- 172.21.15.66:0/2200757296 >> 172.21.15.66:0/2200757296 conn(0x7f84b8079bb0 msgr2=0x7f84b8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 -- 172.21.15.66:0/2200757296 shutdown_connections 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.466+0000 7f84bee52700 1 -- 172.21.15.66:0/2200757296 wait complete. 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 Processor -- start 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 -- start start 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 0x7f84b81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b8103480 0x7f84b8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b8108a70 0x7f84b819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8112d50 con 0x7f84b8102ac0 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8112bd0 con 0x7f84b8108a70 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bee52700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8112ed0 con 0x7f84b8103480 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 0x7f84b81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 0x7f84b81992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50978/0 (socket says 172.21.15.66:50978) 2024-04-21T09:06:42.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 -- 172.21.15.66:0/2590496343 learned_addr learned my addr 172.21.15.66:0/2590496343 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84b7fff700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b8103480 0x7f84b8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bd3ef700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b8108a70 0x7f84b819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 -- 172.21.15.66:0/2590496343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b8103480 msgr2=0x7f84b8199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b8103480 0x7f84b8199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 -- 172.21.15.66:0/2590496343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b8108a70 msgr2=0x7f84b819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b8108a70 0x7f84b819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 -- 172.21.15.66:0/2590496343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b819e2d0 con 0x7f84b8102ac0 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84b7fff700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84b8103480 0x7f84b8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.467+0000 7f84bcbee700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 0x7f84b81992c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b530 tx=0x7f84ac00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.468+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac015020 con 0x7f84b8102ac0 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.468+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ac011ce0 con 0x7f84b8102ac0 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.468+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b819e5c0 con 0x7f84b8102ac0 2024-04-21T09:06:42.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.468+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac010d20 con 0x7f84b8102ac0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.468+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b81a5e40 con 0x7f84b8102ac0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.469+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84b8063150 con 0x7f84b8102ac0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.470+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f84ac018020 con 0x7f84b8102ac0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84b5ffb700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84a80a3cd0 0x7f84a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84b7fff700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84a80a3cd0 0x7f84a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f84ac0d06f0 con 0x7f84b8102ac0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84b5ffb700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] conn(0x7f84a80aee30 0x7f84a80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 --> [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84ac01deb0 con 0x7f84a80aee30 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f84ac0d0b40 con 0x7f84b8102ac0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84bd3ef700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] conn(0x7f84a80aee30 0x7f84a80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84b7fff700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84a80a3cd0 0x7f84a80a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009750 tx=0x7f84a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.471+0000 7f84bd3ef700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] conn(0x7f84a80aee30 0x7f84a80b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:42.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.472+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== osd.0 v2:172.21.15.66:6802/1993480786 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84ac01deb0 con 0x7f84a80aee30 2024-04-21T09:06:42.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.488+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 --> [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84b80635d0 con 0x7f84a80aee30 2024-04-21T09:06:42.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84b5ffb700 1 -- 172.21.15.66:0/2590496343 <== osd.0 v2:172.21.15.66:6802/1993480786 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f84b80635d0 con 0x7f84a80aee30 2024-04-21T09:06:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] conn(0x7f84a80aee30 msgr2=0x7f84a80b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] conn(0x7f84a80aee30 0x7f84a80b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84a80a3cd0 msgr2=0x7f84a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f84a80a3cd0 0x7f84a80a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009750 tx=0x7f84a4005fd0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 msgr2=0x7f84b81992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 --2- 172.21.15.66:0/2590496343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84b8102ac0 0x7f84b81992c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b530 tx=0x7f84ac00b8f0 comp rx=0 tx=0).stop 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bd3ef700 1 -- 172.21.15.66:0/2590496343 reap_dead start 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 shutdown_connections 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.489+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 >> 172.21.15.66:0/2590496343 conn(0x7f84b8079bb0 msgr2=0x7f84b8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.490+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 shutdown_connections 2024-04-21T09:06:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:42.490+0000 7f84bee52700 1 -- 172.21.15.66:0/2590496343 wait complete. 2024-04-21T09:06:42.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:42 smithi186 ceph-mon[31505]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:06:42.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:42 smithi083 ceph-mon[31413]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:06:43.036 INFO:teuthology.orchestra.run.smithi066.stdout:347892351000 2024-04-21T09:06:43.036 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-21T09:06:43.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:43 smithi066 ceph-mon[27953]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 150 B/s, 0 objects/s recovering 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 -- 172.21.15.66:0/2168591051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c06b110 msgr2=0x7f666c06b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 --2- 172.21.15.66:0/2168591051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c06b110 0x7f666c06b570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6660009990 tx=0x7f6660009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 -- 172.21.15.66:0/2168591051 shutdown_connections 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 --2- 172.21.15.66:0/2168591051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f666c06bab0 0x7f666c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 --2- 172.21.15.66:0/2168591051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f666c108a50 0x7f666c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 --2- 172.21.15.66:0/2168591051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c06b110 0x7f666c06b570 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 -- 172.21.15.66:0/2168591051 >> 172.21.15.66:0/2168591051 conn(0x7f666c0788f0 msgr2=0x7f666c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:43.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 -- 172.21.15.66:0/2168591051 shutdown_connections 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.425+0000 7f6671a6f700 1 -- 172.21.15.66:0/2168591051 wait complete. 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.426+0000 7f6671a6f700 1 Processor -- start 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.426+0000 7f6671a6f700 1 -- start start 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.426+0000 7f6671a6f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f666c06b110 0x7f666c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.426+0000 7f6671a6f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f666c06bab0 0x7f666c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6671a6f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c108a50 0x7f666c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6671a6f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f666c112c50 con 0x7f666c108a50 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6671a6f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f666c112ad0 con 0x7f666c06b110 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6671a6f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f666c112dd0 con 0x7f666c06bab0 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c108a50 0x7f666c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c108a50 0x7f666c19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50990/0 (socket says 172.21.15.66:50990) 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 -- 172.21.15.66:0/1048964574 learned_addr learned my addr 172.21.15.66:0/1048964574 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:43.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666b7fe700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f666c06b110 0x7f666c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666affd700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f666c06bab0 0x7f666c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 -- 172.21.15.66:0/1048964574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f666c06bab0 msgr2=0x7f666c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f666c06bab0 0x7f666c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 -- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f666c06b110 msgr2=0x7f666c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f666c06b110 0x7f666c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 -- 172.21.15.66:0/1048964574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c19e190 con 0x7f666c108a50 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f666bfff700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c108a50 0x7f666c19da50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f666400b970 tx=0x7f666400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664019070 con 0x7f666c108a50 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c19e480 con 0x7f666c108a50 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f666400cde0 con 0x7f666c108a50 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664007370 con 0x7f666c108a50 2024-04-21T09:06:43.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.427+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c1a5d00 con 0x7f666c108a50 2024-04-21T09:06:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.428+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f666c063150 con 0x7f666c108a50 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.430+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f666401c070 con 0x7f666c108a50 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.430+0000 7f6668ff9700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f66540a3cd0 0x7f66540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.430+0000 7f666b7fe700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f66540a3cd0 0x7f66540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.430+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f66640d06e0 con 0x7f666c108a50 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.431+0000 7f6668ff9700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] conn(0x7f66540aee30 0x7f66540b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.431+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 --> [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66540b19a0 con 0x7f66540aee30 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.431+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f66640d0b30 con 0x7f666c108a50 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.431+0000 7f666b7fe700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f66540a3cd0 0x7f66540a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f665c00ac50 tx=0x7f665c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.431+0000 7f666affd700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] conn(0x7f66540aee30 0x7f66540b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.431+0000 7f666affd700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] conn(0x7f66540aee30 0x7f66540b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:43.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.432+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== osd.7 v2:172.21.15.83:6816/1973931443 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66540b19a0 con 0x7f66540aee30 2024-04-21T09:06:43.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.449+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 --> [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f666c0635d0 con 0x7f66540aee30 2024-04-21T09:06:43.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.450+0000 7f6668ff9700 1 -- 172.21.15.66:0/1048964574 <== osd.7 v2:172.21.15.83:6816/1973931443 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f666c0635d0 con 0x7f66540aee30 2024-04-21T09:06:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.450+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] conn(0x7f66540aee30 msgr2=0x7f66540b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.450+0000 7f6671a6f700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] conn(0x7f66540aee30 0x7f66540b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.450+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f66540a3cd0 msgr2=0x7f66540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.450+0000 7f6671a6f700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f66540a3cd0 0x7f66540a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f665c00ac50 tx=0x7f665c00a380 comp rx=0 tx=0).stop 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.450+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c108a50 msgr2=0x7f666c19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.450+0000 7f6671a6f700 1 --2- 172.21.15.66:0/1048964574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f666c108a50 0x7f666c19da50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f666400b970 tx=0x7f666400bc80 comp rx=0 tx=0).stop 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.451+0000 7f666bfff700 1 -- 172.21.15.66:0/1048964574 reap_dead start 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.451+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 shutdown_connections 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.451+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 >> 172.21.15.66:0/1048964574 conn(0x7f666c0788f0 msgr2=0x7f666c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.451+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 shutdown_connections 2024-04-21T09:06:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:43.451+0000 7f6671a6f700 1 -- 172.21.15.66:0/1048964574 wait complete. 2024-04-21T09:06:43.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:43 smithi186 ceph-mon[31505]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 150 B/s, 0 objects/s recovering 2024-04-21T09:06:43.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:43 smithi083 ceph-mon[31413]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 150 B/s, 0 objects/s recovering 2024-04-21T09:06:45.412 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:45 smithi066 ceph-mon[27953]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s; 153 B/s, 0 objects/s recovering 2024-04-21T09:06:45.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.519+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/3959584858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac107b20 msgr2=0x7fb4ac107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:45.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.519+0000 7fb4b1d1f700 1 --2- 172.21.15.66:0/3959584858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac107b20 0x7fb4ac107f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009990 tx=0x7fb49c009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/3959584858 shutdown_connections 2024-04-21T09:06:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 --2- 172.21.15.66:0/3959584858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac101dc0 0x7fb4ac10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 --2- 172.21.15.66:0/3959584858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac107b20 0x7fb4ac107f00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 --2- 172.21.15.66:0/3959584858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac101400 0x7fb4ac101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/3959584858 >> 172.21.15.66:0/3959584858 conn(0x7fb4ac076c60 msgr2=0x7fb4ac077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/3959584858 shutdown_connections 2024-04-21T09:06:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/3959584858 wait complete. 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.520+0000 7fb4b1d1f700 1 Processor -- start 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4b1d1f700 1 -- start start 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4b1d1f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac101400 0x7fb4ac1025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4b1d1f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac101dc0 0x7fb4ac102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4b1d1f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac107b20 0x7fb4ac103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4b1d1f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac112d10 con 0x7fb4ac101400 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4b1d1f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac112b90 con 0x7fb4ac107b20 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4b1d1f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac112e90 con 0x7fb4ac101dc0 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac101400 0x7fb4ac1025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac101400 0x7fb4ac1025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57924/0 (socket says 172.21.15.66:57924) 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 -- 172.21.15.66:0/2262556689 learned_addr learned my addr 172.21.15.66:0/2262556689 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4abfff700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac107b20 0x7fb4ac103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4aaffd700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac101dc0 0x7fb4ac102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 -- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac101dc0 msgr2=0x7fb4ac102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac101dc0 0x7fb4ac102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 -- 172.21.15.66:0/2262556689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac107b20 msgr2=0x7fb4ac103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac107b20 0x7fb4ac103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4ab7fe700 1 -- 172.21.15.66:0/2262556689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac1a77a0 con 0x7fb4ac101400 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.521+0000 7fb4abfff700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac107b20 0x7fb4ac103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.522+0000 7fb4ab7fe700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac101400 0x7fb4ac1025e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000c00 tx=0x7fb49c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.522+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c027070 con 0x7fb4ac101400 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.522+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c0041d0 con 0x7fb4ac101400 2024-04-21T09:06:45.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.522+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac1a7a30 con 0x7fb4ac101400 2024-04-21T09:06:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.522+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00f460 con 0x7fb4ac101400 2024-04-21T09:06:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.522+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac1a7e80 con 0x7fb4ac101400 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.523+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4ac0631d0 con 0x7fb4ac101400 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.524+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb49c02a070 con 0x7fb4ac101400 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.525+0000 7fb4a8ff9700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb4940a3cd0 0x7fb4940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.525+0000 7fb4aaffd700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb4940a3cd0 0x7fb4940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.525+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fb49c0d2070 con 0x7fb4ac101400 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.525+0000 7fb4a8ff9700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] conn(0x7fb4940aee30 0x7fb4940b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.525+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 --> [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb49c0043d0 con 0x7fb4940aee30 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.525+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fb49c0d24c0 con 0x7fb4ac101400 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.525+0000 7fb4aaffd700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb4940a3cd0 0x7fb4940a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006fd0 tx=0x7fb4a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.526+0000 7fb4abfff700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] conn(0x7fb4940aee30 0x7fb4940b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:45.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.526+0000 7fb4abfff700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] conn(0x7fb4940aee30 0x7fb4940b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:45.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.527+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== osd.8 v2:172.21.15.186:6816/2023460936 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb49c0043d0 con 0x7fb4940aee30 2024-04-21T09:06:45.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.543+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 --> [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4ac063650 con 0x7fb4940aee30 2024-04-21T09:06:45.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.544+0000 7fb4a8ff9700 1 -- 172.21.15.66:0/2262556689 <== osd.8 v2:172.21.15.186:6816/2023460936 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4ac063650 con 0x7fb4940aee30 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] conn(0x7fb4940aee30 msgr2=0x7fb4940b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] conn(0x7fb4940aee30 0x7fb4940b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb4940a3cd0 msgr2=0x7fb4940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb4940a3cd0 0x7fb4940a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006fd0 tx=0x7fb4a0008040 comp rx=0 tx=0).stop 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac101400 msgr2=0x7fb4ac1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 --2- 172.21.15.66:0/2262556689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ac101400 0x7fb4ac1025e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000c00 tx=0x7fb49c0047b0 comp rx=0 tx=0).stop 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4abfff700 1 -- 172.21.15.66:0/2262556689 reap_dead start 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 shutdown_connections 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 >> 172.21.15.66:0/2262556689 conn(0x7fb4ac076c60 msgr2=0x7fb4ac1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:45.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 shutdown_connections 2024-04-21T09:06:45.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:45.545+0000 7fb4b1d1f700 1 -- 172.21.15.66:0/2262556689 wait complete. 2024-04-21T09:06:45.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:45 smithi186 ceph-mon[31505]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s; 153 B/s, 0 objects/s recovering 2024-04-21T09:06:45.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:45 smithi083 ceph-mon[31413]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s; 153 B/s, 0 objects/s recovering 2024-04-21T09:06:46.097 INFO:teuthology.orchestra.run.smithi066.stdout:77309411398 2024-04-21T09:06:46.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-21T09:06:46.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:46 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:06:46.682 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:46 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:06:46.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:46 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:06:47.343 INFO:teuthology.orchestra.run.smithi066.stdout:146028888124 2024-04-21T09:06:47.344 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-21T09:06:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:47 smithi066 ceph-mon[27953]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 153 B/s, 0 objects/s recovering 2024-04-21T09:06:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:47 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:47.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:47 smithi186 ceph-mon[31505]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 153 B/s, 0 objects/s recovering 2024-04-21T09:06:47.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:47.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:47 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:47.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:47 smithi083 ceph-mon[31413]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 153 B/s, 0 objects/s recovering 2024-04-21T09:06:47.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:47.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:47 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:48.655 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:48 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:48.655 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:48 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.663+0000 7f8559c4a700 1 -- 172.21.15.66:0/1088688645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406b980 msgr2=0x7f855410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.663+0000 7f8559c4a700 1 --2- 172.21.15.66:0/1088688645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406b980 0x7f855410f8f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b0a0 tx=0x7f854c00b3b0 comp rx=0 tx=0).stop 2024-04-21T09:06:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.663+0000 7f8559c4a700 1 -- 172.21.15.66:0/1088688645 shutdown_connections 2024-04-21T09:06:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.663+0000 7f8559c4a700 1 --2- 172.21.15.66:0/1088688645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406b980 0x7f855410f8f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.663+0000 7f8559c4a700 1 --2- 172.21.15.66:0/1088688645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8554108a20 0x7f8554108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.663+0000 7f8559c4a700 1 --2- 172.21.15.66:0/1088688645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f855406afe0 0x7f855406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.663+0000 7f8559c4a700 1 -- 172.21.15.66:0/1088688645 >> 172.21.15.66:0/1088688645 conn(0x7f85540788b0 msgr2=0x7f8554078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.664+0000 7f8559c4a700 1 -- 172.21.15.66:0/1088688645 shutdown_connections 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.664+0000 7f8559c4a700 1 -- 172.21.15.66:0/1088688645 wait complete. 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.664+0000 7f8559c4a700 1 Processor -- start 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.664+0000 7f8559c4a700 1 -- start start 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.664+0000 7f8559c4a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406afe0 0x7f8554196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.664+0000 7f8559c4a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f855406b980 0x7f8554197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8559c4a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8554108a20 0x7f855419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8559c4a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8554110a50 con 0x7f855406afe0 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8559c4a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85541108d0 con 0x7f855406b980 2024-04-21T09:06:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8559c4a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8554110bd0 con 0x7f8554108a20 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406afe0 0x7f8554196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406afe0 0x7f8554196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57934/0 (socket says 172.21.15.66:57934) 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 -- 172.21.15.66:0/4003099585 learned_addr learned my addr 172.21.15.66:0/4003099585 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8553fff700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8554108a20 0x7f855419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8552ffd700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f855406b980 0x7f8554197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 -- 172.21.15.66:0/4003099585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8554108a20 msgr2=0x7f855419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8554108a20 0x7f855419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 -- 172.21.15.66:0/4003099585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f855406b980 msgr2=0x7f8554197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f855406b980 0x7f8554197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 -- 172.21.15.66:0/4003099585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855419bf50 con 0x7f855406afe0 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8553fff700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8554108a20 0x7f855419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f85537fe700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406afe0 0x7f8554196f40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9c0 tx=0x7f854400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400cbd0 con 0x7f855406afe0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854400cd70 con 0x7f855406afe0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855419c240 con 0x7f855406afe0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544010430 con 0x7f855406afe0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.665+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541a3ac0 con 0x7f855406afe0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.666+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8554063150 con 0x7f855406afe0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.668+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8544004750 con 0x7f855406afe0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.668+0000 7f8550ff9700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f853c0a3cd0 0x7f853c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.668+0000 7f8552ffd700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f853c0a3cd0 0x7f853c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.668+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f85440d1070 con 0x7f855406afe0 2024-04-21T09:06:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.668+0000 7f8550ff9700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] conn(0x7f853c0aee30 0x7f853c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.668+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 --> [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f853c0b19a0 con 0x7f853c0aee30 2024-04-21T09:06:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.668+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f85440d14c0 con 0x7f855406afe0 2024-04-21T09:06:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.669+0000 7f8553fff700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] conn(0x7f853c0aee30 0x7f853c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.669+0000 7f8552ffd700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f853c0a3cd0 0x7f853c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8554078e20 tx=0x7f8548011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.669+0000 7f8553fff700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] conn(0x7f853c0aee30 0x7f853c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.670+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== osd.6 v2:172.21.15.66:6818/455614173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f853c0b19a0 con 0x7f853c0aee30 2024-04-21T09:06:48.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.687+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 --> [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85540635d0 con 0x7f853c0aee30 2024-04-21T09:06:48.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.687+0000 7f8550ff9700 1 -- 172.21.15.66:0/4003099585 <== osd.6 v2:172.21.15.66:6818/455614173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f85540635d0 con 0x7f853c0aee30 2024-04-21T09:06:48.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.687+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] conn(0x7f853c0aee30 msgr2=0x7f853c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:48.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.687+0000 7f8559c4a700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] conn(0x7f853c0aee30 0x7f853c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:48.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f853c0a3cd0 msgr2=0x7f853c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:48.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f853c0a3cd0 0x7f853c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8554078e20 tx=0x7f8548011040 comp rx=0 tx=0).stop 2024-04-21T09:06:48.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406afe0 msgr2=0x7f8554196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:48.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 --2- 172.21.15.66:0/4003099585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f855406afe0 0x7f8554196f40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9c0 tx=0x7f854400ed80 comp rx=0 tx=0).stop 2024-04-21T09:06:48.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8553fff700 1 -- 172.21.15.66:0/4003099585 reap_dead start 2024-04-21T09:06:48.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 shutdown_connections 2024-04-21T09:06:48.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 >> 172.21.15.66:0/4003099585 conn(0x7f85540788b0 msgr2=0x7f85540ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:48.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 shutdown_connections 2024-04-21T09:06:48.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:48.688+0000 7f8559c4a700 1 -- 172.21.15.66:0/4003099585 wait complete. 2024-04-21T09:06:48.819 INFO:teuthology.orchestra.run.smithi066.stdout:292057776161 2024-04-21T09:06:48.820 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-21T09:06:48.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:48 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:48.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:48 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:48.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:48 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:48.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:48 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:06:49.207 INFO:teuthology.orchestra.run.smithi066.stdout:257698037800 2024-04-21T09:06:49.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-21T09:06:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:49 smithi066 ceph-mon[27953]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 134 B/s, 0 objects/s recovering 2024-04-21T09:06:49.738 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:49 smithi186 ceph-mon[31505]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 134 B/s, 0 objects/s recovering 2024-04-21T09:06:49.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:49 smithi083 ceph-mon[31413]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 134 B/s, 0 objects/s recovering 2024-04-21T09:06:50.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.055+0000 7f025cf9f700 1 -- 172.21.15.66:0/1850758223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 msgr2=0x7f0258101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.055+0000 7f025cf9f700 1 --2- 172.21.15.66:0/1850758223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 0x7f0258101de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f024c009990 tx=0x7f024c009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 -- 172.21.15.66:0/1850758223 shutdown_connections 2024-04-21T09:06:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 --2- 172.21.15.66:0/1850758223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0258107960 0x7f0258107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 --2- 172.21.15.66:0/1850758223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0258102320 0x7f025810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 --2- 172.21.15.66:0/1850758223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 0x7f0258101de0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 -- 172.21.15.66:0/1850758223 >> 172.21.15.66:0/1850758223 conn(0x7f02580789e0 msgr2=0x7f0258078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 -- 172.21.15.66:0/1850758223 shutdown_connections 2024-04-21T09:06:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 -- 172.21.15.66:0/1850758223 wait complete. 2024-04-21T09:06:50.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.056+0000 7f025cf9f700 1 Processor -- start 2024-04-21T09:06:50.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025cf9f700 1 -- start start 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025cf9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 0x7f0258199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025cf9f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0258102320 0x7f025819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025cf9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0258107960 0x7f02581a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025cf9f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0258111b70 con 0x7f0258101960 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025cf9f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02581119f0 con 0x7f0258107960 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025cf9f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0258111cf0 con 0x7f0258102320 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 0x7f0258199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 0x7f0258199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57974/0 (socket says 172.21.15.66:57974) 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 -- 172.21.15.66:0/1897970772 learned_addr learned my addr 172.21.15.66:0/1897970772 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f0255d9b700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0258102320 0x7f025819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f0256d9d700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0258107960 0x7f02581a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 -- 172.21.15.66:0/1897970772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0258102320 msgr2=0x7f025819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0258102320 0x7f025819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 -- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0258107960 msgr2=0x7f02581a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0258107960 0x7f02581a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.057+0000 7f025659c700 1 -- 172.21.15.66:0/1897970772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581a54f0 con 0x7f0258101960 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.058+0000 7f025659c700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 0x7f0258199f30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f024800c920 tx=0x7f024800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.058+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248007b00 con 0x7f0258101960 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.058+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0248004ce0 con 0x7f0258101960 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.058+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02581a5720 con 0x7f0258101960 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.058+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248005650 con 0x7f0258101960 2024-04-21T09:06:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.058+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581a5cb0 con 0x7f0258101960 2024-04-21T09:06:50.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.059+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0258063110 con 0x7f0258101960 2024-04-21T09:06:50.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.060+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0248004750 con 0x7f0258101960 2024-04-21T09:06:50.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.061+0000 7f02477fe700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f02400a3d50 0x7f02400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.061+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f02480d08b0 con 0x7f0258101960 2024-04-21T09:06:50.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.061+0000 7f0255d9b700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f02400a3d50 0x7f02400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.061+0000 7f02477fe700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] conn(0x7f02400aeeb0 0x7f02400b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.061+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 --> [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02400b1a20 con 0x7f02400aeeb0 2024-04-21T09:06:50.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.061+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f02480d0d00 con 0x7f0258101960 2024-04-21T09:06:50.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.062+0000 7f0256d9d700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] conn(0x7f02400aeeb0 0x7f02400b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.062+0000 7f0255d9b700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f02400a3d50 0x7f02400a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f024c006010 tx=0x7f024c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:50.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.062+0000 7f0256d9d700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] conn(0x7f02400aeeb0 0x7f02400b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:50.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.063+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== osd.1 v2:172.21.15.83:6800/2284873950 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f02400b1a20 con 0x7f02400aeeb0 2024-04-21T09:06:50.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.079+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 --> [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0258063510 con 0x7f02400aeeb0 2024-04-21T09:06:50.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f02477fe700 1 -- 172.21.15.66:0/1897970772 <== osd.1 v2:172.21.15.83:6800/2284873950 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0258063510 con 0x7f02400aeeb0 2024-04-21T09:06:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] conn(0x7f02400aeeb0 msgr2=0x7f02400b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] conn(0x7f02400aeeb0 0x7f02400b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f02400a3d50 msgr2=0x7f02400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f02400a3d50 0x7f02400a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f024c006010 tx=0x7f024c01a040 comp rx=0 tx=0).stop 2024-04-21T09:06:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 msgr2=0x7f0258199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 --2- 172.21.15.66:0/1897970772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0258101960 0x7f0258199f30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f024800c920 tx=0x7f024800cce0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f0256d9d700 1 -- 172.21.15.66:0/1897970772 reap_dead start 2024-04-21T09:06:50.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 shutdown_connections 2024-04-21T09:06:50.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 >> 172.21.15.66:0/1897970772 conn(0x7f02580789e0 msgr2=0x7f02581044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:50.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 shutdown_connections 2024-04-21T09:06:50.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:50.080+0000 7f025cf9f700 1 -- 172.21.15.66:0/1897970772 wait complete. 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.734+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/3249382509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81019d0 msgr2=0x7fa3d810f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.734+0000 7fa3dc8f5700 1 --2- 172.21.15.66:0/3249382509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81019d0 0x7fa3d810f910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000b0a0 tx=0x7fa3d000b3b0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.734+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/3249382509 shutdown_connections 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.734+0000 7fa3dc8f5700 1 --2- 172.21.15.66:0/3249382509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d8108ad0 0x7fa3d8108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.734+0000 7fa3dc8f5700 1 --2- 172.21.15.66:0/3249382509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81019d0 0x7fa3d810f910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.734+0000 7fa3dc8f5700 1 --2- 172.21.15.66:0/3249382509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d8101030 0x7fa3d8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.735+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/3249382509 >> 172.21.15.66:0/3249382509 conn(0x7fa3d8077fb0 msgr2=0x7fa3d80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:50.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.735+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/3249382509 shutdown_connections 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.735+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/3249382509 wait complete. 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.735+0000 7fa3dc8f5700 1 Processor -- start 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.735+0000 7fa3dc8f5700 1 -- start start 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8101030 0x7fa3d8199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d81019d0 0x7fa3d81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d8108ad0 0x7fa3d819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8112c20 con 0x7fa3d8101030 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8112aa0 con 0x7fa3d8108ad0 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8112da0 con 0x7fa3d81019d0 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8101030 0x7fa3d8199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8101030 0x7fa3d8199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57950/0 (socket says 172.21.15.66:57950) 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 -- 172.21.15.66:0/1198650868 learned_addr learned my addr 172.21.15.66:0/1198650868 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d5d9b700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d81019d0 0x7fa3d81995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 -- 172.21.15.66:0/1198650868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d81019d0 msgr2=0x7fa3d81995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d81019d0 0x7fa3d81995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 -- 172.21.15.66:0/1198650868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d8108ad0 msgr2=0x7fa3d819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d6d9d700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d8108ad0 0x7fa3d819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d8108ad0 0x7fa3d819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 -- 172.21.15.66:0/1198650868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d819e0a0 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d5d9b700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d81019d0 0x7fa3d81995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3d659c700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8101030 0x7fa3d8199090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800d950 tx=0x7fa3c800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8016070 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c8010460 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d819e390 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800f620 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.736+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81a5c10 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.737+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3d80631d0 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.738+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa3c8019070 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.739+0000 7fa3c77fe700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa3c00a3cd0 0x7fa3c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.739+0000 7fa3d5d9b700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa3c00a3cd0 0x7fa3c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.739+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fa3c80d09f0 con 0x7fa3d8101030 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.739+0000 7fa3c77fe700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] conn(0x7fa3c00aee30 0x7fa3c00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:50.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.739+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 --> [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3c00b19a0 con 0x7fa3c00aee30 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.739+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fa3c80d0e40 con 0x7fa3d8101030 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.740+0000 7fa3d6d9d700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] conn(0x7fa3c00aee30 0x7fa3c00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.740+0000 7fa3d5d9b700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa3c00a3cd0 0x7fa3c00a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80ff060 tx=0x7fa3cc00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.740+0000 7fa3d6d9d700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] conn(0x7fa3c00aee30 0x7fa3c00b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.741+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== osd.3 v2:172.21.15.66:6810/4132171023 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3c00b19a0 con 0x7fa3c00aee30 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.757+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 --> [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3d8063650 con 0x7fa3c00aee30 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.757+0000 7fa3c77fe700 1 -- 172.21.15.66:0/1198650868 <== osd.3 v2:172.21.15.66:6810/4132171023 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa3d8063650 con 0x7fa3c00aee30 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.757+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] conn(0x7fa3c00aee30 msgr2=0x7fa3c00b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.757+0000 7fa3dc8f5700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] conn(0x7fa3c00aee30 0x7fa3c00b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa3c00a3cd0 msgr2=0x7fa3c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa3c00a3cd0 0x7fa3c00a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80ff060 tx=0x7fa3cc00b2c0 comp rx=0 tx=0).stop 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8101030 msgr2=0x7fa3d8199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:50.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 --2- 172.21.15.66:0/1198650868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8101030 0x7fa3d8199090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800d950 tx=0x7fa3c800dd10 comp rx=0 tx=0).stop 2024-04-21T09:06:50.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3d6d9d700 1 -- 172.21.15.66:0/1198650868 reap_dead start 2024-04-21T09:06:50.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 shutdown_connections 2024-04-21T09:06:50.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 >> 172.21.15.66:0/1198650868 conn(0x7fa3d8077fb0 msgr2=0x7fa3d80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:50.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 shutdown_connections 2024-04-21T09:06:50.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:49.758+0000 7fa3dc8f5700 1 -- 172.21.15.66:0/1198650868 wait complete. 2024-04-21T09:06:51.614 INFO:teuthology.orchestra.run.smithi066.stdout:201863462963 2024-04-21T09:06:51.614 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-21T09:06:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:51 smithi186 ceph-mon[31505]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 120 B/s, 0 objects/s recovering 2024-04-21T09:06:51.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:51 smithi083 ceph-mon[31413]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 120 B/s, 0 objects/s recovering 2024-04-21T09:06:51.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:51 smithi066 ceph-mon[27953]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 120 B/s, 0 objects/s recovering 2024-04-21T09:06:52.258 INFO:teuthology.orchestra.run.smithi066.stdout:223338299438 2024-04-21T09:06:52.258 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-21T09:06:53.424 INFO:teuthology.orchestra.run.smithi066.stdout:55834574925 2024-04-21T09:06:53.424 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-21T09:06:53.819 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:53 smithi066 ceph-mon[27953]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 102 B/s, 0 objects/s recovering 2024-04-21T09:06:53.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:53 smithi186 ceph-mon[31505]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 102 B/s, 0 objects/s recovering 2024-04-21T09:06:53.845 INFO:teuthology.orchestra.run.smithi066.stdout:111669149762 2024-04-21T09:06:53.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-21T09:06:53.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:53 smithi083 ceph-mon[31413]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 102 B/s, 0 objects/s recovering 2024-04-21T09:06:53.967 INFO:teuthology.orchestra.run.smithi066.stdout:34359738449 2024-04-21T09:06:53.968 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-21T09:06:54.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 -- 172.21.15.66:0/862282801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c108a70 msgr2=0x7fa92c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:54.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 --2- 172.21.15.66:0/862282801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c108a70 0x7fa92c108e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa928009990 tx=0x7fa928009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 -- 172.21.15.66:0/862282801 shutdown_connections 2024-04-21T09:06:54.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 --2- 172.21.15.66:0/862282801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92c103480 0x7fa92c10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 --2- 172.21.15.66:0/862282801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c108a70 0x7fa92c108e50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 --2- 172.21.15.66:0/862282801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa92c102ac0 0x7fa92c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 -- 172.21.15.66:0/862282801 >> 172.21.15.66:0/862282801 conn(0x7fa92c079bb0 msgr2=0x7fa92c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 -- 172.21.15.66:0/862282801 shutdown_connections 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 -- 172.21.15.66:0/862282801 wait complete. 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.767+0000 7fa933ca5700 1 Processor -- start 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa933ca5700 1 -- start start 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa933ca5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c102ac0 0x7fa92c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa933ca5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92c103480 0x7fa92c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa933ca5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa92c108a70 0x7fa92c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa933ca5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c112c00 con 0x7fa92c102ac0 2024-04-21T09:06:54.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa933ca5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c112a80 con 0x7fa92c103480 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa933ca5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c112d80 con 0x7fa92c108a70 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c102ac0 0x7fa92c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c102ac0 0x7fa92c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48690/0 (socket says 172.21.15.66:48690) 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 -- 172.21.15.66:0/202649900 learned_addr learned my addr 172.21.15.66:0/202649900 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa932242700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa92c108a70 0x7fa92c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931240700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92c103480 0x7fa92c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 -- 172.21.15.66:0/202649900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa92c108a70 msgr2=0x7fa92c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa92c108a70 0x7fa92c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 -- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92c103480 msgr2=0x7fa92c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92c103480 0x7fa92c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 -- 172.21.15.66:0/202649900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c1a75b0 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931240700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92c103480 0x7fa92c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.768+0000 7fa931a41700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c102ac0 0x7fa92c074d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa928000c00 tx=0x7fa9280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.769+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928027070 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.769+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa928003f20 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.769+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928022cf0 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.769+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c1a7840 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.769+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c1a7cd0 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.769+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa92c063150 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.771+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa92802a070 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.771+0000 7fa922ffd700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9180a3d50 0x7fa9180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.771+0000 7fa931240700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9180a3d50 0x7fa9180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.771+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fa9280d1c60 con 0x7fa92c102ac0 2024-04-21T09:06:54.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.772+0000 7fa922ffd700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] conn(0x7fa9180aeeb0 0x7fa9180b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.772+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 --> [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9280095d0 con 0x7fa9180aeeb0 2024-04-21T09:06:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.772+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fa9280d20b0 con 0x7fa92c102ac0 2024-04-21T09:06:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.772+0000 7fa931240700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9180a3d50 0x7fa9180a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005fd0 tx=0x7fa91c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.772+0000 7fa932242700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] conn(0x7fa9180aeeb0 0x7fa9180b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.773+0000 7fa932242700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] conn(0x7fa9180aeeb0 0x7fa9180b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:54.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.774+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== osd.10 v2:172.21.15.83:6824/2551875994 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9280095d0 con 0x7fa9180aeeb0 2024-04-21T09:06:54.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 --> [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa92c063620 con 0x7fa9180aeeb0 2024-04-21T09:06:54.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa922ffd700 1 -- 172.21.15.66:0/202649900 <== osd.10 v2:172.21.15.83:6824/2551875994 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa92c063620 con 0x7fa9180aeeb0 2024-04-21T09:06:54.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] conn(0x7fa9180aeeb0 msgr2=0x7fa9180b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:54.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] conn(0x7fa9180aeeb0 0x7fa9180b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9180a3d50 msgr2=0x7fa9180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9180a3d50 0x7fa9180a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005fd0 tx=0x7fa91c005e20 comp rx=0 tx=0).stop 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c102ac0 msgr2=0x7fa92c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 --2- 172.21.15.66:0/202649900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa92c102ac0 0x7fa92c074d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa928000c00 tx=0x7fa9280047b0 comp rx=0 tx=0).stop 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa932242700 1 -- 172.21.15.66:0/202649900 reap_dead start 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 shutdown_connections 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.794+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 >> 172.21.15.66:0/202649900 conn(0x7fa92c079bb0 msgr2=0x7fa92c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:54.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.795+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 shutdown_connections 2024-04-21T09:06:54.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:54.795+0000 7fa933ca5700 1 -- 172.21.15.66:0/202649900 wait complete. 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 -- 172.21.15.66:0/3166781646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 msgr2=0x7f3abc107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 --2- 172.21.15.66:0/3166781646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 0x7f3abc107d70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009990 tx=0x7f3aac009ca0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 -- 172.21.15.66:0/3166781646 shutdown_connections 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 --2- 172.21.15.66:0/3166781646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc102300 0x7f3abc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 --2- 172.21.15.66:0/3166781646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 0x7f3abc107d70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 --2- 172.21.15.66:0/3166781646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3abc101940 0x7f3abc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 -- 172.21.15.66:0/3166781646 >> 172.21.15.66:0/3166781646 conn(0x7f3abc0788f0 msgr2=0x7f3abc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 -- 172.21.15.66:0/3166781646 shutdown_connections 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 -- 172.21.15.66:0/3166781646 wait complete. 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.200+0000 7f3ac248c700 1 Processor -- start 2024-04-21T09:06:55.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac248c700 1 -- start start 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac248c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3abc101940 0x7f3abc199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac248c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc102300 0x7f3abc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac248c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 0x7f3abc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac248c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc111b60 con 0x7f3abc107990 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac248c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc1119e0 con 0x7f3abc101940 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac248c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc111ce0 con 0x7f3abc102300 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 0x7f3abc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 0x7f3abc19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48712/0 (socket says 172.21.15.66:48712) 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 -- 172.21.15.66:0/1889536060 learned_addr learned my addr 172.21.15.66:0/1889536060 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3abbfff700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3abc101940 0x7f3abc199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3abb7fe700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc102300 0x7f3abc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 -- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc102300 msgr2=0x7f3abc1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:55.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc102300 0x7f3abc1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 -- 172.21.15.66:0/1889536060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3abc101940 msgr2=0x7f3abc199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:55.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3abc101940 0x7f3abc199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3ac0a29700 1 -- 172.21.15.66:0/1889536060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc19e170 con 0x7f3abc107990 2024-04-21T09:06:55.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.201+0000 7f3abb7fe700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc102300 0x7f3abc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.202+0000 7f3ac0a29700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 0x7f3abc19da30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4007ee0 tx=0x7f3ab400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.202+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab400dbd0 con 0x7f3abc107990 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.202+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab400dd70 con 0x7f3abc107990 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.202+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4005990 con 0x7f3abc107990 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.202+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc19e460 con 0x7f3abc107990 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.202+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc1a5ce0 con 0x7f3abc107990 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.203+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3abc106950 con 0x7f3abc107990 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.204+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3ab401a020 con 0x7f3abc107990 2024-04-21T09:06:55.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.205+0000 7f3ab97fa700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3aa40a3cd0 0x7f3aa40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:55.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.205+0000 7f3abbfff700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3aa40a3cd0 0x7f3aa40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:55.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.205+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f3ab40d0880 con 0x7f3abc107990 2024-04-21T09:06:55.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.205+0000 7f3ab97fa700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] conn(0x7f3aa40aee30 0x7f3aa40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:06:55.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.205+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 --> [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3aa40b19a0 con 0x7f3aa40aee30 2024-04-21T09:06:55.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.205+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f3ab40d0cd0 con 0x7f3abc107990 2024-04-21T09:06:55.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.205+0000 7f3abbfff700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3aa40a3cd0 0x7f3aa40a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:55.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.206+0000 7f3abb7fe700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] conn(0x7f3aa40aee30 0x7f3aa40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:06:55.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.206+0000 7f3abb7fe700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] conn(0x7f3aa40aee30 0x7f3aa40b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:06:55.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.207+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== osd.5 v2:172.21.15.186:6808/3723805891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3aa40b19a0 con 0x7f3aa40aee30 2024-04-21T09:06:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.228+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 --> [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3abc103060 con 0x7f3aa40aee30 2024-04-21T09:06:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.228+0000 7f3ab97fa700 1 -- 172.21.15.66:0/1889536060 <== osd.5 v2:172.21.15.186:6808/3723805891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3abc103060 con 0x7f3aa40aee30 2024-04-21T09:06:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] conn(0x7f3aa40aee30 msgr2=0x7f3aa40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] conn(0x7f3aa40aee30 0x7f3aa40b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3aa40a3cd0 msgr2=0x7f3aa40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f3aa40a3cd0 0x7f3aa40a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac01a040 comp rx=0 tx=0).stop 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 msgr2=0x7f3abc19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 --2- 172.21.15.66:0/1889536060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3abc107990 0x7f3abc19da30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4007ee0 tx=0x7f3ab400d3b0 comp rx=0 tx=0).stop 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac0a29700 1 -- 172.21.15.66:0/1889536060 reap_dead start 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 shutdown_connections 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 >> 172.21.15.66:0/1889536060 conn(0x7f3abc0788f0 msgr2=0x7f3abc06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.229+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 shutdown_connections 2024-04-21T09:06:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:06:55.230+0000 7f3ac248c700 1 -- 172.21.15.66:0/1889536060 wait complete. 2024-04-21T09:06:55.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:55 smithi186 ceph-mon[31505]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-21T09:06:55.841 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:55 smithi066 ceph-mon[27953]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-21T09:06:55.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:55 smithi083 ceph-mon[31413]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-21T09:06:55.870 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:55.870 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:55.872 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:55.872 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:55.876 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:55.878 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:55.889 INFO:teuthology.orchestra.run.smithi066.stdout:167503724602 2024-04-21T09:06:55.890 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-21T09:06:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:56.329 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:56.330 INFO:teuthology.orchestra.run.smithi066.stdout:313532612639 2024-04-21T09:06:56.331 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-21T09:06:56.995 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:06:57.683 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:57 smithi066 ceph-mon[27953]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:06:57.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:57 smithi186 ceph-mon[31505]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:06:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:57 smithi083 ceph-mon[31413]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:06:59.531 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:06:59 smithi066 ceph-mon[27953]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:06:59.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:06:59 smithi186 ceph-mon[31505]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:06:59.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:06:59 smithi083 ceph-mon[31413]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:01.544 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:01 smithi066 ceph-mon[27953]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:01 smithi186 ceph-mon[31505]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:01.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:01 smithi083 ceph-mon[31413]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:02.721 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:02.722 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:03.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:03 smithi066 ceph-mon[27953]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-21T09:07:03.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 -- 172.21.15.66:0/3495053297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90074de0 msgr2=0x7f9a900751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:03.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 --2- 172.21.15.66:0/3495053297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90074de0 0x7f9a900751c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009990 tx=0x7f9a8c009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 -- 172.21.15.66:0/3495053297 shutdown_connections 2024-04-21T09:07:03.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 --2- 172.21.15.66:0/3495053297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90113970 0x7f9a90115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 --2- 172.21.15.66:0/3495053297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90074de0 0x7f9a900751c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 --2- 172.21.15.66:0/3495053297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a90075700 0x7f9a90113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 -- 172.21.15.66:0/3495053297 >> 172.21.15.66:0/3495053297 conn(0x7f9a900fe450 msgr2=0x7f9a90100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 -- 172.21.15.66:0/3495053297 shutdown_connections 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 -- 172.21.15.66:0/3495053297 wait complete. 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 Processor -- start 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.701+0000 7f9a971a1700 1 -- start start 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a971a1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a90074de0 0x7f9a901a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a971a1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90075700 0x7f9a901a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a971a1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90113970 0x7f9a9019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a971a1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a90119230 con 0x7f9a90075700 2024-04-21T09:07:03.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a971a1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a901190b0 con 0x7f9a90113970 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a971a1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a901193b0 con 0x7f9a90074de0 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90075700 0x7f9a901a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90075700 0x7f9a901a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48730/0 (socket says 172.21.15.66:48730) 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 -- 172.21.15.66:0/551558875 learned_addr learned my addr 172.21.15.66:0/551558875 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9619f700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a90074de0 0x7f9a901a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a969a0700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90113970 0x7f9a9019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 -- 172.21.15.66:0/551558875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a90074de0 msgr2=0x7f9a901a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a90074de0 0x7f9a901a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 -- 172.21.15.66:0/551558875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90113970 msgr2=0x7f9a9019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90113970 0x7f9a9019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 -- 172.21.15.66:0/551558875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9019fd30 con 0x7f9a90075700 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.702+0000 7f9a9599e700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90075700 0x7f9a901a5950 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000b530 tx=0x7f9a8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.703+0000 7f9a877fe700 1 -- 172.21.15.66:0/551558875 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80015020 con 0x7f9a90075700 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.703+0000 7f9a877fe700 1 -- 172.21.15.66:0/551558875 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a80011ce0 con 0x7f9a90075700 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.703+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a901a0020 con 0x7f9a90075700 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.703+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a901ac2e0 con 0x7f9a90075700 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.703+0000 7f9a877fe700 1 -- 172.21.15.66:0/551558875 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80010e40 con 0x7f9a90075700 2024-04-21T09:07:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.704+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a74005330 con 0x7f9a90075700 2024-04-21T09:07:03.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.705+0000 7f9a877fe700 1 -- 172.21.15.66:0/551558875 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9a80018020 con 0x7f9a90075700 2024-04-21T09:07:03.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.706+0000 7f9a877fe700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9a7c0a3cd0 0x7f9a7c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:03.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.706+0000 7f9a9619f700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9a7c0a3cd0 0x7f9a7c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:03.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.706+0000 7f9a877fe700 1 -- 172.21.15.66:0/551558875 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f9a800d0470 con 0x7f9a90075700 2024-04-21T09:07:03.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.707+0000 7f9a9619f700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9a7c0a3cd0 0x7f9a7c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00b5c0 tx=0x7f9a8c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:03.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.710+0000 7f9a877fe700 1 -- 172.21.15.66:0/551558875 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a8001f020 con 0x7f9a90075700 2024-04-21T09:07:03.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:03 smithi186 ceph-mon[31505]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-21T09:07:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:03 smithi083 ceph-mon[31413]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-21T09:07:03.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.923+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9a740058f0 con 0x7f9a90075700 2024-04-21T09:07:03.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.923+0000 7f9a877fe700 1 -- 172.21.15.66:0/551558875 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9a8009caa0 con 0x7f9a90075700 2024-04-21T09:07:03.923 INFO:teuthology.orchestra.run.smithi066.stdout:257698037803 2024-04-21T09:07:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9a7c0a3cd0 msgr2=0x7f9a7c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9a7c0a3cd0 0x7f9a7c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00b5c0 tx=0x7f9a8c005960 comp rx=0 tx=0).stop 2024-04-21T09:07:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90075700 msgr2=0x7f9a901a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90075700 0x7f9a901a5950 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000b530 tx=0x7f9a8000b840 comp rx=0 tx=0).stop 2024-04-21T09:07:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 shutdown_connections 2024-04-21T09:07:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9a7c0a3cd0 0x7f9a7c0a6190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90113970 0x7f9a9019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a90074de0 0x7f9a901a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 --2- 172.21.15.66:0/551558875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a90075700 0x7f9a901a5950 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 >> 172.21.15.66:0/551558875 conn(0x7f9a900fe450 msgr2=0x7f9a900ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 shutdown_connections 2024-04-21T09:07:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:03.925+0000 7f9a971a1700 1 -- 172.21.15.66:0/551558875 wait complete. 2024-04-21T09:07:04.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:04 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/551558875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T09:07:04.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:04 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/551558875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T09:07:04.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:04 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/551558875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T09:07:05.768 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:05 smithi066 ceph-mon[27953]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-21T09:07:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:05 smithi186 ceph-mon[31505]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-21T09:07:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:05 smithi083 ceph-mon[31413]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-21T09:07:06.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 -- 172.21.15.66:0/3496478925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 msgr2=0x7f1e0010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:06.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/3496478925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e0010e7f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b0a0 tx=0x7f1df800b3b0 comp rx=0 tx=0).stop 2024-04-21T09:07:06.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 -- 172.21.15.66:0/3496478925 shutdown_connections 2024-04-21T09:07:06.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/3496478925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e0010e7f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:06.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/3496478925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00107990 0x7f1e00107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:06.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/3496478925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e00101940 0x7f1e00101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 -- 172.21.15.66:0/3496478925 >> 172.21.15.66:0/3496478925 conn(0x7f1e000788f0 msgr2=0x7f1e00078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 -- 172.21.15.66:0/3496478925 shutdown_connections 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 -- 172.21.15.66:0/3496478925 wait complete. 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.900+0000 7f1e05b7c700 1 Processor -- start 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1e05b7c700 1 -- start start 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1e05b7c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00101940 0x7f1e001034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1e05b7c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e00103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1e05b7c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e00107990 0x7f1e00103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1e05b7c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00111b30 con 0x7f1e00102300 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1e05b7c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001119b0 con 0x7f1e00107990 2024-04-21T09:07:06.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1e05b7c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00111cb0 con 0x7f1e00101940 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e00103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00101940 0x7f1e001034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e00103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50622/0 (socket says 172.21.15.66:50622) 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 -- 172.21.15.66:0/1300311127 learned_addr learned my addr 172.21.15.66:0/1300311127 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfffff700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e00107990 0x7f1e00103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 -- 172.21.15.66:0/1300311127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00101940 msgr2=0x7f1e001034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00101940 0x7f1e001034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 -- 172.21.15.66:0/1300311127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e00107990 msgr2=0x7f1e00103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e00107990 0x7f1e00103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 -- 172.21.15.66:0/1300311127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e001a7660 con 0x7f1e00102300 2024-04-21T09:07:06.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.901+0000 7f1dfeffd700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e00103a30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c920 tx=0x7f1df400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:06.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.902+0000 7f1dfcff9700 1 -- 172.21.15.66:0/1300311127 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4004a30 con 0x7f1e00102300 2024-04-21T09:07:06.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.902+0000 7f1dfcff9700 1 -- 172.21.15.66:0/1300311127 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df40054d0 con 0x7f1e00102300 2024-04-21T09:07:06.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.902+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e001a7950 con 0x7f1e00102300 2024-04-21T09:07:06.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.902+0000 7f1dfcff9700 1 -- 172.21.15.66:0/1300311127 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4007c70 con 0x7f1e00102300 2024-04-21T09:07:06.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.902+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001a7e90 con 0x7f1e00102300 2024-04-21T09:07:06.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.903+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e00069730 con 0x7f1e00102300 2024-04-21T09:07:06.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.904+0000 7f1dfcff9700 1 -- 172.21.15.66:0/1300311127 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1df4004c60 con 0x7f1e00102300 2024-04-21T09:07:06.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.905+0000 7f1dfcff9700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1de80a3cd0 0x7f1de80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:06.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.905+0000 7f1dff7fe700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1de80a3cd0 0x7f1de80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:06.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.905+0000 7f1dfcff9700 1 -- 172.21.15.66:0/1300311127 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f1df40d0740 con 0x7f1e00102300 2024-04-21T09:07:06.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.905+0000 7f1dff7fe700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1de80a3cd0 0x7f1de80a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1df0005950 tx=0x7f1df000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:06.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:06.909+0000 7f1dfcff9700 1 -- 172.21.15.66:0/1300311127 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1df4014030 con 0x7f1e00102300 2024-04-21T09:07:07.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.124+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1e0002df80 con 0x7f1e00102300 2024-04-21T09:07:07.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.124+0000 7f1dfcff9700 1 -- 172.21.15.66:0/1300311127 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1df409cc60 con 0x7f1e00102300 2024-04-21T09:07:07.124 INFO:teuthology.orchestra.run.smithi066.stdout:77309411404 2024-04-21T09:07:07.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.126+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1de80a3cd0 msgr2=0x7f1de80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:07.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.126+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1de80a3cd0 0x7f1de80a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1df0005950 tx=0x7f1df000b410 comp rx=0 tx=0).stop 2024-04-21T09:07:07.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.126+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 msgr2=0x7f1e00103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.126+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e00103a30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c920 tx=0x7f1df400cc30 comp rx=0 tx=0).stop 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 shutdown_connections 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1de80a3cd0 0x7f1de80a6190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e00107990 0x7f1e00103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00101940 0x7f1e001034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 --2- 172.21.15.66:0/1300311127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00102300 0x7f1e00103a30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 >> 172.21.15.66:0/1300311127 conn(0x7f1e000788f0 msgr2=0x7f1e0006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 shutdown_connections 2024-04-21T09:07:07.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:07.127+0000 7f1e05b7c700 1 -- 172.21.15.66:0/1300311127 wait complete. 2024-04-21T09:07:07.770 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:07 smithi066 ceph-mon[27953]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:07.770 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:07 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1300311127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T09:07:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:07 smithi186 ceph-mon[31505]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:07.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:07 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1300311127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T09:07:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:07 smithi083 ceph-mon[31413]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:07 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1300311127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T09:07:09.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037803 for osd.8 2024-04-21T09:07:09.546 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:09.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:09 smithi066 ceph-mon[27953]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:09.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 -- 172.21.15.66:0/2600501328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc108a20 msgr2=0x7f65dc108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 --2- 172.21.15.66:0/2600501328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc108a20 0x7f65dc108e00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b0a0 tx=0x7f65cc00b3b0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 -- 172.21.15.66:0/2600501328 shutdown_connections 2024-04-21T09:07:09.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 --2- 172.21.15.66:0/2600501328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc06b980 0x7f65dc10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 --2- 172.21.15.66:0/2600501328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc108a20 0x7f65dc108e00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 --2- 172.21.15.66:0/2600501328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc06afe0 0x7f65dc06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 -- 172.21.15.66:0/2600501328 >> 172.21.15.66:0/2600501328 conn(0x7f65dc0788b0 msgr2=0x7f65dc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:09.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 -- 172.21.15.66:0/2600501328 shutdown_connections 2024-04-21T09:07:09.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.760+0000 7f65e08a6700 1 -- 172.21.15.66:0/2600501328 wait complete. 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 Processor -- start 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 -- start start 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc06afe0 0x7f65dc199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc06b980 0x7f65dc1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc108a20 0x7f65dc19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112ba0 con 0x7f65dc06afe0 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112a20 con 0x7f65dc06b980 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.761+0000 7f65e08a6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112d20 con 0x7f65dc108a20 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc06afe0 0x7f65dc199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc06afe0 0x7f65dc199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50638/0 (socket says 172.21.15.66:50638) 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 -- 172.21.15.66:0/1026436958 learned_addr learned my addr 172.21.15.66:0/1026436958 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:09.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65dad9d700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc108a20 0x7f65dc19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65d9d9b700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc06b980 0x7f65dc1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 -- 172.21.15.66:0/1026436958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc108a20 msgr2=0x7f65dc19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc108a20 0x7f65dc19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 -- 172.21.15.66:0/1026436958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc06b980 msgr2=0x7f65dc1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc06b980 0x7f65dc1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 -- 172.21.15.66:0/1026436958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc19e080 con 0x7f65dc06afe0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65da59c700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc06afe0 0x7f65dc199070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b070 tx=0x7f65cc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65d37fe700 1 -- 172.21.15.66:0/1026436958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc021070 con 0x7f65dc06afe0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc19e310 con 0x7f65dc06afe0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65d37fe700 1 -- 172.21.15.66:0/1026436958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65cc007e30 con 0x7f65dc06afe0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65d37fe700 1 -- 172.21.15.66:0/1026436958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc004840 con 0x7f65dc06afe0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.762+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a5bf0 con 0x7f65dc06afe0 2024-04-21T09:07:09.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.763+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc069730 con 0x7f65dc06afe0 2024-04-21T09:07:09.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.764+0000 7f65d37fe700 1 -- 172.21.15.66:0/1026436958 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f65cc007980 con 0x7f65dc06afe0 2024-04-21T09:07:09.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.765+0000 7f65d37fe700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f65c80a3cd0 0x7f65c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.765+0000 7f65d9d9b700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f65c80a3cd0 0x7f65c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.765+0000 7f65d37fe700 1 -- 172.21.15.66:0/1026436958 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f65cc00f020 con 0x7f65dc06afe0 2024-04-21T09:07:09.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.766+0000 7f65d9d9b700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f65c80a3cd0 0x7f65c80a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005ea0 tx=0x7f65c4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:09.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.769+0000 7f65d37fe700 1 -- 172.21.15.66:0/1026436958 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65cc02a020 con 0x7f65dc06afe0 2024-04-21T09:07:09.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:09 smithi186 ceph-mon[31505]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:09 smithi083 ceph-mon[31413]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 -- 172.21.15.66:0/4087924242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 msgr2=0x7f791c10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 --2- 172.21.15.66:0/4087924242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c10e7e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f791400b0a0 tx=0x7f791400b3b0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 -- 172.21.15.66:0/4087924242 shutdown_connections 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 --2- 172.21.15.66:0/4087924242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f791c107980 0x7f791c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 --2- 172.21.15.66:0/4087924242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c10e7e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 --2- 172.21.15.66:0/4087924242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f791c101930 0x7f791c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 -- 172.21.15.66:0/4087924242 >> 172.21.15.66:0/4087924242 conn(0x7f791c0788b0 msgr2=0x7f791c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 -- 172.21.15.66:0/4087924242 shutdown_connections 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 -- 172.21.15.66:0/4087924242 wait complete. 2024-04-21T09:07:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.965+0000 7f7923615700 1 Processor -- start 2024-04-21T09:07:09.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7923615700 1 -- start start 2024-04-21T09:07:09.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7923615700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f791c101930 0x7f791c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7923615700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7923615700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f791c107980 0x7f791c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7923615700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f791c111ad0 con 0x7f791c1022f0 2024-04-21T09:07:09.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7923615700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f791c111950 con 0x7f791c101930 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7923615700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f791c111c50 con 0x7f791c107980 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50658/0 (socket says 172.21.15.66:50658) 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 -- 172.21.15.66:0/4070795402 learned_addr learned my addr 172.21.15.66:0/4070795402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7921bb2700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f791c107980 0x7f791c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f79213b1700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f791c101930 0x7f791c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 -- 172.21.15.66:0/4070795402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f791c107980 msgr2=0x7f791c19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f791c107980 0x7f791c19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 -- 172.21.15.66:0/4070795402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f791c101930 msgr2=0x7f791c199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f791c101930 0x7f791c199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.966+0000 7f7920bb0700 1 -- 172.21.15.66:0/4070795402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c19e050 con 0x7f791c1022f0 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.967+0000 7f7920bb0700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c199580 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f790c00c920 tx=0x7f790c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.967+0000 7f79127fc700 1 -- 172.21.15.66:0/4070795402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c004a30 con 0x7f791c1022f0 2024-04-21T09:07:09.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.967+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c19e340 con 0x7f791c1022f0 2024-04-21T09:07:09.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.967+0000 7f79127fc700 1 -- 172.21.15.66:0/4070795402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790c0054d0 con 0x7f791c1022f0 2024-04-21T09:07:09.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.967+0000 7f79127fc700 1 -- 172.21.15.66:0/4070795402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c007bd0 con 0x7f791c1022f0 2024-04-21T09:07:09.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.967+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1a5bc0 con 0x7f791c1022f0 2024-04-21T09:07:09.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.968+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c069730 con 0x7f791c1022f0 2024-04-21T09:07:09.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.969+0000 7f79127fc700 1 -- 172.21.15.66:0/4070795402 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f790c004bd0 con 0x7f791c1022f0 2024-04-21T09:07:09.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.970+0000 7f79127fc700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f79080a3cd0 0x7f79080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:09.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.970+0000 7f79213b1700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f79080a3cd0 0x7f79080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:09.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.970+0000 7f79127fc700 1 -- 172.21.15.66:0/4070795402 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f790c0d0800 con 0x7f791c1022f0 2024-04-21T09:07:09.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.970+0000 7f79213b1700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f79080a3cd0 0x7f79080a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7918007b60 tx=0x7f79180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:09.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.978+0000 7f79127fc700 1 -- 172.21.15.66:0/4070795402 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f790c014030 con 0x7f791c1022f0 2024-04-21T09:07:09.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.986+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f65dc02df80 con 0x7f65dc06afe0 2024-04-21T09:07:09.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.986+0000 7f65d37fe700 1 -- 172.21.15.66:0/1026436958 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f65cc09e7b0 con 0x7f65dc06afe0 2024-04-21T09:07:09.985 INFO:teuthology.orchestra.run.smithi066.stdout:34359738454 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f65c80a3cd0 msgr2=0x7f65c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f65c80a3cd0 0x7f65c80a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005ea0 tx=0x7f65c4005e30 comp rx=0 tx=0).stop 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc06afe0 msgr2=0x7f65dc199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc06afe0 0x7f65dc199070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b070 tx=0x7f65cc004d00 comp rx=0 tx=0).stop 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 shutdown_connections 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f65c80a3cd0 0x7f65c80a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc108a20 0x7f65dc19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65dc06afe0 0x7f65dc199070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 --2- 172.21.15.66:0/1026436958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc06b980 0x7f65dc1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 >> 172.21.15.66:0/1026436958 conn(0x7f65dc0788b0 msgr2=0x7f65dc0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 shutdown_connections 2024-04-21T09:07:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:09.988+0000 7f65e08a6700 1 -- 172.21.15.66:0/1026436958 wait complete. 2024-04-21T09:07:10.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.199+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f791c02df80 con 0x7f791c1022f0 2024-04-21T09:07:10.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.199+0000 7f79127fc700 1 -- 172.21.15.66:0/4070795402 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f790c09cd20 con 0x7f791c1022f0 2024-04-21T09:07:10.199 INFO:teuthology.orchestra.run.smithi066.stdout:292057776167 2024-04-21T09:07:10.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f79080a3cd0 msgr2=0x7f79080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:10.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f79080a3cd0 0x7f79080a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7918007b60 tx=0x7f79180073d0 comp rx=0 tx=0).stop 2024-04-21T09:07:10.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 msgr2=0x7f791c199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:10.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c199580 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f790c00c920 tx=0x7f790c00cc30 comp rx=0 tx=0).stop 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 shutdown_connections 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f79080a3cd0 0x7f79080a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f791c101930 0x7f791c199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f791c107980 0x7f791c19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 --2- 172.21.15.66:0/4070795402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f791c1022f0 0x7f791c199580 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 >> 172.21.15.66:0/4070795402 conn(0x7f791c0788b0 msgr2=0x7f791c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 shutdown_connections 2024-04-21T09:07:10.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.202+0000 7f7923615700 1 -- 172.21.15.66:0/4070795402 wait complete. 2024-04-21T09:07:10.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1026436958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T09:07:10.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4070795402' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T09:07:10.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1026436958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T09:07:10.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4070795402' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T09:07:10.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1026436958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T09:07:10.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4070795402' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T09:07:11.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.417+0000 7f88a0e46700 1 -- 172.21.15.66:0/1054559967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c103450 msgr2=0x7f889c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.417+0000 7f88a0e46700 1 --2- 172.21.15.66:0/1054559967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c103450 0x7f889c10f8b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f889400b1a0 tx=0x7f889400b4b0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.418+0000 7f88a0e46700 1 -- 172.21.15.66:0/1054559967 shutdown_connections 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.418+0000 7f88a0e46700 1 --2- 172.21.15.66:0/1054559967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889c108a50 0x7f889c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.418+0000 7f88a0e46700 1 --2- 172.21.15.66:0/1054559967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c103450 0x7f889c10f8b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.418+0000 7f88a0e46700 1 --2- 172.21.15.66:0/1054559967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c102a90 0x7f889c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.418+0000 7f88a0e46700 1 -- 172.21.15.66:0/1054559967 >> 172.21.15.66:0/1054559967 conn(0x7f889c079ba0 msgr2=0x7f889c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.418+0000 7f88a0e46700 1 -- 172.21.15.66:0/1054559967 shutdown_connections 2024-04-21T09:07:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.418+0000 7f88a0e46700 1 -- 172.21.15.66:0/1054559967 wait complete. 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 Processor -- start 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 -- start start 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889c102a90 0x7f889c196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c103450 0x7f889c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c108a50 0x7f889c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f889c110ad0 con 0x7f889c108a50 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f889c110950 con 0x7f889c102a90 2024-04-21T09:07:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f88a0e46700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f889c110c50 con 0x7f889c103450 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c108a50 0x7f889c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c108a50 0x7f889c19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50716/0 (socket says 172.21.15.66:50716) 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 -- 172.21.15.66:0/2325650463 learned_addr learned my addr 172.21.15.66:0/2325650463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889a59c700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889c102a90 0x7f889c196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f8899d9b700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c103450 0x7f889c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 -- 172.21.15.66:0/2325650463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c103450 msgr2=0x7f889c197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c103450 0x7f889c197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 -- 172.21.15.66:0/2325650463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889c102a90 msgr2=0x7f889c196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889c102a90 0x7f889c196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.419+0000 7f889ad9d700 1 -- 172.21.15.66:0/2325650463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c19bf50 con 0x7f889c108a50 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.420+0000 7f889ad9d700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c108a50 0x7f889c19b810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f889400bfc0 tx=0x7f889400bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.420+0000 7f88937fe700 1 -- 172.21.15.66:0/2325650463 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894026070 con 0x7f889c108a50 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.420+0000 7f88937fe700 1 -- 172.21.15.66:0/2325650463 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8894007ce0 con 0x7f889c108a50 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.420+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c19c1e0 con 0x7f889c108a50 2024-04-21T09:07:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.420+0000 7f88937fe700 1 -- 172.21.15.66:0/2325650463 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88940104d0 con 0x7f889c108a50 2024-04-21T09:07:11.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.420+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c1a3ac0 con 0x7f889c108a50 2024-04-21T09:07:11.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.421+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c069730 con 0x7f889c108a50 2024-04-21T09:07:11.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.423+0000 7f88937fe700 1 -- 172.21.15.66:0/2325650463 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8894029070 con 0x7f889c108a50 2024-04-21T09:07:11.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.423+0000 7f88937fe700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f88880a3cd0 0x7f88880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.423+0000 7f889a59c700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f88880a3cd0 0x7f88880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.423+0000 7f88937fe700 1 -- 172.21.15.66:0/2325650463 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f88940d4fb0 con 0x7f889c108a50 2024-04-21T09:07:11.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.424+0000 7f889a59c700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f88880a3cd0 0x7f88880a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8884007b60 tx=0x7f88840073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.427+0000 7f88937fe700 1 -- 172.21.15.66:0/2325650463 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88940a1670 con 0x7f889c108a50 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 -- 172.21.15.66:0/3396199334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978101940 msgr2=0x7f6978101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 --2- 172.21.15.66:0/3396199334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978101940 0x7f6978101dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f696c009990 tx=0x7f696c009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 -- 172.21.15.66:0/3396199334 shutdown_connections 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 --2- 172.21.15.66:0/3396199334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6978107990 0x7f6978107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 --2- 172.21.15.66:0/3396199334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6978102300 0x7f697810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 --2- 172.21.15.66:0/3396199334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978101940 0x7f6978101dc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 -- 172.21.15.66:0/3396199334 >> 172.21.15.66:0/3396199334 conn(0x7f69780788f0 msgr2=0x7f6978078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 -- 172.21.15.66:0/3396199334 shutdown_connections 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.790+0000 7f697e94d700 1 -- 172.21.15.66:0/3396199334 wait complete. 2024-04-21T09:07:11.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 Processor -- start 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 -- start start 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6978101940 0x7f69781034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6978102300 0x7f6978103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978107990 0x7f6978103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6978111b60 con 0x7f6978107990 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69781119e0 con 0x7f6978102300 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697e94d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6978111ce0 con 0x7f6978101940 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697ceea700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978107990 0x7f6978103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697ceea700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978107990 0x7f6978103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50688/0 (socket says 172.21.15.66:50688) 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f697ceea700 1 -- 172.21.15.66:0/4258902416 learned_addr learned my addr 172.21.15.66:0/4258902416 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f69777fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6978102300 0x7f6978103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.791+0000 7f6977fff700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6978101940 0x7f69781034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697ceea700 1 -- 172.21.15.66:0/4258902416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6978101940 msgr2=0x7f69781034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697ceea700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6978101940 0x7f69781034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697ceea700 1 -- 172.21.15.66:0/4258902416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6978102300 msgr2=0x7f6978103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697ceea700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6978102300 0x7f6978103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697ceea700 1 -- 172.21.15.66:0/4258902416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69781a76b0 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697ceea700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978107990 0x7f6978103f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f697000ada0 tx=0x7f697000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f69757fa700 1 -- 172.21.15.66:0/4258902416 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970017020 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f69757fa700 1 -- 172.21.15.66:0/4258902416 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f697000cd80 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69781a79a0 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f69757fa700 1 -- 172.21.15.66:0/4258902416 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970005260 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.792+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69781a7e60 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.793+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6978069730 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.794+0000 7f69757fa700 1 -- 172.21.15.66:0/4258902416 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f697000c8d0 con 0x7f6978107990 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.795+0000 7f69757fa700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69600a3cd0 0x7f69600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:11.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.795+0000 7f6977fff700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69600a3cd0 0x7f69600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:11.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.795+0000 7f69757fa700 1 -- 172.21.15.66:0/4258902416 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f69700d0020 con 0x7f6978107990 2024-04-21T09:07:11.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.795+0000 7f6977fff700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69600a3cd0 0x7f69600a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6968009a10 tx=0x7f6968009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:11.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:10.800+0000 7f69757fa700 1 -- 172.21.15.66:0/4258902416 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6970021020 con 0x7f6978107990 2024-04-21T09:07:11.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.013+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f697802e140 con 0x7f6978107990 2024-04-21T09:07:11.508 INFO:teuthology.orchestra.run.smithi066.stdout:223338299443 2024-04-21T09:07:11.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.013+0000 7f69757fa700 1 -- 172.21.15.66:0/4258902416 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f697009c5d0 con 0x7f6978107990 2024-04-21T09:07:11.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.015+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69600a3cd0 msgr2=0x7f69600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.015+0000 7f697e94d700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69600a3cd0 0x7f69600a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6968009a10 tx=0x7f6968009190 comp rx=0 tx=0).stop 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.015+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978107990 msgr2=0x7f6978103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.015+0000 7f697e94d700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978107990 0x7f6978103f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f697000ada0 tx=0x7f697000c330 comp rx=0 tx=0).stop 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.015+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 shutdown_connections 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.016+0000 7f697e94d700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69600a3cd0 0x7f69600a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.016+0000 7f697e94d700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6978107990 0x7f6978103f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.016+0000 7f697e94d700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6978101940 0x7f69781034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.016+0000 7f697e94d700 1 --2- 172.21.15.66:0/4258902416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6978102300 0x7f6978103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.016+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 >> 172.21.15.66:0/4258902416 conn(0x7f69780788f0 msgr2=0x7f697806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.016+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 shutdown_connections 2024-04-21T09:07:11.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.016+0000 7f697e94d700 1 -- 172.21.15.66:0/4258902416 wait complete. 2024-04-21T09:07:11.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:11 smithi066 ceph-mon[27953]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:11.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4258902416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T09:07:11.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.639+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f889c02df80 con 0x7f889c108a50 2024-04-21T09:07:11.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.639+0000 7f88937fe700 1 -- 172.21.15.66:0/2325650463 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88940a0f70 con 0x7f889c108a50 2024-04-21T09:07:11.640 INFO:teuthology.orchestra.run.smithi066.stdout:55834574929 2024-04-21T09:07:11.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f88880a3cd0 msgr2=0x7f88880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f88880a3cd0 0x7f88880a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8884007b60 tx=0x7f88840073d0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c108a50 msgr2=0x7f889c19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c108a50 0x7f889c19b810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f889400bfc0 tx=0x7f889400bff0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 shutdown_connections 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f88880a3cd0 0x7f88880a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889c102a90 0x7f889c196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c108a50 0x7f889c19b810 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 --2- 172.21.15.66:0/2325650463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c103450 0x7f889c197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.641+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 >> 172.21.15.66:0/2325650463 conn(0x7f889c079ba0 msgr2=0x7f889c101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.642+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 shutdown_connections 2024-04-21T09:07:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:11.642+0000 7f88a0e46700 1 -- 172.21.15.66:0/2325650463 wait complete. 2024-04-21T09:07:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:11 smithi186 ceph-mon[31505]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4258902416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T09:07:11.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:11 smithi083 ceph-mon[31413]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:11.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4258902416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T09:07:12.358 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574929 for osd.1 2024-04-21T09:07:12.358 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:12.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2325650463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T09:07:12.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2325650463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T09:07:12.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2325650463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T09:07:12.962 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738454 for osd.0 2024-04-21T09:07:12.963 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:13.623 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:13 smithi066 ceph-mon[27953]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-21T09:07:13.796 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411404 for osd.2 2024-04-21T09:07:13.796 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:13 smithi186 ceph-mon[31505]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-21T09:07:13.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:13 smithi083 ceph-mon[31413]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-21T09:07:14.438 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776167 for osd.9 2024-04-21T09:07:14.439 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:15.663 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299438 got 223338299443 for osd.7 2024-04-21T09:07:15.663 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:15.760 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:15 smithi066 ceph-mon[27953]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:15 smithi186 ceph-mon[31505]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:15 smithi083 ceph-mon[31413]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:16.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/481939809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 -- 172.21.15.66:0/2239587679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8101940 msgr2=0x7f77c8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 --2- 172.21.15.66:0/2239587679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8101940 0x7f77c8101dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009990 tx=0x7f77b8009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 -- 172.21.15.66:0/2239587679 shutdown_connections 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 --2- 172.21.15.66:0/2239587679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c8107990 0x7f77c8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 --2- 172.21.15.66:0/2239587679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8102300 0x7f77c810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 --2- 172.21.15.66:0/2239587679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8101940 0x7f77c8101dc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 -- 172.21.15.66:0/2239587679 >> 172.21.15.66:0/2239587679 conn(0x7f77c80788f0 msgr2=0x7f77c8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 -- 172.21.15.66:0/2239587679 shutdown_connections 2024-04-21T09:07:16.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.867+0000 7f77cf671700 1 -- 172.21.15.66:0/2239587679 wait complete. 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 Processor -- start 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 -- start start 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101940 0x7f77c81034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c8102300 0x7f77c8103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8107990 0x7f77c8103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8111ae0 con 0x7f77c8107990 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8111960 con 0x7f77c8102300 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cf671700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8111c60 con 0x7f77c8101940 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8107990 0x7f77c8103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:16.830 INFO:teuthology.orchestra.run.smithi066.stdout:146028888131 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8107990 0x7f77c8103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46188/0 (socket says 172.21.15.66:46188) 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 -- 172.21.15.66:0/481939809 learned_addr learned my addr 172.21.15.66:0/481939809 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cd40d700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101940 0x7f77c81034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77ccc0c700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c8102300 0x7f77c8103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 -- 172.21.15.66:0/481939809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101940 msgr2=0x7f77c81034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101940 0x7f77c81034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 -- 172.21.15.66:0/481939809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c8102300 msgr2=0x7f77c8103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:16.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c8102300 0x7f77c8103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 -- 172.21.15.66:0/481939809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c81a7630 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cd40d700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101940 0x7f77c81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77ccc0c700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c8102300 0x7f77c8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.868+0000 7f77cdc0e700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8107990 0x7f77c8103f70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f77c000ada0 tx=0x7f77c000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.869+0000 7f77be7fc700 1 -- 172.21.15.66:0/481939809 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0017020 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.869+0000 7f77be7fc700 1 -- 172.21.15.66:0/481939809 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c000ce30 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.869+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c81a7920 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.869+0000 7f77be7fc700 1 -- 172.21.15.66:0/481939809 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c00054a0 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.869+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c81a7e10 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.870+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c8069730 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.871+0000 7f77be7fc700 1 -- 172.21.15.66:0/481939809 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f77c000c980 con 0x7f77c8107990 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.871+0000 7f77be7fc700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f77b40a3cd0 0x7f77b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.871+0000 7f77cd40d700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f77b40a3cd0 0x7f77b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.872+0000 7f77be7fc700 1 -- 172.21.15.66:0/481939809 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f77c00d0630 con 0x7f77c8107990 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.872+0000 7f77cd40d700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f77b40a3cd0 0x7f77b40a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77c400bb90 tx=0x7f77c4005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:15.875+0000 7f77be7fc700 1 -- 172.21.15.66:0/481939809 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77c0021020 con 0x7f77c8107990 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.085+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f77c802e140 con 0x7f77c8107990 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.086+0000 7f77be7fc700 1 -- 172.21.15.66:0/481939809 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f77c009cbe0 con 0x7f77c8107990 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f77b40a3cd0 msgr2=0x7f77b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f77b40a3cd0 0x7f77b40a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77c400bb90 tx=0x7f77c4005d80 comp rx=0 tx=0).stop 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8107990 msgr2=0x7f77c8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8107990 0x7f77c8103f70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f77c000ada0 tx=0x7f77c000c330 comp rx=0 tx=0).stop 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 shutdown_connections 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f77b40a3cd0 0x7f77b40a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77c8107990 0x7f77c8103f70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101940 0x7f77c81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 --2- 172.21.15.66:0/481939809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c8102300 0x7f77c8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 >> 172.21.15.66:0/481939809 conn(0x7f77c80788f0 msgr2=0x7f77c806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 shutdown_connections 2024-04-21T09:07:16.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.088+0000 7f77cf671700 1 -- 172.21.15.66:0/481939809 wait complete. 2024-04-21T09:07:16.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/481939809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T09:07:16.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/481939809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 -- 172.21.15.66:0/2647666111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 msgr2=0x7f20f4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 --2- 172.21.15.66:0/2647666111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f4108e50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009990 tx=0x7f20e4009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 -- 172.21.15.66:0/2647666111 shutdown_connections 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 --2- 172.21.15.66:0/2647666111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f4103460 0x7f20f410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 --2- 172.21.15.66:0/2647666111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f4108e50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 --2- 172.21.15.66:0/2647666111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f4102ac0 0x7f20f4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 -- 172.21.15.66:0/2647666111 >> 172.21.15.66:0/2647666111 conn(0x7f20f4079b10 msgr2=0x7f20f4079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 -- 172.21.15.66:0/2647666111 shutdown_connections 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.049+0000 7f20f8a37700 1 -- 172.21.15.66:0/2647666111 wait complete. 2024-04-21T09:07:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 Processor -- start 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 -- start start 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f4102ac0 0x7f20f4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f4103460 0x7f20f4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4112bf0 con 0x7f20f4108a70 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4112a70 con 0x7f20f4103460 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f8a37700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4112d70 con 0x7f20f4102ac0 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f40757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46226/0 (socket says 172.21.15.66:46226) 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f4102ac0 0x7f20f4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 -- 172.21.15.66:0/4185424320 learned_addr learned my addr 172.21.15.66:0/4185424320 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f1d9b700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f4103460 0x7f20f4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 -- 172.21.15.66:0/4185424320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f4102ac0 msgr2=0x7f20f4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f4102ac0 0x7f20f4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 -- 172.21.15.66:0/4185424320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f4103460 msgr2=0x7f20f4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f4103460 0x7f20f4075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.050+0000 7f20f2d9d700 1 -- 172.21.15.66:0/4185424320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f41a7650 con 0x7f20f4108a70 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.051+0000 7f20f2d9d700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f40757c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00ada0 tx=0x7f20ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.051+0000 7f20e37fe700 1 -- 172.21.15.66:0/4185424320 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec017020 con 0x7f20f4108a70 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.051+0000 7f20e37fe700 1 -- 172.21.15.66:0/4185424320 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20ec00cd80 con 0x7f20f4108a70 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.051+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f41a7940 con 0x7f20f4108a70 2024-04-21T09:07:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.051+0000 7f20e37fe700 1 -- 172.21.15.66:0/4185424320 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec005260 con 0x7f20f4108a70 2024-04-21T09:07:17.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.051+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f41a7e80 con 0x7f20f4108a70 2024-04-21T09:07:17.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.052+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f4069730 con 0x7f20f4108a70 2024-04-21T09:07:17.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.053+0000 7f20e37fe700 1 -- 172.21.15.66:0/4185424320 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f20ec00c8d0 con 0x7f20f4108a70 2024-04-21T09:07:17.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.054+0000 7f20e37fe700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f20dc0a3d50 0x7f20dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.054+0000 7f20f259c700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f20dc0a3d50 0x7f20dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.054+0000 7f20e37fe700 1 -- 172.21.15.66:0/4185424320 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f20ec0cff00 con 0x7f20f4108a70 2024-04-21T09:07:17.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.054+0000 7f20f259c700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f20dc0a3d50 0x7f20dc0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b5c0 tx=0x7f20e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.059+0000 7f20e37fe700 1 -- 172.21.15.66:0/4185424320 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20ec021020 con 0x7f20f4108a70 2024-04-21T09:07:17.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.270+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f20f4076270 con 0x7f20f4108a70 2024-04-21T09:07:17.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.270+0000 7f20e37fe700 1 -- 172.21.15.66:0/4185424320 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f20ec09c4b0 con 0x7f20f4108a70 2024-04-21T09:07:17.270 INFO:teuthology.orchestra.run.smithi066.stdout:201863462968 2024-04-21T09:07:17.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f20dc0a3d50 msgr2=0x7f20dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f20dc0a3d50 0x7f20dc0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b5c0 tx=0x7f20e4005960 comp rx=0 tx=0).stop 2024-04-21T09:07:17.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 msgr2=0x7f20f40757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f40757c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00ada0 tx=0x7f20ec00c330 comp rx=0 tx=0).stop 2024-04-21T09:07:17.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 shutdown_connections 2024-04-21T09:07:17.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f20dc0a3d50 0x7f20dc0a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4108a70 0x7f20f40757c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f4102ac0 0x7f20f4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.272+0000 7f20f8a37700 1 --2- 172.21.15.66:0/4185424320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f4103460 0x7f20f4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.273+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 >> 172.21.15.66:0/4185424320 conn(0x7f20f4079b10 msgr2=0x7f20f4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:17.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.273+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 shutdown_connections 2024-04-21T09:07:17.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.273+0000 7f20f8a37700 1 -- 172.21.15.66:0/4185424320 wait complete. 2024-04-21T09:07:17.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:17 smithi066 ceph-mon[27953]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:17.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1838367311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T09:07:17.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4185424320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T09:07:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:17 smithi186 ceph-mon[31505]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1838367311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T09:07:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4185424320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T09:07:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:17 smithi083 ceph-mon[31413]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1838367311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T09:07:17.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4185424320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T09:07:17.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/366511200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4075590 msgr2=0x7fa9e4113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/366511200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4075590 0x7fa9e4113700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009990 tx=0x7fa9d4009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/366511200 shutdown_connections 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/366511200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4113c40 0x7fa9e4116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/366511200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e4074c70 0x7fa9e4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/366511200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4075590 0x7fa9e4113700 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/366511200 >> 172.21.15.66:0/366511200 conn(0x7fa9e4079bb0 msgr2=0x7fa9e4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/366511200 shutdown_connections 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/366511200 wait complete. 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 Processor -- start 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.923+0000 7fa9ebe2f700 1 -- start start 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ebe2f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4074c70 0x7fa9e41a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ebe2f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e4075590 0x7fa9e41a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ebe2f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4113c40 0x7fa9e419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ebe2f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41192a0 con 0x7fa9e4074c70 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ebe2f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4119120 con 0x7fa9e4113c40 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ebe2f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4119420 con 0x7fa9e4075590 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4074c70 0x7fa9e41a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4074c70 0x7fa9e41a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46252/0 (socket says 172.21.15.66:46252) 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 -- 172.21.15.66:0/1210408463 learned_addr learned my addr 172.21.15.66:0/1210408463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e93ca700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e4075590 0x7fa9e41a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ea3cc700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4113c40 0x7fa9e419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 -- 172.21.15.66:0/1210408463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e4075590 msgr2=0x7fa9e41a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e4075590 0x7fa9e41a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 -- 172.21.15.66:0/1210408463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4113c40 msgr2=0x7fa9e419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4113c40 0x7fa9e419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 -- 172.21.15.66:0/1210408463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e419fd60 con 0x7fa9e4074c70 2024-04-21T09:07:17.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9ea3cc700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4113c40 0x7fa9e419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9e9bcb700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4074c70 0x7fa9e41a5440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000b530 tx=0x7fa9e000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.924+0000 7fa9daffd700 1 -- 172.21.15.66:0/1210408463 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0015020 con 0x7fa9e4074c70 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.925+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e41a0050 con 0x7fa9e4074c70 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.925+0000 7fa9daffd700 1 -- 172.21.15.66:0/1210408463 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e0011ce0 con 0x7fa9e4074c70 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.925+0000 7fa9daffd700 1 -- 172.21.15.66:0/1210408463 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0010e40 con 0x7fa9e4074c70 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.925+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41ac310 con 0x7fa9e4074c70 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.925+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e4069730 con 0x7fa9e4074c70 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.927+0000 7fa9daffd700 1 -- 172.21.15.66:0/1210408463 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa9e0018020 con 0x7fa9e4074c70 2024-04-21T09:07:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.927+0000 7fa9daffd700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:17.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.927+0000 7fa9e93ca700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:17.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.927+0000 7fa9daffd700 1 -- 172.21.15.66:0/1210408463 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fa9e00d05b0 con 0x7fa9e4074c70 2024-04-21T09:07:17.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.928+0000 7fa9e93ca700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400b5c0 tx=0x7fa9d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:17.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:17.931+0000 7fa9daffd700 1 -- 172.21.15.66:0/1210408463 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9e001f020 con 0x7fa9e4074c70 2024-04-21T09:07:18.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.142+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa9e41a0b30 con 0x7fa9e4074c70 2024-04-21T09:07:18.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.142+0000 7fa9daffd700 1 -- 172.21.15.66:0/1210408463 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa9e009cbe0 con 0x7fa9e4074c70 2024-04-21T09:07:18.142 INFO:teuthology.orchestra.run.smithi066.stdout:111669149768 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9d00a3cd0 msgr2=0x7fa9d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400b5c0 tx=0x7fa9d4005960 comp rx=0 tx=0).stop 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4074c70 msgr2=0x7fa9e41a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4074c70 0x7fa9e41a5440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000b530 tx=0x7fa9e000b840 comp rx=0 tx=0).stop 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 shutdown_connections 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4113c40 0x7fa9e419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9e4074c70 0x7fa9e41a5440 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 --2- 172.21.15.66:0/1210408463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e4075590 0x7fa9e41a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 >> 172.21.15.66:0/1210408463 conn(0x7fa9e4079bb0 msgr2=0x7fa9e4112a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 shutdown_connections 2024-04-21T09:07:18.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.144+0000 7fa9ebe2f700 1 -- 172.21.15.66:0/1210408463 wait complete. 2024-04-21T09:07:18.422 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462968 for osd.6 2024-04-21T09:07:18.423 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 -- 172.21.15.66:0/3624119468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670101940 msgr2=0x7f3670101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 --2- 172.21.15.66:0/3624119468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670101940 0x7f3670101dc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3664009990 tx=0x7f3664009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 -- 172.21.15.66:0/3624119468 shutdown_connections 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 --2- 172.21.15.66:0/3624119468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3670107990 0x7f3670107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 --2- 172.21.15.66:0/3624119468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3670102300 0x7f367010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 --2- 172.21.15.66:0/3624119468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670101940 0x7f3670101dc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 -- 172.21.15.66:0/3624119468 >> 172.21.15.66:0/3624119468 conn(0x7f36700788f0 msgr2=0x7f3670078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 -- 172.21.15.66:0/3624119468 shutdown_connections 2024-04-21T09:07:18.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.616+0000 7f3676fd9700 1 -- 172.21.15.66:0/3624119468 wait complete. 2024-04-21T09:07:18.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 Processor -- start 2024-04-21T09:07:18.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 -- start start 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3670101940 0x7f3670199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670102300 0x7f367019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3670107990 0x7f36701a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3670111ae0 con 0x7f3670102300 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3670111960 con 0x7f3670107990 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3676fd9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3670111c60 con 0x7f3670101940 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f366ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670102300 0x7f367019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f366ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670102300 0x7f367019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46270/0 (socket says 172.21.15.66:46270) 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f366ffff700 1 -- 172.21.15.66:0/1904713764 learned_addr learned my addr 172.21.15.66:0/1904713764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3674d75700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3670101940 0x7f3670199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.617+0000 7f3675576700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3670107990 0x7f36701a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366ffff700 1 -- 172.21.15.66:0/1904713764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3670101940 msgr2=0x7f3670199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366ffff700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3670101940 0x7f3670199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366ffff700 1 -- 172.21.15.66:0/1904713764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3670107990 msgr2=0x7f36701a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366ffff700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3670107990 0x7f36701a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366ffff700 1 -- 172.21.15.66:0/1904713764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36701a5540 con 0x7f3670102300 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f3675576700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3670107990 0x7f36701a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366ffff700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670102300 0x7f367019a4c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3664000c00 tx=0x7f36640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366dffb700 1 -- 172.21.15.66:0/1904713764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664027070 con 0x7f3670102300 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366dffb700 1 -- 172.21.15.66:0/1904713764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36640041d0 con 0x7f3670102300 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36701a5710 con 0x7f3670102300 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f366dffb700 1 -- 172.21.15.66:0/1904713764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366400f460 con 0x7f3670102300 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.618+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701a5b60 con 0x7f3670102300 2024-04-21T09:07:18.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.619+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3670069730 con 0x7f3670102300 2024-04-21T09:07:18.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.621+0000 7f366dffb700 1 -- 172.21.15.66:0/1904713764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f366402a070 con 0x7f3670102300 2024-04-21T09:07:18.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.622+0000 7f366dffb700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36580a3cd0 0x7f36580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.622+0000 7f3674d75700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36580a3cd0 0x7f36580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.622+0000 7f366dffb700 1 -- 172.21.15.66:0/1904713764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f36640d1cd0 con 0x7f3670102300 2024-04-21T09:07:18.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.622+0000 7f3674d75700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36580a3cd0 0x7f36580a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f366800e270 tx=0x7f3668009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:18.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.625+0000 7f366dffb700 1 -- 172.21.15.66:0/1904713764 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3664031070 con 0x7f3670102300 2024-04-21T09:07:18.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 -- 172.21.15.66:0/3904837837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec101970 msgr2=0x7f23ec101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 --2- 172.21.15.66:0/3904837837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec101970 0x7f23ec101df0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009990 tx=0x7f23e0009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 -- 172.21.15.66:0/3904837837 shutdown_connections 2024-04-21T09:07:18.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 --2- 172.21.15.66:0/3904837837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ec102330 0x7f23ec10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 --2- 172.21.15.66:0/3904837837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ec107920 0x7f23ec107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 --2- 172.21.15.66:0/3904837837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec101970 0x7f23ec101df0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 -- 172.21.15.66:0/3904837837 >> 172.21.15.66:0/3904837837 conn(0x7f23ec078910 msgr2=0x7f23ec078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 -- 172.21.15.66:0/3904837837 shutdown_connections 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.597+0000 7f23f2a54700 1 -- 172.21.15.66:0/3904837837 wait complete. 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 Processor -- start 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 -- start start 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ec101970 0x7f23ec197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ec102330 0x7f23ec198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec107920 0x7f23ec1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec111b60 con 0x7f23ec107920 2024-04-21T09:07:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1119e0 con 0x7f23ec102330 2024-04-21T09:07:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f2a54700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec111ce0 con 0x7f23ec101970 2024-04-21T09:07:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec107920 0x7f23ec1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec107920 0x7f23ec1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46212/0 (socket says 172.21.15.66:46212) 2024-04-21T09:07:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 -- 172.21.15.66:0/1838367311 learned_addr learned my addr 172.21.15.66:0/1838367311 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23eb7fe700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ec102330 0x7f23ec198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23ebfff700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ec101970 0x7f23ec197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.718 INFO:teuthology.orchestra.run.smithi066.stdout:347892351007 2024-04-21T09:07:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 -- 172.21.15.66:0/1838367311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ec101970 msgr2=0x7f23ec197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ec101970 0x7f23ec197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 -- 172.21.15.66:0/1838367311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ec102330 msgr2=0x7f23ec198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ec102330 0x7f23ec198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23f0ff1700 1 -- 172.21.15.66:0/1838367311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec1a7640 con 0x7f23ec107920 2024-04-21T09:07:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23eb7fe700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ec102330 0x7f23ec198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.598+0000 7f23ebfff700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ec101970 0x7f23ec197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.599+0000 7f23f0ff1700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec107920 0x7f23ec1988b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f23e4007ee0 tx=0x7f23e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.599+0000 7f23e97fa700 1 -- 172.21.15.66:0/1838367311 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e400dbd0 con 0x7f23ec107920 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.599+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec1a7930 con 0x7f23ec107920 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.599+0000 7f23e97fa700 1 -- 172.21.15.66:0/1838367311 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e400dd70 con 0x7f23ec107920 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.599+0000 7f23e97fa700 1 -- 172.21.15.66:0/1838367311 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4005990 con 0x7f23ec107920 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.599+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec1a7e70 con 0x7f23ec107920 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.600+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ec069730 con 0x7f23ec107920 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.601+0000 7f23e97fa700 1 -- 172.21.15.66:0/1838367311 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f23e401a020 con 0x7f23ec107920 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.602+0000 7f23e97fa700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23d40a3cd0 0x7f23d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.602+0000 7f23ebfff700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23d40a3cd0 0x7f23d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.602+0000 7f23e97fa700 1 -- 172.21.15.66:0/1838367311 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f23e40d01c0 con 0x7f23ec107920 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.602+0000 7f23ebfff700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23d40a3cd0 0x7f23d40a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0099d0 tx=0x7f23dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.606+0000 7f23e97fa700 1 -- 172.21.15.66:0/1838367311 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23e4010070 con 0x7f23ec107920 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.820+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f23ec02df80 con 0x7f23ec107920 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.820+0000 7f23e97fa700 1 -- 172.21.15.66:0/1838367311 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f23e409c6e0 con 0x7f23ec107920 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.822+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23d40a3cd0 msgr2=0x7f23d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.822+0000 7f23f2a54700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23d40a3cd0 0x7f23d40a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0099d0 tx=0x7f23dc008040 comp rx=0 tx=0).stop 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.822+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec107920 msgr2=0x7f23ec1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.822+0000 7f23f2a54700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec107920 0x7f23ec1988b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f23e4007ee0 tx=0x7f23e400d3b0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.822+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 shutdown_connections 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.823+0000 7f23f2a54700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23d40a3cd0 0x7f23d40a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.823+0000 7f23f2a54700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ec107920 0x7f23ec1988b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.823+0000 7f23f2a54700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ec101970 0x7f23ec197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.823+0000 7f23f2a54700 1 --2- 172.21.15.66:0/1838367311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ec102330 0x7f23ec198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.823+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 >> 172.21.15.66:0/1838367311 conn(0x7f23ec078910 msgr2=0x7f23ec0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:18.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.823+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 shutdown_connections 2024-04-21T09:07:18.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:16.823+0000 7f23f2a54700 1 -- 172.21.15.66:0/1838367311 wait complete. 2024-04-21T09:07:18.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:18 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1210408463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T09:07:18.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.834+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f367002df80 con 0x7f3670102300 2024-04-21T09:07:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.834+0000 7f366dffb700 1 -- 172.21.15.66:0/1904713764 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3664013070 con 0x7f3670102300 2024-04-21T09:07:18.834 INFO:teuthology.orchestra.run.smithi066.stdout:167503724607 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36580a3cd0 msgr2=0x7f36580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36580a3cd0 0x7f36580a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f366800e270 tx=0x7f3668009040 comp rx=0 tx=0).stop 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670102300 msgr2=0x7f367019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670102300 0x7f367019a4c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3664000c00 tx=0x7f36640047b0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 shutdown_connections 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36580a3cd0 0x7f36580a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3670107990 0x7f36701a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3670101940 0x7f3670199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 --2- 172.21.15.66:0/1904713764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3670102300 0x7f367019a4c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.836+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 >> 172.21.15.66:0/1904713764 conn(0x7f36700788f0 msgr2=0x7f36700ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.837+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 shutdown_connections 2024-04-21T09:07:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:18.837+0000 7f3676fd9700 1 -- 172.21.15.66:0/1904713764 wait complete. 2024-04-21T09:07:18.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:18 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1210408463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T09:07:18.869 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:18 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1210408463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T09:07:19.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888124 got 146028888131 for osd.4 2024-04-21T09:07:19.486 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:19.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:19 smithi186 ceph-mon[31505]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:19.739 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:19 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1904713764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T09:07:19.759 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:19 smithi066 ceph-mon[27953]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:19.759 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:19 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1904713764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T09:07:19.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:19 smithi083 ceph-mon[31413]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:19.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:19 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1904713764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T09:07:20.078 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724602 got 167503724607 for osd.5 2024-04-21T09:07:21.394 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:21.396 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149762 got 111669149768 for osd.3 2024-04-21T09:07:21.396 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:21.397 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:21.397 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351000 got 347892351007 for osd.11 2024-04-21T09:07:21.397 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:21.527 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:21 smithi066 ceph-mon[27953]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:21 smithi186 ceph-mon[31505]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:21 smithi083 ceph-mon[31413]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 -- 172.21.15.66:0/3277094522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e5006bb30 msgr2=0x7f1e5010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 --2- 172.21.15.66:0/3277094522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e5006bb30 0x7f1e5010f900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800b0a0 tx=0x7f1e4800b3b0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 -- 172.21.15.66:0/3277094522 shutdown_connections 2024-04-21T09:07:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 --2- 172.21.15.66:0/3277094522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e5006bb30 0x7f1e5010f900 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 --2- 172.21.15.66:0/3277094522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e50108a30 0x7f1e50108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 --2- 172.21.15.66:0/3277094522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e5006b190 0x7f1e5006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 -- 172.21.15.66:0/3277094522 >> 172.21.15.66:0/3277094522 conn(0x7f1e50078910 msgr2=0x7f1e50078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 -- 172.21.15.66:0/3277094522 shutdown_connections 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 -- 172.21.15.66:0/3277094522 wait complete. 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.136+0000 7f1e56cde700 1 Processor -- start 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e56cde700 1 -- start start 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e56cde700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e5006b190 0x7f1e50199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e56cde700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e5006bb30 0x7f1e50199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e56cde700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e50108a30 0x7f1e5019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e56cde700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50112bd0 con 0x7f1e50108a30 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e56cde700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50112a50 con 0x7f1e5006bb30 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e56cde700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50112d50 con 0x7f1e5006b190 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e50108a30 0x7f1e5019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e50108a30 0x7f1e5019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46298/0 (socket says 172.21.15.66:46298) 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 -- 172.21.15.66:0/197835940 learned_addr learned my addr 172.21.15.66:0/197835940 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e54a7a700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e5006b190 0x7f1e50199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e4ffff700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e5006bb30 0x7f1e50199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 -- 172.21.15.66:0/197835940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e5006b190 msgr2=0x7f1e50199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e5006b190 0x7f1e50199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 -- 172.21.15.66:0/197835940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e5006bb30 msgr2=0x7f1e50199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e5006bb30 0x7f1e50199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 -- 172.21.15.66:0/197835940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5019e0e0 con 0x7f1e50108a30 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.137+0000 7f1e5527b700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e50108a30 0x7f1e5019d9a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1e480062a0 tx=0x7f1e48004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.138+0000 7f1e4dffb700 1 -- 172.21.15.66:0/197835940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48021070 con 0x7f1e50108a30 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.138+0000 7f1e4dffb700 1 -- 172.21.15.66:0/197835940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e48007e30 con 0x7f1e50108a30 2024-04-21T09:07:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.138+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5019e370 con 0x7f1e50108a30 2024-04-21T09:07:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.138+0000 7f1e4dffb700 1 -- 172.21.15.66:0/197835940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480046d0 con 0x7f1e50108a30 2024-04-21T09:07:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.138+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501a5c50 con 0x7f1e50108a30 2024-04-21T09:07:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.139+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e50069730 con 0x7f1e50108a30 2024-04-21T09:07:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.140+0000 7f1e4dffb700 1 -- 172.21.15.66:0/197835940 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e48007980 con 0x7f1e50108a30 2024-04-21T09:07:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.141+0000 7f1e4dffb700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e400a3d50 0x7f1e400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.141+0000 7f1e54a7a700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e400a3d50 0x7f1e400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.141+0000 7f1e4dffb700 1 -- 172.21.15.66:0/197835940 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f1e4800f020 con 0x7f1e50108a30 2024-04-21T09:07:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.142+0000 7f1e54a7a700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e400a3d50 0x7f1e400a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009be0 tx=0x7f1e44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.146+0000 7f1e4dffb700 1 -- 172.21.15.66:0/197835940 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e4802a020 con 0x7f1e50108a30 2024-04-21T09:07:23.366 INFO:teuthology.orchestra.run.smithi066.stdout:313532612645 2024-04-21T09:07:23.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.364+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1e5002df80 con 0x7f1e50108a30 2024-04-21T09:07:23.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.365+0000 7f1e4dffb700 1 -- 172.21.15.66:0/197835940 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1e4809e200 con 0x7f1e50108a30 2024-04-21T09:07:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e400a3d50 msgr2=0x7f1e400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e400a3d50 0x7f1e400a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009be0 tx=0x7f1e44009260 comp rx=0 tx=0).stop 2024-04-21T09:07:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e50108a30 msgr2=0x7f1e5019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e50108a30 0x7f1e5019d9a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1e480062a0 tx=0x7f1e48004c80 comp rx=0 tx=0).stop 2024-04-21T09:07:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 shutdown_connections 2024-04-21T09:07:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1e400a3d50 0x7f1e400a6210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e50108a30 0x7f1e5019d9a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e5006b190 0x7f1e50199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.367+0000 7f1e56cde700 1 --2- 172.21.15.66:0/197835940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e5006bb30 0x7f1e50199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.368+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 >> 172.21.15.66:0/197835940 conn(0x7f1e50078910 msgr2=0x7f1e500ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.368+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 shutdown_connections 2024-04-21T09:07:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:23.368+0000 7f1e56cde700 1 -- 172.21.15.66:0/197835940 wait complete. 2024-04-21T09:07:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:23 smithi066 ceph-mon[27953]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:07:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:23 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:23 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/197835940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T09:07:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:23 smithi186 ceph-mon[31505]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:07:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:23 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:23 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/197835940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T09:07:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:23 smithi083 ceph-mon[31413]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:07:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:23 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:23 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/197835940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T09:07:24.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612639 got 313532612645 for osd.10 2024-04-21T09:07:24.079 DEBUG:teuthology.parallel:result is None 2024-04-21T09:07:24.079 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-21T09:07:24.079 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T09:07:24.334 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:24.731 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:07:24.732 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:07:24.732 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:24 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:24.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:07:24.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:07:24.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:24 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:24.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:07:24.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:07:24.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:24 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' 2024-04-21T09:07:25.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:25 smithi066 ceph-mon[27953]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:25.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:25 smithi186 ceph-mon[31505]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:25.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:25 smithi083 ceph-mon[31413]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 -- 172.21.15.66:0/3116118834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8107960 msgr2=0x7f9ce8107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 --2- 172.21.15.66:0/3116118834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8107960 0x7f9ce8107d40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009990 tx=0x7f9cd8009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 -- 172.21.15.66:0/3116118834 shutdown_connections 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 --2- 172.21.15.66:0/3116118834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce8102320 0x7f9ce810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 --2- 172.21.15.66:0/3116118834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8107960 0x7f9ce8107d40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 --2- 172.21.15.66:0/3116118834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce8101960 0x7f9ce8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 -- 172.21.15.66:0/3116118834 >> 172.21.15.66:0/3116118834 conn(0x7f9ce80789e0 msgr2=0x7f9ce8078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 -- 172.21.15.66:0/3116118834 shutdown_connections 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 -- 172.21.15.66:0/3116118834 wait complete. 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.193+0000 7f9ced40b700 1 Processor -- start 2024-04-21T09:07:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ced40b700 1 -- start start 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ced40b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8101960 0x7f9ce8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ced40b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce8102320 0x7f9ce8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ced40b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce8107960 0x7f9ce819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ced40b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8111ba0 con 0x7f9ce8101960 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ced40b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8111a20 con 0x7f9ce8107960 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ced40b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8111d20 con 0x7f9ce8102320 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8101960 0x7f9ce8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8101960 0x7f9ce8198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56038/0 (socket says 172.21.15.66:56038) 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 -- 172.21.15.66:0/1015536664 learned_addr learned my addr 172.21.15.66:0/1015536664 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce67fc700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce8102320 0x7f9ce8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce77fe700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce8107960 0x7f9ce819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 -- 172.21.15.66:0/1015536664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce8102320 msgr2=0x7f9ce8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce8102320 0x7f9ce8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 -- 172.21.15.66:0/1015536664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce8107960 msgr2=0x7f9ce819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce8107960 0x7f9ce819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 -- 172.21.15.66:0/1015536664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce819d040 con 0x7f9ce8101960 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.194+0000 7f9ce6ffd700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8101960 0x7f9ce8198030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000c00 tx=0x7f9cd80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.195+0000 7f9ccffff700 1 -- 172.21.15.66:0/1015536664 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8027070 con 0x7f9ce8101960 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.195+0000 7f9ccffff700 1 -- 172.21.15.66:0/1015536664 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd80041d0 con 0x7f9ce8101960 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.195+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce819d2d0 con 0x7f9ce8101960 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.195+0000 7f9ccffff700 1 -- 172.21.15.66:0/1015536664 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800f460 con 0x7f9ce8101960 2024-04-21T09:07:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.195+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce806b170 con 0x7f9ce8101960 2024-04-21T09:07:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.196+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce81030a0 con 0x7f9ce8101960 2024-04-21T09:07:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.197+0000 7f9ccffff700 1 -- 172.21.15.66:0/1015536664 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9cd802a070 con 0x7f9ce8101960 2024-04-21T09:07:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.197+0000 7f9ccffff700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9cd00a3cd0 0x7f9cd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.198+0000 7f9ce67fc700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9cd00a3cd0 0x7f9cd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.198+0000 7f9ccffff700 1 -- 172.21.15.66:0/1015536664 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f9cd80d1cd0 con 0x7f9ce8101960 2024-04-21T09:07:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.198+0000 7f9ce67fc700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9cd00a3cd0 0x7f9cd00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006fd0 tx=0x7f9cdc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.201+0000 7f9ccffff700 1 -- 172.21.15.66:0/1015536664 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cd8031070 con 0x7f9ce8101960 2024-04-21T09:07:26.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.407+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ce806b3d0 con 0x7f9cd00a3cd0 2024-04-21T09:07:26.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.417+0000 7f9ccffff700 1 -- 172.21.15.66:0/1015536664 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340639 (secure 0 0 0) 0x7f9ce806b3d0 con 0x7f9cd00a3cd0 2024-04-21T09:07:26.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9cd00a3cd0 msgr2=0x7f9cd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9cd00a3cd0 0x7f9cd00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006fd0 tx=0x7f9cdc008040 comp rx=0 tx=0).stop 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8101960 msgr2=0x7f9ce8198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8101960 0x7f9ce8198030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000c00 tx=0x7f9cd80047b0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 shutdown_connections 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9cd00a3cd0 0x7f9cd00a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce8107960 0x7f9ce819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce8101960 0x7f9ce8198030 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 --2- 172.21.15.66:0/1015536664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce8102320 0x7f9ce8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 >> 172.21.15.66:0/1015536664 conn(0x7f9ce80789e0 msgr2=0x7f9ce8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:26.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 shutdown_connections 2024-04-21T09:07:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:26.420+0000 7f9ced40b700 1 -- 172.21.15.66:0/1015536664 wait complete. 2024-04-21T09:07:26.427 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:07:27.109 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-21T09:07:26.322815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655987,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":580,"num_read_kb":411,"num_write":341,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357436,"kb_used_data":12380,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584388,"statfs":{"total":1151940427776,"available":1151574413312,"internally_reserved":0,"allocated":12677120,"data_stored":8103167,"data_compressed":394290,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003825"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169425+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:06:40.169425+0000","last_peered":"2024-04-21T09:06:40.169425+0000","last_clean":"2024-04-21T09:06:40.169425+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:06:40.169425+0000","last_undegraded":"2024-04-21T09:06:40.169425+0000","last_fullsized":"2024-04-21T09:06:40.169425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172692+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:06:40.172692+0000","last_peered":"2024-04-21T09:06:40.172692+0000","last_clean":"2024-04-21T09:06:40.172692+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:06:40.172692+0000","last_undegraded":"2024-04-21T09:06:40.172692+0000","last_fullsized":"2024-04-21T09:06:40.172692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172529+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:06:40.172529+0000","last_peered":"2024-04-21T09:06:40.172529+0000","last_clean":"2024-04-21T09:06:40.172529+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:06:40.172529+0000","last_undegraded":"2024-04-21T09:06:40.172529+0000","last_fullsized":"2024-04-21T09:06:40.172529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165192+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:06:40.165192+0000","last_peered":"2024-04-21T09:06:40.165192+0000","last_clean":"2024-04-21T09:06:40.165192+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:06:40.165192+0000","last_undegraded":"2024-04-21T09:06:40.165192+0000","last_fullsized":"2024-04-21T09:06:40.165192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173412+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:06:40.173412+0000","last_peered":"2024-04-21T09:06:40.173412+0000","last_clean":"2024-04-21T09:06:40.173412+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:06:40.173412+0000","last_undegraded":"2024-04-21T09:06:40.173412+0000","last_fullsized":"2024-04-21T09:06:40.173412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167298+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:06:40.167298+0000","last_peered":"2024-04-21T09:06:40.167298+0000","last_clean":"2024-04-21T09:06:40.167298+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:06:40.167298+0000","last_undegraded":"2024-04-21T09:06:40.167298+0000","last_fullsized":"2024-04-21T09:06:40.167298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171293+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:06:40.171293+0000","last_peered":"2024-04-21T09:06:40.171293+0000","last_clean":"2024-04-21T09:06:40.171293+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:06:40.171293+0000","last_undegraded":"2024-04-21T09:06:40.171293+0000","last_fullsized":"2024-04-21T09:06:40.171293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715065+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:06:40.715065+0000","last_peered":"2024-04-21T09:06:40.715065+0000","last_clean":"2024-04-21T09:06:40.715065+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:06:40.715065+0000","last_undegraded":"2024-04-21T09:06:40.715065+0000","last_fullsized":"2024-04-21T09:06:40.715065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169019+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:06:40.169019+0000","last_peered":"2024-04-21T09:06:40.169019+0000","last_clean":"2024-04-21T09:06:40.169019+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:06:40.169019+0000","last_undegraded":"2024-04-21T09:06:40.169019+0000","last_fullsized":"2024-04-21T09:06:40.169019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168208+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:06:40.168208+0000","last_peered":"2024-04-21T09:06:40.168208+0000","last_clean":"2024-04-21T09:06:40.168208+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:06:40.168208+0000","last_undegraded":"2024-04-21T09:06:40.168208+0000","last_fullsized":"2024-04-21T09:06:40.168208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171466+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:06:40.171466+0000","last_peered":"2024-04-21T09:06:40.171466+0000","last_clean":"2024-04-21T09:06:40.171466+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:06:40.171466+0000","last_undegraded":"2024-04-21T09:06:40.171466+0000","last_fullsized":"2024-04-21T09:06:40.171466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165088+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:06:40.165088+0000","last_peered":"2024-04-21T09:06:40.165088+0000","last_clean":"2024-04-21T09:06:40.165088+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:06:40.165088+0000","last_undegraded":"2024-04-21T09:06:40.165088+0000","last_fullsized":"2024-04-21T09:06:40.165088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164047+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:06:40.164047+0000","last_peered":"2024-04-21T09:06:40.164047+0000","last_clean":"2024-04-21T09:06:40.164047+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:06:40.164047+0000","last_undegraded":"2024-04-21T09:06:40.164047+0000","last_fullsized":"2024-04-21T09:06:40.164047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171830+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:06:40.171830+0000","last_peered":"2024-04-21T09:06:40.171830+0000","last_clean":"2024-04-21T09:06:40.171830+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:06:40.171830+0000","last_undegraded":"2024-04-21T09:06:40.171830+0000","last_fullsized":"2024-04-21T09:06:40.171830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172027+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:06:40.172027+0000","last_peered":"2024-04-21T09:06:40.172027+0000","last_clean":"2024-04-21T09:06:40.172027+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:06:40.172027+0000","last_undegraded":"2024-04-21T09:06:40.172027+0000","last_fullsized":"2024-04-21T09:06:40.172027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168275+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:06:40.168275+0000","last_peered":"2024-04-21T09:06:40.168275+0000","last_clean":"2024-04-21T09:06:40.168275+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:06:40.168275+0000","last_undegraded":"2024-04-21T09:06:40.168275+0000","last_fullsized":"2024-04-21T09:06:40.168275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169143+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:06:40.169143+0000","last_peered":"2024-04-21T09:06:40.169143+0000","last_clean":"2024-04-21T09:06:40.169143+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:06:40.169143+0000","last_undegraded":"2024-04-21T09:06:40.169143+0000","last_fullsized":"2024-04-21T09:06:40.169143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716653+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:06:40.716653+0000","last_peered":"2024-04-21T09:06:40.716653+0000","last_clean":"2024-04-21T09:06:40.716653+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:06:40.716653+0000","last_undegraded":"2024-04-21T09:06:40.716653+0000","last_fullsized":"2024-04-21T09:06:40.716653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170406+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:06:40.170406+0000","last_peered":"2024-04-21T09:06:40.170406+0000","last_clean":"2024-04-21T09:06:40.170406+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:06:40.170406+0000","last_undegraded":"2024-04-21T09:06:40.170406+0000","last_fullsized":"2024-04-21T09:06:40.170406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170256+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:06:40.170256+0000","last_peered":"2024-04-21T09:06:40.170256+0000","last_clean":"2024-04-21T09:06:40.170256+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:06:40.170256+0000","last_undegraded":"2024-04-21T09:06:40.170256+0000","last_fullsized":"2024-04-21T09:06:40.170256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716638+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:06:40.716638+0000","last_peered":"2024-04-21T09:06:40.716638+0000","last_clean":"2024-04-21T09:06:40.716638+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:06:40.716638+0000","last_undegraded":"2024-04-21T09:06:40.716638+0000","last_fullsized":"2024-04-21T09:06:40.716638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715807+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:06:40.715807+0000","last_peered":"2024-04-21T09:06:40.715807+0000","last_clean":"2024-04-21T09:06:40.715807+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:06:40.715807+0000","last_undegraded":"2024-04-21T09:06:40.715807+0000","last_fullsized":"2024-04-21T09:06:40.715807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164569+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:06:40.164569+0000","last_peered":"2024-04-21T09:06:40.164569+0000","last_clean":"2024-04-21T09:06:40.164569+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:06:40.164569+0000","last_undegraded":"2024-04-21T09:06:40.164569+0000","last_fullsized":"2024-04-21T09:06:40.164569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716132+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:06:40.716132+0000","last_peered":"2024-04-21T09:06:40.716132+0000","last_clean":"2024-04-21T09:06:40.716132+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:06:40.716132+0000","last_undegraded":"2024-04-21T09:06:40.716132+0000","last_fullsized":"2024-04-21T09:06:40.716132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172602+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:06:40.172602+0000","last_peered":"2024-04-21T09:06:40.172602+0000","last_clean":"2024-04-21T09:06:40.172602+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:06:40.172602+0000","last_undegraded":"2024-04-21T09:06:40.172602+0000","last_fullsized":"2024-04-21T09:06:40.172602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.717725+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:06:40.717725+0000","last_peered":"2024-04-21T09:06:40.717725+0000","last_clean":"2024-04-21T09:06:40.717725+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:06:40.717725+0000","last_undegraded":"2024-04-21T09:06:40.717725+0000","last_fullsized":"2024-04-21T09:06:40.717725+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174392+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:06:40.174392+0000","last_peered":"2024-04-21T09:06:40.174392+0000","last_clean":"2024-04-21T09:06:40.174392+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:06:40.174392+0000","last_undegraded":"2024-04-21T09:06:40.174392+0000","last_fullsized":"2024-04-21T09:06:40.174392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169896+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:06:40.169896+0000","last_peered":"2024-04-21T09:06:40.169896+0000","last_clean":"2024-04-21T09:06:40.169896+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:06:40.169896+0000","last_undegraded":"2024-04-21T09:06:40.169896+0000","last_fullsized":"2024-04-21T09:06:40.169896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164573+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:06:40.164573+0000","last_peered":"2024-04-21T09:06:40.164573+0000","last_clean":"2024-04-21T09:06:40.164573+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:06:40.164573+0000","last_undegraded":"2024-04-21T09:06:40.164573+0000","last_fullsized":"2024-04-21T09:06:40.164573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173290+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:06:40.173290+0000","last_peered":"2024-04-21T09:06:40.173290+0000","last_clean":"2024-04-21T09:06:40.173290+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:06:40.173290+0000","last_undegraded":"2024-04-21T09:06:40.173290+0000","last_fullsized":"2024-04-21T09:06:40.173290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165679+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:06:40.165679+0000","last_peered":"2024-04-21T09:06:40.165679+0000","last_clean":"2024-04-21T09:06:40.165679+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:06:40.165679+0000","last_undegraded":"2024-04-21T09:06:40.165679+0000","last_fullsized":"2024-04-21T09:06:40.165679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170155+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:06:40.170155+0000","last_peered":"2024-04-21T09:06:40.170155+0000","last_clean":"2024-04-21T09:06:40.170155+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:06:40.170155+0000","last_undegraded":"2024-04-21T09:06:40.170155+0000","last_fullsized":"2024-04-21T09:06:40.170155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:21.237590+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:07:21.237590+0000","last_peered":"2024-04-21T09:07:21.237590+0000","last_clean":"2024-04-21T09:07:21.237590+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:07:21.237590+0000","last_undegraded":"2024-04-21T09:07:21.237590+0000","last_fullsized":"2024-04-21T09:07:21.237590+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174355+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:06:40.174355+0000","last_peered":"2024-04-21T09:06:40.174355+0000","last_clean":"2024-04-21T09:06:40.174355+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:06:40.174355+0000","last_undegraded":"2024-04-21T09:06:40.174355+0000","last_fullsized":"2024-04-21T09:06:40.174355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169074+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:06:40.169074+0000","last_peered":"2024-04-21T09:06:40.169074+0000","last_clean":"2024-04-21T09:06:40.169074+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:06:40.169074+0000","last_undegraded":"2024-04-21T09:06:40.169074+0000","last_fullsized":"2024-04-21T09:06:40.169074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167192+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:06:40.167192+0000","last_peered":"2024-04-21T09:06:40.167192+0000","last_clean":"2024-04-21T09:06:40.167192+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:06:40.167192+0000","last_undegraded":"2024-04-21T09:06:40.167192+0000","last_fullsized":"2024-04-21T09:06:40.167192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165915+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:06:40.165915+0000","last_peered":"2024-04-21T09:06:40.165915+0000","last_clean":"2024-04-21T09:06:40.165915+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:06:40.165915+0000","last_undegraded":"2024-04-21T09:06:40.165915+0000","last_fullsized":"2024-04-21T09:06:40.165915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169006+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:06:40.169006+0000","last_peered":"2024-04-21T09:06:40.169006+0000","last_clean":"2024-04-21T09:06:40.169006+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:06:40.169006+0000","last_undegraded":"2024-04-21T09:06:40.169006+0000","last_fullsized":"2024-04-21T09:06:40.169006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.163779+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:06:40.163779+0000","last_peered":"2024-04-21T09:06:40.163779+0000","last_clean":"2024-04-21T09:06:40.163779+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:06:40.163779+0000","last_undegraded":"2024-04-21T09:06:40.163779+0000","last_fullsized":"2024-04-21T09:06:40.163779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715275+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:06:40.715275+0000","last_peered":"2024-04-21T09:06:40.715275+0000","last_clean":"2024-04-21T09:06:40.715275+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:06:40.715275+0000","last_undegraded":"2024-04-21T09:06:40.715275+0000","last_fullsized":"2024-04-21T09:06:40.715275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172570+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:06:40.172570+0000","last_peered":"2024-04-21T09:06:40.172570+0000","last_clean":"2024-04-21T09:06:40.172570+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:06:40.172570+0000","last_undegraded":"2024-04-21T09:06:40.172570+0000","last_fullsized":"2024-04-21T09:06:40.172570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165095+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:06:40.165095+0000","last_peered":"2024-04-21T09:06:40.165095+0000","last_clean":"2024-04-21T09:06:40.165095+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:06:40.165095+0000","last_undegraded":"2024-04-21T09:06:40.165095+0000","last_fullsized":"2024-04-21T09:06:40.165095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716260+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:06:40.716260+0000","last_peered":"2024-04-21T09:06:40.716260+0000","last_clean":"2024-04-21T09:06:40.716260+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:06:40.716260+0000","last_undegraded":"2024-04-21T09:06:40.716260+0000","last_fullsized":"2024-04-21T09:06:40.716260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715452+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:06:40.715452+0000","last_peered":"2024-04-21T09:06:40.715452+0000","last_clean":"2024-04-21T09:06:40.715452+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:06:40.715452+0000","last_undegraded":"2024-04-21T09:06:40.715452+0000","last_fullsized":"2024-04-21T09:06:40.715452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:23.239955+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:07:23.239955+0000","last_peered":"2024-04-21T09:07:23.239955+0000","last_clean":"2024-04-21T09:07:23.239955+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:07:23.239955+0000","last_undegraded":"2024-04-21T09:07:23.239955+0000","last_fullsized":"2024-04-21T09:07:23.239955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169189+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:06:40.169189+0000","last_peered":"2024-04-21T09:06:40.169189+0000","last_clean":"2024-04-21T09:06:40.169189+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:06:40.169189+0000","last_undegraded":"2024-04-21T09:06:40.169189+0000","last_fullsized":"2024-04-21T09:06:40.169189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715876+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:06:40.715876+0000","last_peered":"2024-04-21T09:06:40.715876+0000","last_clean":"2024-04-21T09:06:40.715876+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:06:40.715876+0000","last_undegraded":"2024-04-21T09:06:40.715876+0000","last_fullsized":"2024-04-21T09:06:40.715876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171413+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:06:40.171413+0000","last_peered":"2024-04-21T09:06:40.171413+0000","last_clean":"2024-04-21T09:06:40.171413+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:06:40.171413+0000","last_undegraded":"2024-04-21T09:06:40.171413+0000","last_fullsized":"2024-04-21T09:06:40.171413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172723+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:06:40.172723+0000","last_peered":"2024-04-21T09:06:40.172723+0000","last_clean":"2024-04-21T09:06:40.172723+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:06:40.172723+0000","last_undegraded":"2024-04-21T09:06:40.172723+0000","last_fullsized":"2024-04-21T09:06:40.172723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.177313+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:06:40.177313+0000","last_peered":"2024-04-21T09:06:40.177313+0000","last_clean":"2024-04-21T09:06:40.177313+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:06:40.177313+0000","last_undegraded":"2024-04-21T09:06:40.177313+0000","last_fullsized":"2024-04-21T09:06:40.177313+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165470+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:06:40.165470+0000","last_peered":"2024-04-21T09:06:40.165470+0000","last_clean":"2024-04-21T09:06:40.165470+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:06:40.165470+0000","last_undegraded":"2024-04-21T09:06:40.165470+0000","last_fullsized":"2024-04-21T09:06:40.165470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.175441+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:06:40.175441+0000","last_peered":"2024-04-21T09:06:40.175441+0000","last_clean":"2024-04-21T09:06:40.175441+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:06:40.175441+0000","last_undegraded":"2024-04-21T09:06:40.175441+0000","last_fullsized":"2024-04-21T09:06:40.175441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715915+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:06:40.715915+0000","last_peered":"2024-04-21T09:06:40.715915+0000","last_clean":"2024-04-21T09:06:40.715915+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:06:40.715915+0000","last_undegraded":"2024-04-21T09:06:40.715915+0000","last_fullsized":"2024-04-21T09:06:40.715915+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":211,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:24.241532+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:07:24.241532+0000","last_peered":"2024-04-21T09:07:24.241532+0000","last_clean":"2024-04-21T09:07:24.241532+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:07:24.241532+0000","last_undegraded":"2024-04-21T09:07:24.241532+0000","last_fullsized":"2024-04-21T09:07:24.241532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61775,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":151,"num_read_kb":136,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174135+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:06:40.174135+0000","last_peered":"2024-04-21T09:06:40.174135+0000","last_clean":"2024-04-21T09:06:40.174135+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:06:40.174135+0000","last_undegraded":"2024-04-21T09:06:40.174135+0000","last_fullsized":"2024-04-21T09:06:40.174135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.166234+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:06:40.166234+0000","last_peered":"2024-04-21T09:06:40.166234+0000","last_clean":"2024-04-21T09:06:40.166234+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:06:40.166234+0000","last_undegraded":"2024-04-21T09:06:40.166234+0000","last_fullsized":"2024-04-21T09:06:40.166234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168743+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:06:40.168743+0000","last_peered":"2024-04-21T09:06:40.168743+0000","last_clean":"2024-04-21T09:06:40.168743+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:06:40.168743+0000","last_undegraded":"2024-04-21T09:06:40.168743+0000","last_fullsized":"2024-04-21T09:06:40.168743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.163985+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:06:40.163985+0000","last_peered":"2024-04-21T09:06:40.163985+0000","last_clean":"2024-04-21T09:06:40.163985+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:06:40.163985+0000","last_undegraded":"2024-04-21T09:06:40.163985+0000","last_fullsized":"2024-04-21T09:06:40.163985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169191+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:06:40.169191+0000","last_peered":"2024-04-21T09:06:40.169191+0000","last_clean":"2024-04-21T09:06:40.169191+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:06:40.169191+0000","last_undegraded":"2024-04-21T09:06:40.169191+0000","last_fullsized":"2024-04-21T09:06:40.169191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165427+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:06:40.165427+0000","last_peered":"2024-04-21T09:06:40.165427+0000","last_clean":"2024-04-21T09:06:40.165427+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:06:40.165427+0000","last_undegraded":"2024-04-21T09:06:40.165427+0000","last_fullsized":"2024-04-21T09:06:40.165427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171919+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:06:40.171919+0000","last_peered":"2024-04-21T09:06:40.171919+0000","last_clean":"2024-04-21T09:06:40.171919+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:06:40.171919+0000","last_undegraded":"2024-04-21T09:06:40.171919+0000","last_fullsized":"2024-04-21T09:06:40.171919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716249+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:06:40.716249+0000","last_peered":"2024-04-21T09:06:40.716249+0000","last_clean":"2024-04-21T09:06:40.716249+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:06:40.716249+0000","last_undegraded":"2024-04-21T09:06:40.716249+0000","last_fullsized":"2024-04-21T09:06:40.716249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":107,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:24.210308+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:07:24.210308+0000","last_peered":"2024-04-21T09:07:24.210308+0000","last_clean":"2024-04-21T09:07:24.210308+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:07:24.210308+0000","last_undegraded":"2024-04-21T09:07:24.210308+0000","last_fullsized":"2024-04-21T09:07:24.210308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170963+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:06:40.170963+0000","last_peered":"2024-04-21T09:06:40.170963+0000","last_clean":"2024-04-21T09:06:40.170963+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:06:40.170963+0000","last_undegraded":"2024-04-21T09:06:40.170963+0000","last_fullsized":"2024-04-21T09:06:40.170963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164829+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:06:40.164829+0000","last_peered":"2024-04-21T09:06:40.164829+0000","last_clean":"2024-04-21T09:06:40.164829+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:06:40.164829+0000","last_undegraded":"2024-04-21T09:06:40.164829+0000","last_fullsized":"2024-04-21T09:06:40.164829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173763+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:06:40.173763+0000","last_peered":"2024-04-21T09:06:40.173763+0000","last_clean":"2024-04-21T09:06:40.173763+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:06:40.173763+0000","last_undegraded":"2024-04-21T09:06:40.173763+0000","last_fullsized":"2024-04-21T09:06:40.173763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165077+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:06:40.165077+0000","last_peered":"2024-04-21T09:06:40.165077+0000","last_clean":"2024-04-21T09:06:40.165077+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:06:40.165077+0000","last_undegraded":"2024-04-21T09:06:40.165077+0000","last_fullsized":"2024-04-21T09:06:40.165077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716698+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:06:40.716698+0000","last_peered":"2024-04-21T09:06:40.716698+0000","last_clean":"2024-04-21T09:06:40.716698+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:06:40.716698+0000","last_undegraded":"2024-04-21T09:06:40.716698+0000","last_fullsized":"2024-04-21T09:06:40.716698+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.186783+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:06:40.186783+0000","last_peered":"2024-04-21T09:06:40.186783+0000","last_clean":"2024-04-21T09:06:40.186783+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:06:40.186783+0000","last_undegraded":"2024-04-21T09:06:40.186783+0000","last_fullsized":"2024-04-21T09:06:40.186783+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169546+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:06:40.169546+0000","last_peered":"2024-04-21T09:06:40.169546+0000","last_clean":"2024-04-21T09:06:40.169546+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:06:40.169546+0000","last_undegraded":"2024-04-21T09:06:40.169546+0000","last_fullsized":"2024-04-21T09:06:40.169546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174238+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:06:40.174238+0000","last_peered":"2024-04-21T09:06:40.174238+0000","last_clean":"2024-04-21T09:06:40.174238+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:06:40.174238+0000","last_undegraded":"2024-04-21T09:06:40.174238+0000","last_fullsized":"2024-04-21T09:06:40.174238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716251+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:06:40.716251+0000","last_peered":"2024-04-21T09:06:40.716251+0000","last_clean":"2024-04-21T09:06:40.716251+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:06:40.716251+0000","last_undegraded":"2024-04-21T09:06:40.716251+0000","last_fullsized":"2024-04-21T09:06:40.716251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167342+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:06:40.167342+0000","last_peered":"2024-04-21T09:06:40.167342+0000","last_clean":"2024-04-21T09:06:40.167342+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:06:40.167342+0000","last_undegraded":"2024-04-21T09:06:40.167342+0000","last_fullsized":"2024-04-21T09:06:40.167342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164283+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:06:40.164283+0000","last_peered":"2024-04-21T09:06:40.164283+0000","last_clean":"2024-04-21T09:06:40.164283+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:06:40.164283+0000","last_undegraded":"2024-04-21T09:06:40.164283+0000","last_fullsized":"2024-04-21T09:06:40.164283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171579+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:06:40.171579+0000","last_peered":"2024-04-21T09:06:40.171579+0000","last_clean":"2024-04-21T09:06:40.171579+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:06:40.171579+0000","last_undegraded":"2024-04-21T09:06:40.171579+0000","last_fullsized":"2024-04-21T09:06:40.171579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167699+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:06:40.167699+0000","last_peered":"2024-04-21T09:06:40.167699+0000","last_clean":"2024-04-21T09:06:40.167699+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:06:40.167699+0000","last_undegraded":"2024-04-21T09:06:40.167699+0000","last_fullsized":"2024-04-21T09:06:40.167699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164551+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:06:40.164551+0000","last_peered":"2024-04-21T09:06:40.164551+0000","last_clean":"2024-04-21T09:06:40.164551+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:06:40.164551+0000","last_undegraded":"2024-04-21T09:06:40.164551+0000","last_fullsized":"2024-04-21T09:06:40.164551+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165229+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:06:40.165229+0000","last_peered":"2024-04-21T09:06:40.165229+0000","last_clean":"2024-04-21T09:06:40.165229+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:06:40.165229+0000","last_undegraded":"2024-04-21T09:06:40.165229+0000","last_fullsized":"2024-04-21T09:06:40.165229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:21.204869+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:07:21.204869+0000","last_peered":"2024-04-21T09:07:21.204869+0000","last_clean":"2024-04-21T09:07:21.204869+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:07:21.204869+0000","last_undegraded":"2024-04-21T09:07:21.204869+0000","last_fullsized":"2024-04-21T09:07:21.204869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171313+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:06:40.171313+0000","last_peered":"2024-04-21T09:06:40.171313+0000","last_clean":"2024-04-21T09:06:40.171313+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:06:40.171313+0000","last_undegraded":"2024-04-21T09:06:40.171313+0000","last_fullsized":"2024-04-21T09:06:40.171313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715555+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:06:40.715555+0000","last_peered":"2024-04-21T09:06:40.715555+0000","last_clean":"2024-04-21T09:06:40.715555+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:06:40.715555+0000","last_undegraded":"2024-04-21T09:06:40.715555+0000","last_fullsized":"2024-04-21T09:06:40.715555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164710+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:06:40.164710+0000","last_peered":"2024-04-21T09:06:40.164710+0000","last_clean":"2024-04-21T09:06:40.164710+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:06:40.164710+0000","last_undegraded":"2024-04-21T09:06:40.164710+0000","last_fullsized":"2024-04-21T09:06:40.164710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170796+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:06:40.170796+0000","last_peered":"2024-04-21T09:06:40.170796+0000","last_clean":"2024-04-21T09:06:40.170796+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:06:40.170796+0000","last_undegraded":"2024-04-21T09:06:40.170796+0000","last_fullsized":"2024-04-21T09:06:40.170796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.176390+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:06:40.176390+0000","last_peered":"2024-04-21T09:06:40.176390+0000","last_clean":"2024-04-21T09:06:40.176390+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:06:40.176390+0000","last_undegraded":"2024-04-21T09:06:40.176390+0000","last_fullsized":"2024-04-21T09:06:40.176390+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.718126+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:06:40.718126+0000","last_peered":"2024-04-21T09:06:40.718126+0000","last_clean":"2024-04-21T09:06:40.718126+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:06:40.718126+0000","last_undegraded":"2024-04-21T09:06:40.718126+0000","last_fullsized":"2024-04-21T09:06:40.718126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170876+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:06:40.170876+0000","last_peered":"2024-04-21T09:06:40.170876+0000","last_clean":"2024-04-21T09:06:40.170876+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:06:40.170876+0000","last_undegraded":"2024-04-21T09:06:40.170876+0000","last_fullsized":"2024-04-21T09:06:40.170876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169207+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:06:40.169207+0000","last_peered":"2024-04-21T09:06:40.169207+0000","last_clean":"2024-04-21T09:06:40.169207+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:06:40.169207+0000","last_undegraded":"2024-04-21T09:06:40.169207+0000","last_fullsized":"2024-04-21T09:06:40.169207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174235+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:06:40.174235+0000","last_peered":"2024-04-21T09:06:40.174235+0000","last_clean":"2024-04-21T09:06:40.174235+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:06:40.174235+0000","last_undegraded":"2024-04-21T09:06:40.174235+0000","last_fullsized":"2024-04-21T09:06:40.174235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164062+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:06:40.164062+0000","last_peered":"2024-04-21T09:06:40.164062+0000","last_clean":"2024-04-21T09:06:40.164062+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:06:40.164062+0000","last_undegraded":"2024-04-21T09:06:40.164062+0000","last_fullsized":"2024-04-21T09:06:40.164062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715068+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:06:40.715068+0000","last_peered":"2024-04-21T09:06:40.715068+0000","last_clean":"2024-04-21T09:06:40.715068+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:06:40.715068+0000","last_undegraded":"2024-04-21T09:06:40.715068+0000","last_fullsized":"2024-04-21T09:06:40.715068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716546+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:06:40.716546+0000","last_peered":"2024-04-21T09:06:40.716546+0000","last_clean":"2024-04-21T09:06:40.716546+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:06:40.716546+0000","last_undegraded":"2024-04-21T09:06:40.716546+0000","last_fullsized":"2024-04-21T09:06:40.716546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716682+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:06:40.716682+0000","last_peered":"2024-04-21T09:06:40.716682+0000","last_clean":"2024-04-21T09:06:40.716682+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:06:40.716682+0000","last_undegraded":"2024-04-21T09:06:40.716682+0000","last_fullsized":"2024-04-21T09:06:40.716682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168608+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:06:40.168608+0000","last_peered":"2024-04-21T09:06:40.168608+0000","last_clean":"2024-04-21T09:06:40.168608+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:06:40.168608+0000","last_undegraded":"2024-04-21T09:06:40.168608+0000","last_fullsized":"2024-04-21T09:06:40.168608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165322+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:06:40.165322+0000","last_peered":"2024-04-21T09:06:40.165322+0000","last_clean":"2024-04-21T09:06:40.165322+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:06:40.165322+0000","last_undegraded":"2024-04-21T09:06:40.165322+0000","last_fullsized":"2024-04-21T09:06:40.165322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172651+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:06:40.172651+0000","last_peered":"2024-04-21T09:06:40.172651+0000","last_clean":"2024-04-21T09:06:40.172651+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:06:40.172651+0000","last_undegraded":"2024-04-21T09:06:40.172651+0000","last_fullsized":"2024-04-21T09:06:40.172651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165253+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:06:40.165253+0000","last_peered":"2024-04-21T09:06:40.165253+0000","last_clean":"2024-04-21T09:06:40.165253+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:06:40.165253+0000","last_undegraded":"2024-04-21T09:06:40.165253+0000","last_fullsized":"2024-04-21T09:06:40.165253+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168229+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:06:40.168229+0000","last_peered":"2024-04-21T09:06:40.168229+0000","last_clean":"2024-04-21T09:06:40.168229+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:06:40.168229+0000","last_undegraded":"2024-04-21T09:06:40.168229+0000","last_fullsized":"2024-04-21T09:06:40.168229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173892+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:06:40.173892+0000","last_peered":"2024-04-21T09:06:40.173892+0000","last_clean":"2024-04-21T09:06:40.173892+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:06:40.173892+0000","last_undegraded":"2024-04-21T09:06:40.173892+0000","last_fullsized":"2024-04-21T09:06:40.173892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716498+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:06:40.716498+0000","last_peered":"2024-04-21T09:06:40.716498+0000","last_clean":"2024-04-21T09:06:40.716498+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:06:40.716498+0000","last_undegraded":"2024-04-21T09:06:40.716498+0000","last_fullsized":"2024-04-21T09:06:40.716498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715588+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:06:40.715588+0000","last_peered":"2024-04-21T09:06:40.715588+0000","last_clean":"2024-04-21T09:06:40.715588+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:06:40.715588+0000","last_undegraded":"2024-04-21T09:06:40.715588+0000","last_fullsized":"2024-04-21T09:06:40.715588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169654+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:06:40.169654+0000","last_peered":"2024-04-21T09:06:40.169654+0000","last_clean":"2024-04-21T09:06:40.169654+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:06:40.169654+0000","last_undegraded":"2024-04-21T09:06:40.169654+0000","last_fullsized":"2024-04-21T09:06:40.169654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164935+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:06:40.164935+0000","last_peered":"2024-04-21T09:06:40.164935+0000","last_clean":"2024-04-21T09:06:40.164935+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:06:40.164935+0000","last_undegraded":"2024-04-21T09:06:40.164935+0000","last_fullsized":"2024-04-21T09:06:40.164935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168506+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:06:40.168506+0000","last_peered":"2024-04-21T09:06:40.168506+0000","last_clean":"2024-04-21T09:06:40.168506+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:06:40.168506+0000","last_undegraded":"2024-04-21T09:06:40.168506+0000","last_fullsized":"2024-04-21T09:06:40.168506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716103+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:06:40.716103+0000","last_peered":"2024-04-21T09:06:40.716103+0000","last_clean":"2024-04-21T09:06:40.716103+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:06:40.716103+0000","last_undegraded":"2024-04-21T09:06:40.716103+0000","last_fullsized":"2024-04-21T09:06:40.716103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165101+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:06:40.165101+0000","last_peered":"2024-04-21T09:06:40.165101+0000","last_clean":"2024-04-21T09:06:40.165101+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:06:40.165101+0000","last_undegraded":"2024-04-21T09:06:40.165101+0000","last_fullsized":"2024-04-21T09:06:40.165101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.183343+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:06:40.183343+0000","last_peered":"2024-04-21T09:06:40.183343+0000","last_clean":"2024-04-21T09:06:40.183343+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:06:40.183343+0000","last_undegraded":"2024-04-21T09:06:40.183343+0000","last_fullsized":"2024-04-21T09:06:40.183343+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171731+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:06:40.171731+0000","last_peered":"2024-04-21T09:06:40.171731+0000","last_clean":"2024-04-21T09:06:40.171731+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:06:40.171731+0000","last_undegraded":"2024-04-21T09:06:40.171731+0000","last_fullsized":"2024-04-21T09:06:40.171731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168209+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:06:40.168209+0000","last_peered":"2024-04-21T09:06:40.168209+0000","last_clean":"2024-04-21T09:06:40.168209+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:06:40.168209+0000","last_undegraded":"2024-04-21T09:06:40.168209+0000","last_fullsized":"2024-04-21T09:06:40.168209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.181398+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:06:40.181398+0000","last_peered":"2024-04-21T09:06:40.181398+0000","last_clean":"2024-04-21T09:06:40.181398+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:06:40.181398+0000","last_undegraded":"2024-04-21T09:06:40.181398+0000","last_fullsized":"2024-04-21T09:06:40.181398+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171651+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:06:40.171651+0000","last_peered":"2024-04-21T09:06:40.171651+0000","last_clean":"2024-04-21T09:06:40.171651+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:06:40.171651+0000","last_undegraded":"2024-04-21T09:06:40.171651+0000","last_fullsized":"2024-04-21T09:06:40.171651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171846+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:06:40.171846+0000","last_peered":"2024-04-21T09:06:40.171846+0000","last_clean":"2024-04-21T09:06:40.171846+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:06:40.171846+0000","last_undegraded":"2024-04-21T09:06:40.171846+0000","last_fullsized":"2024-04-21T09:06:40.171846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165864+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:06:40.165864+0000","last_peered":"2024-04-21T09:06:40.165864+0000","last_clean":"2024-04-21T09:06:40.165864+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:06:40.165864+0000","last_undegraded":"2024-04-21T09:06:40.165864+0000","last_fullsized":"2024-04-21T09:06:40.165864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171374+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:06:40.171374+0000","last_peered":"2024-04-21T09:06:40.171374+0000","last_clean":"2024-04-21T09:06:40.171374+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:06:40.171374+0000","last_undegraded":"2024-04-21T09:06:40.171374+0000","last_fullsized":"2024-04-21T09:06:40.171374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164786+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:06:40.164786+0000","last_peered":"2024-04-21T09:06:40.164786+0000","last_clean":"2024-04-21T09:06:40.164786+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:06:40.164786+0000","last_undegraded":"2024-04-21T09:06:40.164786+0000","last_fullsized":"2024-04-21T09:06:40.164786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164715+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:06:40.164715+0000","last_peered":"2024-04-21T09:06:40.164715+0000","last_clean":"2024-04-21T09:06:40.164715+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:06:40.164715+0000","last_undegraded":"2024-04-21T09:06:40.164715+0000","last_fullsized":"2024-04-21T09:06:40.164715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171652+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:06:40.171652+0000","last_peered":"2024-04-21T09:06:40.171652+0000","last_clean":"2024-04-21T09:06:40.171652+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:06:40.171652+0000","last_undegraded":"2024-04-21T09:06:40.171652+0000","last_fullsized":"2024-04-21T09:06:40.171652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168845+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:06:40.168845+0000","last_peered":"2024-04-21T09:06:40.168845+0000","last_clean":"2024-04-21T09:06:40.168845+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:06:40.168845+0000","last_undegraded":"2024-04-21T09:06:40.168845+0000","last_fullsized":"2024-04-21T09:06:40.168845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168828+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:06:40.168828+0000","last_peered":"2024-04-21T09:06:40.168828+0000","last_clean":"2024-04-21T09:06:40.168828+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:06:40.168828+0000","last_undegraded":"2024-04-21T09:06:40.168828+0000","last_fullsized":"2024-04-21T09:06:40.168828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168385+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:06:40.168385+0000","last_peered":"2024-04-21T09:06:40.168385+0000","last_clean":"2024-04-21T09:06:40.168385+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:06:40.168385+0000","last_undegraded":"2024-04-21T09:06:40.168385+0000","last_fullsized":"2024-04-21T09:06:40.168385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172151+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:06:40.172151+0000","last_peered":"2024-04-21T09:06:40.172151+0000","last_clean":"2024-04-21T09:06:40.172151+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:06:40.172151+0000","last_undegraded":"2024-04-21T09:06:40.172151+0000","last_fullsized":"2024-04-21T09:06:40.172151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.163949+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:06:40.163949+0000","last_peered":"2024-04-21T09:06:40.163949+0000","last_clean":"2024-04-21T09:06:40.163949+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:06:40.163949+0000","last_undegraded":"2024-04-21T09:06:40.163949+0000","last_fullsized":"2024-04-21T09:06:40.163949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169626+0000","last_change":"2024-04-21T09:05:46.972797+0000","last_active":"2024-04-21T09:06:40.169626+0000","last_peered":"2024-04-21T09:06:40.169626+0000","last_clean":"2024-04-21T09:06:40.169626+0000","last_became_active":"2024-04-21T09:05:46.972465+0000","last_became_peered":"2024-04-21T09:05:46.972465+0000","last_unstale":"2024-04-21T09:06:40.169626+0000","last_undegraded":"2024-04-21T09:06:40.169626+0000","last_fullsized":"2024-04-21T09:06:40.169626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:29:45.732617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169788+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:06:40.169788+0000","last_peered":"2024-04-21T09:06:40.169788+0000","last_clean":"2024-04-21T09:06:40.169788+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:06:40.169788+0000","last_undegraded":"2024-04-21T09:06:40.169788+0000","last_fullsized":"2024-04-21T09:06:40.169788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716834+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:06:40.716834+0000","last_peered":"2024-04-21T09:06:40.716834+0000","last_clean":"2024-04-21T09:06:40.716834+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:06:40.716834+0000","last_undegraded":"2024-04-21T09:06:40.716834+0000","last_fullsized":"2024-04-21T09:06:40.716834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165173+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:06:40.165173+0000","last_peered":"2024-04-21T09:06:40.165173+0000","last_clean":"2024-04-21T09:06:40.165173+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:06:40.165173+0000","last_undegraded":"2024-04-21T09:06:40.165173+0000","last_fullsized":"2024-04-21T09:06:40.165173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715595+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:06:40.715595+0000","last_peered":"2024-04-21T09:06:40.715595+0000","last_clean":"2024-04-21T09:06:40.715595+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:06:40.715595+0000","last_undegraded":"2024-04-21T09:06:40.715595+0000","last_fullsized":"2024-04-21T09:06:40.715595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169448+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:06:40.169448+0000","last_peered":"2024-04-21T09:06:40.169448+0000","last_clean":"2024-04-21T09:06:40.169448+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:06:40.169448+0000","last_undegraded":"2024-04-21T09:06:40.169448+0000","last_fullsized":"2024-04-21T09:06:40.169448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164147+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:06:40.164147+0000","last_peered":"2024-04-21T09:06:40.164147+0000","last_clean":"2024-04-21T09:06:40.164147+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:06:40.164147+0000","last_undegraded":"2024-04-21T09:06:40.164147+0000","last_fullsized":"2024-04-21T09:06:40.164147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169492+0000","last_change":"2024-04-21T09:05:46.988031+0000","last_active":"2024-04-21T09:06:40.169492+0000","last_peered":"2024-04-21T09:06:40.169492+0000","last_clean":"2024-04-21T09:06:40.169492+0000","last_became_active":"2024-04-21T09:05:46.987629+0000","last_became_peered":"2024-04-21T09:05:46.987629+0000","last_unstale":"2024-04-21T09:06:40.169492+0000","last_undegraded":"2024-04-21T09:06:40.169492+0000","last_fullsized":"2024-04-21T09:06:40.169492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:33.326914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210748,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65385,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":444,"num_read_kb":292,"num_write":158,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196155,"data_compressed":4893,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":513295,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32312,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712840,"statfs":{"total":95995035648,"available":95961948160,"internally_reserved":0,"allocated":1155072,"data_stored":1162316,"data_compressed":60834,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":517605,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":513407,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":1108947,"data_compressed":59203,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":508685,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":570952,"data_compressed":25163,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":510073,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":570516,"data_compressed":25163,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":508879,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":513957,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":1104535,"data_compressed":59203,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62357,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61921,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63353,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:07:27.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T09:07:27.366 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:27.758 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:27 smithi066 ceph-mon[27953]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:27.758 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:27 smithi066 ceph-mon[27953]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:07:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:27 smithi186 ceph-mon[31505]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:27 smithi186 ceph-mon[31505]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:07:27.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:27 smithi083 ceph-mon[31413]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:27.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:27 smithi083 ceph-mon[31413]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:07:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 -- 172.21.15.66:0/3425978333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb22406b1f0 msgr2=0x7fb22406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 --2- 172.21.15.66:0/3425978333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb22406b1f0 0x7fb22406b670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb218009990 tx=0x7fb218009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 -- 172.21.15.66:0/3425978333 shutdown_connections 2024-04-21T09:07:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 --2- 172.21.15.66:0/3425978333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb224108a60 0x7fb224108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 --2- 172.21.15.66:0/3425978333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22406bbb0 0x7fb22410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 --2- 172.21.15.66:0/3425978333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb22406b1f0 0x7fb22406b670 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 -- 172.21.15.66:0/3425978333 >> 172.21.15.66:0/3425978333 conn(0x7fb2240789e0 msgr2=0x7fb224078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 -- 172.21.15.66:0/3425978333 shutdown_connections 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.047+0000 7fb229fe5700 1 -- 172.21.15.66:0/3425978333 wait complete. 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 Processor -- start 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 -- start start 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22406b1f0 0x7fb22419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22406bbb0 0x7fb22419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb224108a60 0x7fb22419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb224112cf0 con 0x7fb224108a60 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb224112b70 con 0x7fb22406b1f0 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.048+0000 7fb229fe5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb224112e70 con 0x7fb22406bbb0 2024-04-21T09:07:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb224108a60 0x7fb22419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb224108a60 0x7fb22419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56052/0 (socket says 172.21.15.66:56052) 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 -- 172.21.15.66:0/1778136426 learned_addr learned my addr 172.21.15.66:0/1778136426 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb2237fe700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22406b1f0 0x7fb22419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb222ffd700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22406bbb0 0x7fb22419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 -- 172.21.15.66:0/1778136426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22406bbb0 msgr2=0x7fb22419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22406bbb0 0x7fb22419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 -- 172.21.15.66:0/1778136426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22406b1f0 msgr2=0x7fb22419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22406b1f0 0x7fb22419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 -- 172.21.15.66:0/1778136426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2241a0330 con 0x7fb224108a60 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb223fff700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb224108a60 0x7fb22419fbf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007d70 tx=0x7fb21c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb220ff9700 1 -- 172.21.15.66:0/1778136426 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c016020 con 0x7fb224108a60 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2241a0620 con 0x7fb224108a60 2024-04-21T09:07:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb220ff9700 1 -- 172.21.15.66:0/1778136426 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb21c00ad40 con 0x7fb224108a60 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb220ff9700 1 -- 172.21.15.66:0/1778136426 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0115e0 con 0x7fb224108a60 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.049+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2241a7ea0 con 0x7fb224108a60 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.050+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb224069730 con 0x7fb224108a60 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.052+0000 7fb220ff9700 1 -- 172.21.15.66:0/1778136426 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb21c019020 con 0x7fb224108a60 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.052+0000 7fb220ff9700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb20c0a3cd0 0x7fb20c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.052+0000 7fb2237fe700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb20c0a3cd0 0x7fb20c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.052+0000 7fb220ff9700 1 -- 172.21.15.66:0/1778136426 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fb21c0d0cf0 con 0x7fb224108a60 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.053+0000 7fb2237fe700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb20c0a3cd0 0x7fb20c0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb214005950 tx=0x7fb21400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.056+0000 7fb220ff9700 1 -- 172.21.15.66:0/1778136426 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb21c020020 con 0x7fb224108a60 2024-04-21T09:07:29.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.261+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb224062e80 con 0x7fb20c0a3cd0 2024-04-21T09:07:29.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.270+0000 7fb220ff9700 1 -- 172.21.15.66:0/1778136426 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340639 (secure 0 0 0) 0x7fb224062e80 con 0x7fb20c0a3cd0 2024-04-21T09:07:29.278 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:29.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.273+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb20c0a3cd0 msgr2=0x7fb20c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:29.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.273+0000 7fb229fe5700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb20c0a3cd0 0x7fb20c0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb214005950 tx=0x7fb21400b410 comp rx=0 tx=0).stop 2024-04-21T09:07:29.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.273+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb224108a60 msgr2=0x7fb22419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:29.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.273+0000 7fb229fe5700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb224108a60 0x7fb22419fbf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007d70 tx=0x7fb21c00a340 comp rx=0 tx=0).stop 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 shutdown_connections 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb20c0a3cd0 0x7fb20c0a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22406b1f0 0x7fb22419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb224108a60 0x7fb22419fbf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 --2- 172.21.15.66:0/1778136426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22406bbb0 0x7fb22419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 >> 172.21.15.66:0/1778136426 conn(0x7fb2240789e0 msgr2=0x7fb22410dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 shutdown_connections 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:29.274+0000 7fb229fe5700 1 -- 172.21.15.66:0/1778136426 wait complete. 2024-04-21T09:07:29.281 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:07:29.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:29 smithi186 ceph-mon[31505]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:29 smithi083 ceph-mon[31413]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:29.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:29 smithi066 ceph-mon[27953]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:29.979 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-04-21T09:07:28.323289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655987,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":590,"num_read_kb":416,"num_write":341,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357436,"kb_used_data":12380,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584388,"statfs":{"total":1151940427776,"available":1151574413312,"internally_reserved":0,"allocated":12677120,"data_stored":8103167,"data_compressed":394290,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003603"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169425+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:06:40.169425+0000","last_peered":"2024-04-21T09:06:40.169425+0000","last_clean":"2024-04-21T09:06:40.169425+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:06:40.169425+0000","last_undegraded":"2024-04-21T09:06:40.169425+0000","last_fullsized":"2024-04-21T09:06:40.169425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172692+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:06:40.172692+0000","last_peered":"2024-04-21T09:06:40.172692+0000","last_clean":"2024-04-21T09:06:40.172692+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:06:40.172692+0000","last_undegraded":"2024-04-21T09:06:40.172692+0000","last_fullsized":"2024-04-21T09:06:40.172692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172529+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:06:40.172529+0000","last_peered":"2024-04-21T09:06:40.172529+0000","last_clean":"2024-04-21T09:06:40.172529+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:06:40.172529+0000","last_undegraded":"2024-04-21T09:06:40.172529+0000","last_fullsized":"2024-04-21T09:06:40.172529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165192+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:06:40.165192+0000","last_peered":"2024-04-21T09:06:40.165192+0000","last_clean":"2024-04-21T09:06:40.165192+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:06:40.165192+0000","last_undegraded":"2024-04-21T09:06:40.165192+0000","last_fullsized":"2024-04-21T09:06:40.165192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173412+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:06:40.173412+0000","last_peered":"2024-04-21T09:06:40.173412+0000","last_clean":"2024-04-21T09:06:40.173412+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:06:40.173412+0000","last_undegraded":"2024-04-21T09:06:40.173412+0000","last_fullsized":"2024-04-21T09:06:40.173412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167298+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:06:40.167298+0000","last_peered":"2024-04-21T09:06:40.167298+0000","last_clean":"2024-04-21T09:06:40.167298+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:06:40.167298+0000","last_undegraded":"2024-04-21T09:06:40.167298+0000","last_fullsized":"2024-04-21T09:06:40.167298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171293+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:06:40.171293+0000","last_peered":"2024-04-21T09:06:40.171293+0000","last_clean":"2024-04-21T09:06:40.171293+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:06:40.171293+0000","last_undegraded":"2024-04-21T09:06:40.171293+0000","last_fullsized":"2024-04-21T09:06:40.171293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715065+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:06:40.715065+0000","last_peered":"2024-04-21T09:06:40.715065+0000","last_clean":"2024-04-21T09:06:40.715065+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:06:40.715065+0000","last_undegraded":"2024-04-21T09:06:40.715065+0000","last_fullsized":"2024-04-21T09:06:40.715065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169019+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:06:40.169019+0000","last_peered":"2024-04-21T09:06:40.169019+0000","last_clean":"2024-04-21T09:06:40.169019+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:06:40.169019+0000","last_undegraded":"2024-04-21T09:06:40.169019+0000","last_fullsized":"2024-04-21T09:06:40.169019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168208+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:06:40.168208+0000","last_peered":"2024-04-21T09:06:40.168208+0000","last_clean":"2024-04-21T09:06:40.168208+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:06:40.168208+0000","last_undegraded":"2024-04-21T09:06:40.168208+0000","last_fullsized":"2024-04-21T09:06:40.168208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171466+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:06:40.171466+0000","last_peered":"2024-04-21T09:06:40.171466+0000","last_clean":"2024-04-21T09:06:40.171466+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:06:40.171466+0000","last_undegraded":"2024-04-21T09:06:40.171466+0000","last_fullsized":"2024-04-21T09:06:40.171466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165088+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:06:40.165088+0000","last_peered":"2024-04-21T09:06:40.165088+0000","last_clean":"2024-04-21T09:06:40.165088+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:06:40.165088+0000","last_undegraded":"2024-04-21T09:06:40.165088+0000","last_fullsized":"2024-04-21T09:06:40.165088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164047+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:06:40.164047+0000","last_peered":"2024-04-21T09:06:40.164047+0000","last_clean":"2024-04-21T09:06:40.164047+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:06:40.164047+0000","last_undegraded":"2024-04-21T09:06:40.164047+0000","last_fullsized":"2024-04-21T09:06:40.164047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171830+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:06:40.171830+0000","last_peered":"2024-04-21T09:06:40.171830+0000","last_clean":"2024-04-21T09:06:40.171830+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:06:40.171830+0000","last_undegraded":"2024-04-21T09:06:40.171830+0000","last_fullsized":"2024-04-21T09:06:40.171830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172027+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:06:40.172027+0000","last_peered":"2024-04-21T09:06:40.172027+0000","last_clean":"2024-04-21T09:06:40.172027+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:06:40.172027+0000","last_undegraded":"2024-04-21T09:06:40.172027+0000","last_fullsized":"2024-04-21T09:06:40.172027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168275+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:06:40.168275+0000","last_peered":"2024-04-21T09:06:40.168275+0000","last_clean":"2024-04-21T09:06:40.168275+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:06:40.168275+0000","last_undegraded":"2024-04-21T09:06:40.168275+0000","last_fullsized":"2024-04-21T09:06:40.168275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169143+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:06:40.169143+0000","last_peered":"2024-04-21T09:06:40.169143+0000","last_clean":"2024-04-21T09:06:40.169143+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:06:40.169143+0000","last_undegraded":"2024-04-21T09:06:40.169143+0000","last_fullsized":"2024-04-21T09:06:40.169143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716653+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:06:40.716653+0000","last_peered":"2024-04-21T09:06:40.716653+0000","last_clean":"2024-04-21T09:06:40.716653+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:06:40.716653+0000","last_undegraded":"2024-04-21T09:06:40.716653+0000","last_fullsized":"2024-04-21T09:06:40.716653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170406+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:06:40.170406+0000","last_peered":"2024-04-21T09:06:40.170406+0000","last_clean":"2024-04-21T09:06:40.170406+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:06:40.170406+0000","last_undegraded":"2024-04-21T09:06:40.170406+0000","last_fullsized":"2024-04-21T09:06:40.170406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170256+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:06:40.170256+0000","last_peered":"2024-04-21T09:06:40.170256+0000","last_clean":"2024-04-21T09:06:40.170256+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:06:40.170256+0000","last_undegraded":"2024-04-21T09:06:40.170256+0000","last_fullsized":"2024-04-21T09:06:40.170256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716638+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:06:40.716638+0000","last_peered":"2024-04-21T09:06:40.716638+0000","last_clean":"2024-04-21T09:06:40.716638+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:06:40.716638+0000","last_undegraded":"2024-04-21T09:06:40.716638+0000","last_fullsized":"2024-04-21T09:06:40.716638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715807+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:06:40.715807+0000","last_peered":"2024-04-21T09:06:40.715807+0000","last_clean":"2024-04-21T09:06:40.715807+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:06:40.715807+0000","last_undegraded":"2024-04-21T09:06:40.715807+0000","last_fullsized":"2024-04-21T09:06:40.715807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164569+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:06:40.164569+0000","last_peered":"2024-04-21T09:06:40.164569+0000","last_clean":"2024-04-21T09:06:40.164569+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:06:40.164569+0000","last_undegraded":"2024-04-21T09:06:40.164569+0000","last_fullsized":"2024-04-21T09:06:40.164569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716132+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:06:40.716132+0000","last_peered":"2024-04-21T09:06:40.716132+0000","last_clean":"2024-04-21T09:06:40.716132+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:06:40.716132+0000","last_undegraded":"2024-04-21T09:06:40.716132+0000","last_fullsized":"2024-04-21T09:06:40.716132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172602+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:06:40.172602+0000","last_peered":"2024-04-21T09:06:40.172602+0000","last_clean":"2024-04-21T09:06:40.172602+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:06:40.172602+0000","last_undegraded":"2024-04-21T09:06:40.172602+0000","last_fullsized":"2024-04-21T09:06:40.172602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.717725+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:06:40.717725+0000","last_peered":"2024-04-21T09:06:40.717725+0000","last_clean":"2024-04-21T09:06:40.717725+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:06:40.717725+0000","last_undegraded":"2024-04-21T09:06:40.717725+0000","last_fullsized":"2024-04-21T09:06:40.717725+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174392+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:06:40.174392+0000","last_peered":"2024-04-21T09:06:40.174392+0000","last_clean":"2024-04-21T09:06:40.174392+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:06:40.174392+0000","last_undegraded":"2024-04-21T09:06:40.174392+0000","last_fullsized":"2024-04-21T09:06:40.174392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169896+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:06:40.169896+0000","last_peered":"2024-04-21T09:06:40.169896+0000","last_clean":"2024-04-21T09:06:40.169896+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:06:40.169896+0000","last_undegraded":"2024-04-21T09:06:40.169896+0000","last_fullsized":"2024-04-21T09:06:40.169896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164573+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:06:40.164573+0000","last_peered":"2024-04-21T09:06:40.164573+0000","last_clean":"2024-04-21T09:06:40.164573+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:06:40.164573+0000","last_undegraded":"2024-04-21T09:06:40.164573+0000","last_fullsized":"2024-04-21T09:06:40.164573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173290+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:06:40.173290+0000","last_peered":"2024-04-21T09:06:40.173290+0000","last_clean":"2024-04-21T09:06:40.173290+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:06:40.173290+0000","last_undegraded":"2024-04-21T09:06:40.173290+0000","last_fullsized":"2024-04-21T09:06:40.173290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165679+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:06:40.165679+0000","last_peered":"2024-04-21T09:06:40.165679+0000","last_clean":"2024-04-21T09:06:40.165679+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:06:40.165679+0000","last_undegraded":"2024-04-21T09:06:40.165679+0000","last_fullsized":"2024-04-21T09:06:40.165679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170155+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:06:40.170155+0000","last_peered":"2024-04-21T09:06:40.170155+0000","last_clean":"2024-04-21T09:06:40.170155+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:06:40.170155+0000","last_undegraded":"2024-04-21T09:06:40.170155+0000","last_fullsized":"2024-04-21T09:06:40.170155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":123,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:26.244992+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:07:26.244992+0000","last_peered":"2024-04-21T09:07:26.244992+0000","last_clean":"2024-04-21T09:07:26.244992+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:07:26.244992+0000","last_undegraded":"2024-04-21T09:07:26.244992+0000","last_fullsized":"2024-04-21T09:07:26.244992+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174355+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:06:40.174355+0000","last_peered":"2024-04-21T09:06:40.174355+0000","last_clean":"2024-04-21T09:06:40.174355+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:06:40.174355+0000","last_undegraded":"2024-04-21T09:06:40.174355+0000","last_fullsized":"2024-04-21T09:06:40.174355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169074+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:06:40.169074+0000","last_peered":"2024-04-21T09:06:40.169074+0000","last_clean":"2024-04-21T09:06:40.169074+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:06:40.169074+0000","last_undegraded":"2024-04-21T09:06:40.169074+0000","last_fullsized":"2024-04-21T09:06:40.169074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167192+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:06:40.167192+0000","last_peered":"2024-04-21T09:06:40.167192+0000","last_clean":"2024-04-21T09:06:40.167192+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:06:40.167192+0000","last_undegraded":"2024-04-21T09:06:40.167192+0000","last_fullsized":"2024-04-21T09:06:40.167192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165915+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:06:40.165915+0000","last_peered":"2024-04-21T09:06:40.165915+0000","last_clean":"2024-04-21T09:06:40.165915+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:06:40.165915+0000","last_undegraded":"2024-04-21T09:06:40.165915+0000","last_fullsized":"2024-04-21T09:06:40.165915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169006+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:06:40.169006+0000","last_peered":"2024-04-21T09:06:40.169006+0000","last_clean":"2024-04-21T09:06:40.169006+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:06:40.169006+0000","last_undegraded":"2024-04-21T09:06:40.169006+0000","last_fullsized":"2024-04-21T09:06:40.169006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.163779+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:06:40.163779+0000","last_peered":"2024-04-21T09:06:40.163779+0000","last_clean":"2024-04-21T09:06:40.163779+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:06:40.163779+0000","last_undegraded":"2024-04-21T09:06:40.163779+0000","last_fullsized":"2024-04-21T09:06:40.163779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715275+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:06:40.715275+0000","last_peered":"2024-04-21T09:06:40.715275+0000","last_clean":"2024-04-21T09:06:40.715275+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:06:40.715275+0000","last_undegraded":"2024-04-21T09:06:40.715275+0000","last_fullsized":"2024-04-21T09:06:40.715275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172570+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:06:40.172570+0000","last_peered":"2024-04-21T09:06:40.172570+0000","last_clean":"2024-04-21T09:06:40.172570+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:06:40.172570+0000","last_undegraded":"2024-04-21T09:06:40.172570+0000","last_fullsized":"2024-04-21T09:06:40.172570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165095+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:06:40.165095+0000","last_peered":"2024-04-21T09:06:40.165095+0000","last_clean":"2024-04-21T09:06:40.165095+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:06:40.165095+0000","last_undegraded":"2024-04-21T09:06:40.165095+0000","last_fullsized":"2024-04-21T09:06:40.165095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716260+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:06:40.716260+0000","last_peered":"2024-04-21T09:06:40.716260+0000","last_clean":"2024-04-21T09:06:40.716260+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:06:40.716260+0000","last_undegraded":"2024-04-21T09:06:40.716260+0000","last_fullsized":"2024-04-21T09:06:40.716260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715452+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:06:40.715452+0000","last_peered":"2024-04-21T09:06:40.715452+0000","last_clean":"2024-04-21T09:06:40.715452+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:06:40.715452+0000","last_undegraded":"2024-04-21T09:06:40.715452+0000","last_fullsized":"2024-04-21T09:06:40.715452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:23.239955+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:07:23.239955+0000","last_peered":"2024-04-21T09:07:23.239955+0000","last_clean":"2024-04-21T09:07:23.239955+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:07:23.239955+0000","last_undegraded":"2024-04-21T09:07:23.239955+0000","last_fullsized":"2024-04-21T09:07:23.239955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169189+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:06:40.169189+0000","last_peered":"2024-04-21T09:06:40.169189+0000","last_clean":"2024-04-21T09:06:40.169189+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:06:40.169189+0000","last_undegraded":"2024-04-21T09:06:40.169189+0000","last_fullsized":"2024-04-21T09:06:40.169189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715876+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:06:40.715876+0000","last_peered":"2024-04-21T09:06:40.715876+0000","last_clean":"2024-04-21T09:06:40.715876+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:06:40.715876+0000","last_undegraded":"2024-04-21T09:06:40.715876+0000","last_fullsized":"2024-04-21T09:06:40.715876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171413+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:06:40.171413+0000","last_peered":"2024-04-21T09:06:40.171413+0000","last_clean":"2024-04-21T09:06:40.171413+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:06:40.171413+0000","last_undegraded":"2024-04-21T09:06:40.171413+0000","last_fullsized":"2024-04-21T09:06:40.171413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172723+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:06:40.172723+0000","last_peered":"2024-04-21T09:06:40.172723+0000","last_clean":"2024-04-21T09:06:40.172723+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:06:40.172723+0000","last_undegraded":"2024-04-21T09:06:40.172723+0000","last_fullsized":"2024-04-21T09:06:40.172723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.177313+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:06:40.177313+0000","last_peered":"2024-04-21T09:06:40.177313+0000","last_clean":"2024-04-21T09:06:40.177313+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:06:40.177313+0000","last_undegraded":"2024-04-21T09:06:40.177313+0000","last_fullsized":"2024-04-21T09:06:40.177313+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165470+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:06:40.165470+0000","last_peered":"2024-04-21T09:06:40.165470+0000","last_clean":"2024-04-21T09:06:40.165470+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:06:40.165470+0000","last_undegraded":"2024-04-21T09:06:40.165470+0000","last_fullsized":"2024-04-21T09:06:40.165470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.175441+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:06:40.175441+0000","last_peered":"2024-04-21T09:06:40.175441+0000","last_clean":"2024-04-21T09:06:40.175441+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:06:40.175441+0000","last_undegraded":"2024-04-21T09:06:40.175441+0000","last_fullsized":"2024-04-21T09:06:40.175441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715915+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:06:40.715915+0000","last_peered":"2024-04-21T09:06:40.715915+0000","last_clean":"2024-04-21T09:06:40.715915+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:06:40.715915+0000","last_undegraded":"2024-04-21T09:06:40.715915+0000","last_fullsized":"2024-04-21T09:06:40.715915+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":211,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:24.241532+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:07:24.241532+0000","last_peered":"2024-04-21T09:07:24.241532+0000","last_clean":"2024-04-21T09:07:24.241532+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:07:24.241532+0000","last_undegraded":"2024-04-21T09:07:24.241532+0000","last_fullsized":"2024-04-21T09:07:24.241532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61775,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":151,"num_read_kb":136,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174135+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:06:40.174135+0000","last_peered":"2024-04-21T09:06:40.174135+0000","last_clean":"2024-04-21T09:06:40.174135+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:06:40.174135+0000","last_undegraded":"2024-04-21T09:06:40.174135+0000","last_fullsized":"2024-04-21T09:06:40.174135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.166234+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:06:40.166234+0000","last_peered":"2024-04-21T09:06:40.166234+0000","last_clean":"2024-04-21T09:06:40.166234+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:06:40.166234+0000","last_undegraded":"2024-04-21T09:06:40.166234+0000","last_fullsized":"2024-04-21T09:06:40.166234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168743+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:06:40.168743+0000","last_peered":"2024-04-21T09:06:40.168743+0000","last_clean":"2024-04-21T09:06:40.168743+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:06:40.168743+0000","last_undegraded":"2024-04-21T09:06:40.168743+0000","last_fullsized":"2024-04-21T09:06:40.168743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.163985+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:06:40.163985+0000","last_peered":"2024-04-21T09:06:40.163985+0000","last_clean":"2024-04-21T09:06:40.163985+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:06:40.163985+0000","last_undegraded":"2024-04-21T09:06:40.163985+0000","last_fullsized":"2024-04-21T09:06:40.163985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169191+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:06:40.169191+0000","last_peered":"2024-04-21T09:06:40.169191+0000","last_clean":"2024-04-21T09:06:40.169191+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:06:40.169191+0000","last_undegraded":"2024-04-21T09:06:40.169191+0000","last_fullsized":"2024-04-21T09:06:40.169191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165427+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:06:40.165427+0000","last_peered":"2024-04-21T09:06:40.165427+0000","last_clean":"2024-04-21T09:06:40.165427+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:06:40.165427+0000","last_undegraded":"2024-04-21T09:06:40.165427+0000","last_fullsized":"2024-04-21T09:06:40.165427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171919+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:06:40.171919+0000","last_peered":"2024-04-21T09:06:40.171919+0000","last_clean":"2024-04-21T09:06:40.171919+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:06:40.171919+0000","last_undegraded":"2024-04-21T09:06:40.171919+0000","last_fullsized":"2024-04-21T09:06:40.171919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716249+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:06:40.716249+0000","last_peered":"2024-04-21T09:06:40.716249+0000","last_clean":"2024-04-21T09:06:40.716249+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:06:40.716249+0000","last_undegraded":"2024-04-21T09:06:40.716249+0000","last_fullsized":"2024-04-21T09:06:40.716249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":107,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:24.210308+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:07:24.210308+0000","last_peered":"2024-04-21T09:07:24.210308+0000","last_clean":"2024-04-21T09:07:24.210308+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:07:24.210308+0000","last_undegraded":"2024-04-21T09:07:24.210308+0000","last_fullsized":"2024-04-21T09:07:24.210308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170963+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:06:40.170963+0000","last_peered":"2024-04-21T09:06:40.170963+0000","last_clean":"2024-04-21T09:06:40.170963+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:06:40.170963+0000","last_undegraded":"2024-04-21T09:06:40.170963+0000","last_fullsized":"2024-04-21T09:06:40.170963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164829+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:06:40.164829+0000","last_peered":"2024-04-21T09:06:40.164829+0000","last_clean":"2024-04-21T09:06:40.164829+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:06:40.164829+0000","last_undegraded":"2024-04-21T09:06:40.164829+0000","last_fullsized":"2024-04-21T09:06:40.164829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173763+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:06:40.173763+0000","last_peered":"2024-04-21T09:06:40.173763+0000","last_clean":"2024-04-21T09:06:40.173763+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:06:40.173763+0000","last_undegraded":"2024-04-21T09:06:40.173763+0000","last_fullsized":"2024-04-21T09:06:40.173763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165077+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:06:40.165077+0000","last_peered":"2024-04-21T09:06:40.165077+0000","last_clean":"2024-04-21T09:06:40.165077+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:06:40.165077+0000","last_undegraded":"2024-04-21T09:06:40.165077+0000","last_fullsized":"2024-04-21T09:06:40.165077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716698+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:06:40.716698+0000","last_peered":"2024-04-21T09:06:40.716698+0000","last_clean":"2024-04-21T09:06:40.716698+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:06:40.716698+0000","last_undegraded":"2024-04-21T09:06:40.716698+0000","last_fullsized":"2024-04-21T09:06:40.716698+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.186783+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:06:40.186783+0000","last_peered":"2024-04-21T09:06:40.186783+0000","last_clean":"2024-04-21T09:06:40.186783+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:06:40.186783+0000","last_undegraded":"2024-04-21T09:06:40.186783+0000","last_fullsized":"2024-04-21T09:06:40.186783+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169546+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:06:40.169546+0000","last_peered":"2024-04-21T09:06:40.169546+0000","last_clean":"2024-04-21T09:06:40.169546+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:06:40.169546+0000","last_undegraded":"2024-04-21T09:06:40.169546+0000","last_fullsized":"2024-04-21T09:06:40.169546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174238+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:06:40.174238+0000","last_peered":"2024-04-21T09:06:40.174238+0000","last_clean":"2024-04-21T09:06:40.174238+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:06:40.174238+0000","last_undegraded":"2024-04-21T09:06:40.174238+0000","last_fullsized":"2024-04-21T09:06:40.174238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716251+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:06:40.716251+0000","last_peered":"2024-04-21T09:06:40.716251+0000","last_clean":"2024-04-21T09:06:40.716251+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:06:40.716251+0000","last_undegraded":"2024-04-21T09:06:40.716251+0000","last_fullsized":"2024-04-21T09:06:40.716251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167342+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:06:40.167342+0000","last_peered":"2024-04-21T09:06:40.167342+0000","last_clean":"2024-04-21T09:06:40.167342+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:06:40.167342+0000","last_undegraded":"2024-04-21T09:06:40.167342+0000","last_fullsized":"2024-04-21T09:06:40.167342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164283+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:06:40.164283+0000","last_peered":"2024-04-21T09:06:40.164283+0000","last_clean":"2024-04-21T09:06:40.164283+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:06:40.164283+0000","last_undegraded":"2024-04-21T09:06:40.164283+0000","last_fullsized":"2024-04-21T09:06:40.164283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171579+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:06:40.171579+0000","last_peered":"2024-04-21T09:06:40.171579+0000","last_clean":"2024-04-21T09:06:40.171579+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:06:40.171579+0000","last_undegraded":"2024-04-21T09:06:40.171579+0000","last_fullsized":"2024-04-21T09:06:40.171579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.167699+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:06:40.167699+0000","last_peered":"2024-04-21T09:06:40.167699+0000","last_clean":"2024-04-21T09:06:40.167699+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:06:40.167699+0000","last_undegraded":"2024-04-21T09:06:40.167699+0000","last_fullsized":"2024-04-21T09:06:40.167699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164551+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:06:40.164551+0000","last_peered":"2024-04-21T09:06:40.164551+0000","last_clean":"2024-04-21T09:06:40.164551+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:06:40.164551+0000","last_undegraded":"2024-04-21T09:06:40.164551+0000","last_fullsized":"2024-04-21T09:06:40.164551+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165229+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:06:40.165229+0000","last_peered":"2024-04-21T09:06:40.165229+0000","last_clean":"2024-04-21T09:06:40.165229+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:06:40.165229+0000","last_undegraded":"2024-04-21T09:06:40.165229+0000","last_fullsized":"2024-04-21T09:06:40.165229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:07:26.212756+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:07:26.212756+0000","last_peered":"2024-04-21T09:07:26.212756+0000","last_clean":"2024-04-21T09:07:26.212756+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:07:26.212756+0000","last_undegraded":"2024-04-21T09:07:26.212756+0000","last_fullsized":"2024-04-21T09:07:26.212756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171313+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:06:40.171313+0000","last_peered":"2024-04-21T09:06:40.171313+0000","last_clean":"2024-04-21T09:06:40.171313+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:06:40.171313+0000","last_undegraded":"2024-04-21T09:06:40.171313+0000","last_fullsized":"2024-04-21T09:06:40.171313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715555+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:06:40.715555+0000","last_peered":"2024-04-21T09:06:40.715555+0000","last_clean":"2024-04-21T09:06:40.715555+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:06:40.715555+0000","last_undegraded":"2024-04-21T09:06:40.715555+0000","last_fullsized":"2024-04-21T09:06:40.715555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164710+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:06:40.164710+0000","last_peered":"2024-04-21T09:06:40.164710+0000","last_clean":"2024-04-21T09:06:40.164710+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:06:40.164710+0000","last_undegraded":"2024-04-21T09:06:40.164710+0000","last_fullsized":"2024-04-21T09:06:40.164710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170796+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:06:40.170796+0000","last_peered":"2024-04-21T09:06:40.170796+0000","last_clean":"2024-04-21T09:06:40.170796+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:06:40.170796+0000","last_undegraded":"2024-04-21T09:06:40.170796+0000","last_fullsized":"2024-04-21T09:06:40.170796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.176390+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:06:40.176390+0000","last_peered":"2024-04-21T09:06:40.176390+0000","last_clean":"2024-04-21T09:06:40.176390+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:06:40.176390+0000","last_undegraded":"2024-04-21T09:06:40.176390+0000","last_fullsized":"2024-04-21T09:06:40.176390+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.718126+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:06:40.718126+0000","last_peered":"2024-04-21T09:06:40.718126+0000","last_clean":"2024-04-21T09:06:40.718126+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:06:40.718126+0000","last_undegraded":"2024-04-21T09:06:40.718126+0000","last_fullsized":"2024-04-21T09:06:40.718126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.170876+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:06:40.170876+0000","last_peered":"2024-04-21T09:06:40.170876+0000","last_clean":"2024-04-21T09:06:40.170876+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:06:40.170876+0000","last_undegraded":"2024-04-21T09:06:40.170876+0000","last_fullsized":"2024-04-21T09:06:40.170876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169207+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:06:40.169207+0000","last_peered":"2024-04-21T09:06:40.169207+0000","last_clean":"2024-04-21T09:06:40.169207+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:06:40.169207+0000","last_undegraded":"2024-04-21T09:06:40.169207+0000","last_fullsized":"2024-04-21T09:06:40.169207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.174235+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:06:40.174235+0000","last_peered":"2024-04-21T09:06:40.174235+0000","last_clean":"2024-04-21T09:06:40.174235+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:06:40.174235+0000","last_undegraded":"2024-04-21T09:06:40.174235+0000","last_fullsized":"2024-04-21T09:06:40.174235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164062+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:06:40.164062+0000","last_peered":"2024-04-21T09:06:40.164062+0000","last_clean":"2024-04-21T09:06:40.164062+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:06:40.164062+0000","last_undegraded":"2024-04-21T09:06:40.164062+0000","last_fullsized":"2024-04-21T09:06:40.164062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715068+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:06:40.715068+0000","last_peered":"2024-04-21T09:06:40.715068+0000","last_clean":"2024-04-21T09:06:40.715068+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:06:40.715068+0000","last_undegraded":"2024-04-21T09:06:40.715068+0000","last_fullsized":"2024-04-21T09:06:40.715068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716546+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:06:40.716546+0000","last_peered":"2024-04-21T09:06:40.716546+0000","last_clean":"2024-04-21T09:06:40.716546+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:06:40.716546+0000","last_undegraded":"2024-04-21T09:06:40.716546+0000","last_fullsized":"2024-04-21T09:06:40.716546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716682+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:06:40.716682+0000","last_peered":"2024-04-21T09:06:40.716682+0000","last_clean":"2024-04-21T09:06:40.716682+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:06:40.716682+0000","last_undegraded":"2024-04-21T09:06:40.716682+0000","last_fullsized":"2024-04-21T09:06:40.716682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168608+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:06:40.168608+0000","last_peered":"2024-04-21T09:06:40.168608+0000","last_clean":"2024-04-21T09:06:40.168608+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:06:40.168608+0000","last_undegraded":"2024-04-21T09:06:40.168608+0000","last_fullsized":"2024-04-21T09:06:40.168608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165322+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:06:40.165322+0000","last_peered":"2024-04-21T09:06:40.165322+0000","last_clean":"2024-04-21T09:06:40.165322+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:06:40.165322+0000","last_undegraded":"2024-04-21T09:06:40.165322+0000","last_fullsized":"2024-04-21T09:06:40.165322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172651+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:06:40.172651+0000","last_peered":"2024-04-21T09:06:40.172651+0000","last_clean":"2024-04-21T09:06:40.172651+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:06:40.172651+0000","last_undegraded":"2024-04-21T09:06:40.172651+0000","last_fullsized":"2024-04-21T09:06:40.172651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165253+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:06:40.165253+0000","last_peered":"2024-04-21T09:06:40.165253+0000","last_clean":"2024-04-21T09:06:40.165253+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:06:40.165253+0000","last_undegraded":"2024-04-21T09:06:40.165253+0000","last_fullsized":"2024-04-21T09:06:40.165253+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168229+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:06:40.168229+0000","last_peered":"2024-04-21T09:06:40.168229+0000","last_clean":"2024-04-21T09:06:40.168229+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:06:40.168229+0000","last_undegraded":"2024-04-21T09:06:40.168229+0000","last_fullsized":"2024-04-21T09:06:40.168229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.173892+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:06:40.173892+0000","last_peered":"2024-04-21T09:06:40.173892+0000","last_clean":"2024-04-21T09:06:40.173892+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:06:40.173892+0000","last_undegraded":"2024-04-21T09:06:40.173892+0000","last_fullsized":"2024-04-21T09:06:40.173892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716498+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:06:40.716498+0000","last_peered":"2024-04-21T09:06:40.716498+0000","last_clean":"2024-04-21T09:06:40.716498+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:06:40.716498+0000","last_undegraded":"2024-04-21T09:06:40.716498+0000","last_fullsized":"2024-04-21T09:06:40.716498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715588+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:06:40.715588+0000","last_peered":"2024-04-21T09:06:40.715588+0000","last_clean":"2024-04-21T09:06:40.715588+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:06:40.715588+0000","last_undegraded":"2024-04-21T09:06:40.715588+0000","last_fullsized":"2024-04-21T09:06:40.715588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169654+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:06:40.169654+0000","last_peered":"2024-04-21T09:06:40.169654+0000","last_clean":"2024-04-21T09:06:40.169654+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:06:40.169654+0000","last_undegraded":"2024-04-21T09:06:40.169654+0000","last_fullsized":"2024-04-21T09:06:40.169654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164935+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:06:40.164935+0000","last_peered":"2024-04-21T09:06:40.164935+0000","last_clean":"2024-04-21T09:06:40.164935+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:06:40.164935+0000","last_undegraded":"2024-04-21T09:06:40.164935+0000","last_fullsized":"2024-04-21T09:06:40.164935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168506+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:06:40.168506+0000","last_peered":"2024-04-21T09:06:40.168506+0000","last_clean":"2024-04-21T09:06:40.168506+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:06:40.168506+0000","last_undegraded":"2024-04-21T09:06:40.168506+0000","last_fullsized":"2024-04-21T09:06:40.168506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716103+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:06:40.716103+0000","last_peered":"2024-04-21T09:06:40.716103+0000","last_clean":"2024-04-21T09:06:40.716103+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:06:40.716103+0000","last_undegraded":"2024-04-21T09:06:40.716103+0000","last_fullsized":"2024-04-21T09:06:40.716103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165101+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:06:40.165101+0000","last_peered":"2024-04-21T09:06:40.165101+0000","last_clean":"2024-04-21T09:06:40.165101+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:06:40.165101+0000","last_undegraded":"2024-04-21T09:06:40.165101+0000","last_fullsized":"2024-04-21T09:06:40.165101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.183343+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:06:40.183343+0000","last_peered":"2024-04-21T09:06:40.183343+0000","last_clean":"2024-04-21T09:06:40.183343+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:06:40.183343+0000","last_undegraded":"2024-04-21T09:06:40.183343+0000","last_fullsized":"2024-04-21T09:06:40.183343+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171731+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:06:40.171731+0000","last_peered":"2024-04-21T09:06:40.171731+0000","last_clean":"2024-04-21T09:06:40.171731+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:06:40.171731+0000","last_undegraded":"2024-04-21T09:06:40.171731+0000","last_fullsized":"2024-04-21T09:06:40.171731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168209+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:06:40.168209+0000","last_peered":"2024-04-21T09:06:40.168209+0000","last_clean":"2024-04-21T09:06:40.168209+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:06:40.168209+0000","last_undegraded":"2024-04-21T09:06:40.168209+0000","last_fullsized":"2024-04-21T09:06:40.168209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.181398+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:06:40.181398+0000","last_peered":"2024-04-21T09:06:40.181398+0000","last_clean":"2024-04-21T09:06:40.181398+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:06:40.181398+0000","last_undegraded":"2024-04-21T09:06:40.181398+0000","last_fullsized":"2024-04-21T09:06:40.181398+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171651+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:06:40.171651+0000","last_peered":"2024-04-21T09:06:40.171651+0000","last_clean":"2024-04-21T09:06:40.171651+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:06:40.171651+0000","last_undegraded":"2024-04-21T09:06:40.171651+0000","last_fullsized":"2024-04-21T09:06:40.171651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171846+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:06:40.171846+0000","last_peered":"2024-04-21T09:06:40.171846+0000","last_clean":"2024-04-21T09:06:40.171846+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:06:40.171846+0000","last_undegraded":"2024-04-21T09:06:40.171846+0000","last_fullsized":"2024-04-21T09:06:40.171846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165864+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:06:40.165864+0000","last_peered":"2024-04-21T09:06:40.165864+0000","last_clean":"2024-04-21T09:06:40.165864+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:06:40.165864+0000","last_undegraded":"2024-04-21T09:06:40.165864+0000","last_fullsized":"2024-04-21T09:06:40.165864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171374+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:06:40.171374+0000","last_peered":"2024-04-21T09:06:40.171374+0000","last_clean":"2024-04-21T09:06:40.171374+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:06:40.171374+0000","last_undegraded":"2024-04-21T09:06:40.171374+0000","last_fullsized":"2024-04-21T09:06:40.171374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164786+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:06:40.164786+0000","last_peered":"2024-04-21T09:06:40.164786+0000","last_clean":"2024-04-21T09:06:40.164786+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:06:40.164786+0000","last_undegraded":"2024-04-21T09:06:40.164786+0000","last_fullsized":"2024-04-21T09:06:40.164786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164715+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:06:40.164715+0000","last_peered":"2024-04-21T09:06:40.164715+0000","last_clean":"2024-04-21T09:06:40.164715+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:06:40.164715+0000","last_undegraded":"2024-04-21T09:06:40.164715+0000","last_fullsized":"2024-04-21T09:06:40.164715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.171652+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:06:40.171652+0000","last_peered":"2024-04-21T09:06:40.171652+0000","last_clean":"2024-04-21T09:06:40.171652+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:06:40.171652+0000","last_undegraded":"2024-04-21T09:06:40.171652+0000","last_fullsized":"2024-04-21T09:06:40.171652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168845+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:06:40.168845+0000","last_peered":"2024-04-21T09:06:40.168845+0000","last_clean":"2024-04-21T09:06:40.168845+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:06:40.168845+0000","last_undegraded":"2024-04-21T09:06:40.168845+0000","last_fullsized":"2024-04-21T09:06:40.168845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168828+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:06:40.168828+0000","last_peered":"2024-04-21T09:06:40.168828+0000","last_clean":"2024-04-21T09:06:40.168828+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:06:40.168828+0000","last_undegraded":"2024-04-21T09:06:40.168828+0000","last_fullsized":"2024-04-21T09:06:40.168828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.168385+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:06:40.168385+0000","last_peered":"2024-04-21T09:06:40.168385+0000","last_clean":"2024-04-21T09:06:40.168385+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:06:40.168385+0000","last_undegraded":"2024-04-21T09:06:40.168385+0000","last_fullsized":"2024-04-21T09:06:40.168385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.172151+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:06:40.172151+0000","last_peered":"2024-04-21T09:06:40.172151+0000","last_clean":"2024-04-21T09:06:40.172151+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:06:40.172151+0000","last_undegraded":"2024-04-21T09:06:40.172151+0000","last_fullsized":"2024-04-21T09:06:40.172151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.163949+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:06:40.163949+0000","last_peered":"2024-04-21T09:06:40.163949+0000","last_clean":"2024-04-21T09:06:40.163949+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:06:40.163949+0000","last_undegraded":"2024-04-21T09:06:40.163949+0000","last_fullsized":"2024-04-21T09:06:40.163949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169626+0000","last_change":"2024-04-21T09:05:46.972797+0000","last_active":"2024-04-21T09:06:40.169626+0000","last_peered":"2024-04-21T09:06:40.169626+0000","last_clean":"2024-04-21T09:06:40.169626+0000","last_became_active":"2024-04-21T09:05:46.972465+0000","last_became_peered":"2024-04-21T09:05:46.972465+0000","last_unstale":"2024-04-21T09:06:40.169626+0000","last_undegraded":"2024-04-21T09:06:40.169626+0000","last_fullsized":"2024-04-21T09:06:40.169626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:29:45.732617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169788+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:06:40.169788+0000","last_peered":"2024-04-21T09:06:40.169788+0000","last_clean":"2024-04-21T09:06:40.169788+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:06:40.169788+0000","last_undegraded":"2024-04-21T09:06:40.169788+0000","last_fullsized":"2024-04-21T09:06:40.169788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.716834+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:06:40.716834+0000","last_peered":"2024-04-21T09:06:40.716834+0000","last_clean":"2024-04-21T09:06:40.716834+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:06:40.716834+0000","last_undegraded":"2024-04-21T09:06:40.716834+0000","last_fullsized":"2024-04-21T09:06:40.716834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.165173+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:06:40.165173+0000","last_peered":"2024-04-21T09:06:40.165173+0000","last_clean":"2024-04-21T09:06:40.165173+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:06:40.165173+0000","last_undegraded":"2024-04-21T09:06:40.165173+0000","last_fullsized":"2024-04-21T09:06:40.165173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.715595+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:06:40.715595+0000","last_peered":"2024-04-21T09:06:40.715595+0000","last_clean":"2024-04-21T09:06:40.715595+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:06:40.715595+0000","last_undegraded":"2024-04-21T09:06:40.715595+0000","last_fullsized":"2024-04-21T09:06:40.715595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169448+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:06:40.169448+0000","last_peered":"2024-04-21T09:06:40.169448+0000","last_clean":"2024-04-21T09:06:40.169448+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:06:40.169448+0000","last_undegraded":"2024-04-21T09:06:40.169448+0000","last_fullsized":"2024-04-21T09:06:40.169448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.164147+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:06:40.164147+0000","last_peered":"2024-04-21T09:06:40.164147+0000","last_clean":"2024-04-21T09:06:40.164147+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:06:40.164147+0000","last_undegraded":"2024-04-21T09:06:40.164147+0000","last_fullsized":"2024-04-21T09:06:40.164147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-21T09:06:40.169492+0000","last_change":"2024-04-21T09:05:46.988031+0000","last_active":"2024-04-21T09:06:40.169492+0000","last_peered":"2024-04-21T09:06:40.169492+0000","last_clean":"2024-04-21T09:06:40.169492+0000","last_became_active":"2024-04-21T09:05:46.987629+0000","last_became_peered":"2024-04-21T09:05:46.987629+0000","last_unstale":"2024-04-21T09:06:40.169492+0000","last_undegraded":"2024-04-21T09:06:40.169492+0000","last_fullsized":"2024-04-21T09:06:40.169492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:33.326914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210748,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65385,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":454,"num_read_kb":297,"num_write":158,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196155,"data_compressed":4893,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":513295,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32312,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712840,"statfs":{"total":95995035648,"available":95961948160,"internally_reserved":0,"allocated":1155072,"data_stored":1162316,"data_compressed":60834,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":517605,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":513407,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":1108947,"data_compressed":59203,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":508685,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":570952,"data_compressed":25163,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":510073,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":570516,"data_compressed":25163,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":508879,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":513957,"data_compressed":23532,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":1104535,"data_compressed":59203,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62357,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61921,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63353,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:07:29.984 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-21T09:07:29.985 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-21T09:07:29.985 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-21T09:07:29.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph health --format=json 2024-04-21T09:07:30.238 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:30.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:30 smithi066 ceph-mon[27953]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:07:30.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:30 smithi186 ceph-mon[31505]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:07:30.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:30 smithi083 ceph-mon[31413]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:07:31.683 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:31 smithi066 ceph-mon[27953]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:31.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:31 smithi186 ceph-mon[31505]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:31.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:31 smithi083 ceph-mon[31413]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:32.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 -- 172.21.15.66:0/2621149843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 msgr2=0x7f3710108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:32.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 --2- 172.21.15.66:0/2621149843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f3710108e10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3700009990 tx=0x7f3700009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 -- 172.21.15.66:0/2621149843 shutdown_connections 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 --2- 172.21.15.66:0/2621149843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f371006b0f0 0x7f371010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 --2- 172.21.15.66:0/2621149843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f3710108e10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 --2- 172.21.15.66:0/2621149843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f371006a750 0x7f371006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 -- 172.21.15.66:0/2621149843 >> 172.21.15.66:0/2621149843 conn(0x7f37100788f0 msgr2=0x7f3710078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 -- 172.21.15.66:0/2621149843 shutdown_connections 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.055+0000 7f3716fd4700 1 -- 172.21.15.66:0/2621149843 wait complete. 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 Processor -- start 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 -- start start 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f371006a750 0x7f3710199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f371006b0f0 0x7f3710199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f371019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:32.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3710112c20 con 0x7f3710108a30 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3710112aa0 con 0x7f371006a750 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3716fd4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3710112da0 con 0x7f371006b0f0 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f371019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f371019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56070/0 (socket says 172.21.15.66:56070) 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 -- 172.21.15.66:0/1281260005 learned_addr learned my addr 172.21.15.66:0/1281260005 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3714d70700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f371006a750 0x7f3710199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f370ffff700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f371006b0f0 0x7f3710199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 -- 172.21.15.66:0/1281260005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f371006b0f0 msgr2=0x7f3710199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f371006b0f0 0x7f3710199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 -- 172.21.15.66:0/1281260005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f371006a750 msgr2=0x7f3710199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f371006a750 0x7f3710199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f3715571700 1 -- 172.21.15.66:0/1281260005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371019e0f0 con 0x7f3710108a30 2024-04-21T09:07:32.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.056+0000 7f370ffff700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f371006b0f0 0x7f3710199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.057+0000 7f3715571700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f371019d9b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3708007ee0 tx=0x7f370800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.057+0000 7f370dffb700 1 -- 172.21.15.66:0/1281260005 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370800dbd0 con 0x7f3710108a30 2024-04-21T09:07:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.057+0000 7f370dffb700 1 -- 172.21.15.66:0/1281260005 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f370800dd70 con 0x7f3710108a30 2024-04-21T09:07:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.057+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371019e3e0 con 0x7f3710108a30 2024-04-21T09:07:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.057+0000 7f370dffb700 1 -- 172.21.15.66:0/1281260005 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708005990 con 0x7f3710108a30 2024-04-21T09:07:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.057+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37101a5c60 con 0x7f3710108a30 2024-04-21T09:07:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.058+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3710069730 con 0x7f3710108a30 2024-04-21T09:07:32.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.059+0000 7f370dffb700 1 -- 172.21.15.66:0/1281260005 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f370801a020 con 0x7f3710108a30 2024-04-21T09:07:32.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.060+0000 7f370dffb700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36f80a3cd0 0x7f36f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:32.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.060+0000 7f370dffb700 1 -- 172.21.15.66:0/1281260005 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f37080d00b0 con 0x7f3710108a30 2024-04-21T09:07:32.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.060+0000 7f3714d70700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36f80a3cd0 0x7f36f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:32.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.060+0000 7f3714d70700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36f80a3cd0 0x7f36f80a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f370000b5c0 tx=0x7f370001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.064+0000 7f370dffb700 1 -- 172.21.15.66:0/1281260005 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3708010070 con 0x7f3710108a30 2024-04-21T09:07:32.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.319+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f371002df80 con 0x7f3710108a30 2024-04-21T09:07:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.319+0000 7f370dffb700 1 -- 172.21.15.66:0/1281260005 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f370809c5d0 con 0x7f3710108a30 2024-04-21T09:07:32.320 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:32.320 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36f80a3cd0 msgr2=0x7f36f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36f80a3cd0 0x7f36f80a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f370000b5c0 tx=0x7f370001a040 comp rx=0 tx=0).stop 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 msgr2=0x7f371019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f371019d9b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3708007ee0 tx=0x7f370800d3b0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 shutdown_connections 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f36f80a3cd0 0x7f36f80a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f371006a750 0x7f3710199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3710108a30 0x7f371019d9b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 --2- 172.21.15.66:0/1281260005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f371006b0f0 0x7f3710199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 >> 172.21.15.66:0/1281260005 conn(0x7f37100788f0 msgr2=0x7f37100ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 shutdown_connections 2024-04-21T09:07:32.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:32.321+0000 7f3716fd4700 1 -- 172.21.15.66:0/1281260005 wait complete. 2024-04-21T09:07:32.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:32 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1281260005' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T09:07:32.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:32 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1281260005' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T09:07:32.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:32 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1281260005' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T09:07:33.089 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-21T09:07:33.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 -- 172.21.15.66:0/577797218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360107c80 msgr2=0x7fe36010a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 --2- 172.21.15.66:0/577797218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360107c80 0x7fe36010a0d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe358009ca0 tx=0x7fe35802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 -- 172.21.15.66:0/577797218 shutdown_connections 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 --2- 172.21.15.66:0/577797218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360107c80 0x7fe36010a0d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 --2- 172.21.15.66:0/577797218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe360105350 0x7fe360107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 --2- 172.21.15.66:0/577797218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360102a20 0x7fe360104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 -- 172.21.15.66:0/577797218 >> 172.21.15.66:0/577797218 conn(0x7fe3600fc410 msgr2=0x7fe3600fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 -- 172.21.15.66:0/577797218 shutdown_connections 2024-04-21T09:07:33.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.201+0000 7fe364eba640 1 -- 172.21.15.66:0/577797218 wait complete. 2024-04-21T09:07:33.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 Processor -- start 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 -- start start 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360102a20 0x7fe360198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360105350 0x7fe360198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe360107c80 0x7fe36019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe36010dae0 con 0x7fe360102a20 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe36010d960 con 0x7fe360105350 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe364eba640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe36010dc60 con 0x7fe360107c80 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360102a20 0x7fe360198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360102a20 0x7fe360198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56080/0 (socket says 172.21.15.66:56080) 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 -- 172.21.15.66:0/3299934883 learned_addr learned my addr 172.21.15.66:0/3299934883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35dd9b640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360105350 0x7fe360198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35ed9d640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe360107c80 0x7fe36019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 -- 172.21.15.66:0/3299934883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe360107c80 msgr2=0x7fe36019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe360107c80 0x7fe36019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 -- 172.21.15.66:0/3299934883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360105350 msgr2=0x7fe360198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360105350 0x7fe360198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.202+0000 7fe35e59c640 1 -- 172.21.15.66:0/3299934883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36019fd70 con 0x7fe360102a20 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.203+0000 7fe35e59c640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360102a20 0x7fe360198150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c890 tx=0x7fe35000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.203+0000 7fe34f7fe640 1 -- 172.21.15.66:0/3299934883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350004220 con 0x7fe360102a20 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.203+0000 7fe34f7fe640 1 -- 172.21.15.66:0/3299934883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3500043c0 con 0x7fe360102a20 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.203+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3601a0060 con 0x7fe360102a20 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.203+0000 7fe34f7fe640 1 -- 172.21.15.66:0/3299934883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350004a00 con 0x7fe360102a20 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.203+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3601a05b0 con 0x7fe360102a20 2024-04-21T09:07:33.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.204+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3600fdf00 con 0x7fe360102a20 2024-04-21T09:07:33.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.205+0000 7fe34f7fe640 1 -- 172.21.15.66:0/3299934883 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe3500079b0 con 0x7fe360102a20 2024-04-21T09:07:33.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.206+0000 7fe34f7fe640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3480a3c40 0x7fe3480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.206+0000 7fe35dd9b640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3480a3c40 0x7fe3480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.206+0000 7fe34f7fe640 1 -- 172.21.15.66:0/3299934883 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fe3500d0580 con 0x7fe360102a20 2024-04-21T09:07:33.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.206+0000 7fe35dd9b640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3480a3c40 0x7fe3480a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe360199660 tx=0x7fe354005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.210+0000 7fe34f7fe640 1 -- 172.21.15.66:0/3299934883 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe350014030 con 0x7fe360102a20 2024-04-21T09:07:33.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.400+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe360062b50 con 0x7fe360102a20 2024-04-21T09:07:33.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.401+0000 7fe34f7fe640 1 -- 172.21.15.66:0/3299934883 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24616 (secure 0 0 0) 0x7fe35009c450 con 0x7fe360102a20 2024-04-21T09:07:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:33.402 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:06:40.160656+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:07:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3480a3c40 msgr2=0x7fe3480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3480a3c40 0x7fe3480a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe360199660 tx=0x7fe354005f90 comp rx=0 tx=0).stop 2024-04-21T09:07:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360102a20 msgr2=0x7fe360198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360102a20 0x7fe360198150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c890 tx=0x7fe35000cd50 comp rx=0 tx=0).stop 2024-04-21T09:07:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 shutdown_connections 2024-04-21T09:07:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3480a3c40 0x7fe3480a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe360107c80 0x7fe36019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360105350 0x7fe360198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 --2- 172.21.15.66:0/3299934883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe360102a20 0x7fe360198150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 >> 172.21.15.66:0/3299934883 conn(0x7fe3600fc410 msgr2=0x7fe360103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:33.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 shutdown_connections 2024-04-21T09:07:33.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.403+0000 7fe364eba640 1 -- 172.21.15.66:0/3299934883 wait complete. 2024-04-21T09:07:33.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:07:33.416 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:07:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 -- 172.21.15.66:0/912727707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 msgr2=0x7fb81c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 --2- 172.21.15.66:0/912727707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c10a3f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb814009ca0 tx=0x7fb81402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 -- 172.21.15.66:0/912727707 shutdown_connections 2024-04-21T09:07:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 --2- 172.21.15.66:0/912727707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c10a3f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 --2- 172.21.15.66:0/912727707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c1031a0 0x7fb81c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 --2- 172.21.15.66:0/912727707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c101fb0 0x7fb81c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 -- 172.21.15.66:0/912727707 >> 172.21.15.66:0/912727707 conn(0x7fb81c0fd520 msgr2=0x7fb81c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 -- 172.21.15.66:0/912727707 shutdown_connections 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 -- 172.21.15.66:0/912727707 wait complete. 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.521+0000 7fb820e4e640 1 Processor -- start 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb820e4e640 1 -- start start 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb820e4e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c101fb0 0x7fb81c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb820e4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c1031a0 0x7fb81c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb820e4e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb820e4e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c10db70 con 0x7fb81c103b60 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb820e4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c10d9f0 con 0x7fb81c1031a0 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb820e4e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c10dcf0 con 0x7fb81c101fb0 2024-04-21T09:07:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56110/0 (socket says 172.21.15.66:56110) 2024-04-21T09:07:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 -- 172.21.15.66:0/3367287996 learned_addr learned my addr 172.21.15.66:0/3367287996 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb819d9b640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c1031a0 0x7fb81c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81a59c640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c101fb0 0x7fb81c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 -- 172.21.15.66:0/3367287996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c101fb0 msgr2=0x7fb81c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c101fb0 0x7fb81c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 -- 172.21.15.66:0/3367287996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c1031a0 msgr2=0x7fb81c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c1031a0 0x7fb81c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 -- 172.21.15.66:0/3367287996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c19fe60 con 0x7fb81c103b60 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb81ad9d640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c19f7f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb81402c700 tx=0x7fb814035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.522+0000 7fb8137fe640 1 -- 172.21.15.66:0/3367287996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814039040 con 0x7fb81c103b60 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.523+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c1a00f0 con 0x7fb81c103b60 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.523+0000 7fb8137fe640 1 -- 172.21.15.66:0/3367287996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb814044070 con 0x7fb81c103b60 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.523+0000 7fb8137fe640 1 -- 172.21.15.66:0/3367287996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814004440 con 0x7fb81c103b60 2024-04-21T09:07:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.523+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c1a05e0 con 0x7fb81c103b60 2024-04-21T09:07:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.524+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81c109370 con 0x7fb81c103b60 2024-04-21T09:07:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.525+0000 7fb8137fe640 1 -- 172.21.15.66:0/3367287996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb8140045e0 con 0x7fb81c103b60 2024-04-21T09:07:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.525+0000 7fb8137fe640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8080a3c40 0x7fb8080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.526+0000 7fb8137fe640 1 -- 172.21.15.66:0/3367287996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fb8140f39f0 con 0x7fb81c103b60 2024-04-21T09:07:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.526+0000 7fb81a59c640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8080a3c40 0x7fb8080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.526+0000 7fb81a59c640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8080a3c40 0x7fb8080a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb81c103000 tx=0x7fb804008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.529+0000 7fb8137fe640 1 -- 172.21.15.66:0/3367287996 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8140bffc0 con 0x7fb81c103b60 2024-04-21T09:07:33.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.714+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb81c02d5c0 con 0x7fb81c103b60 2024-04-21T09:07:33.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.715+0000 7fb8137fe640 1 -- 172.21.15.66:0/3367287996 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb8140bf8c0 con 0x7fb81c103b60 2024-04-21T09:07:33.715 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:07:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.716+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8080a3c40 msgr2=0x7fb8080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.716+0000 7fb820e4e640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8080a3c40 0x7fb8080a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb81c103000 tx=0x7fb804008040 comp rx=0 tx=0).stop 2024-04-21T09:07:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.716+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 msgr2=0x7fb81c19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.716+0000 7fb820e4e640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c19f7f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb81402c700 tx=0x7fb814035d60 comp rx=0 tx=0).stop 2024-04-21T09:07:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 shutdown_connections 2024-04-21T09:07:33.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fb8080a3c40 0x7fb8080a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb81c103b60 0x7fb81c19f7f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c1031a0 0x7fb81c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 --2- 172.21.15.66:0/3367287996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c101fb0 0x7fb81c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 >> 172.21.15.66:0/3367287996 conn(0x7fb81c0fd520 msgr2=0x7fb81c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 shutdown_connections 2024-04-21T09:07:33.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.717+0000 7fb820e4e640 1 -- 172.21.15.66:0/3367287996 wait complete. 2024-04-21T09:07:33.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:33 smithi066 ceph-mon[27953]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:07:33.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:33 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3299934883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:33.728 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:07:33.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 -- 172.21.15.66:0/2296451341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90107c70 msgr2=0x7f4e9010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/2296451341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90107c70 0x7f4e9010a0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009ca0 tx=0x7f4e8802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 -- 172.21.15.66:0/2296451341 shutdown_connections 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/2296451341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90107c70 0x7f4e9010a0c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/2296451341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e90105340 0x7f4e90107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/2296451341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e90102a10 0x7f4e90104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 -- 172.21.15.66:0/2296451341 >> 172.21.15.66:0/2296451341 conn(0x7f4e900fc400 msgr2=0x7f4e900fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 -- 172.21.15.66:0/2296451341 shutdown_connections 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.832+0000 7f4e95b9d640 1 -- 172.21.15.66:0/2296451341 wait complete. 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 Processor -- start 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 -- start start 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90102a10 0x7f4e901981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e90105340 0x7f4e901986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e90107c70 0x7f4e9019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9010dae0 con 0x7f4e90102a10 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9010d960 con 0x7f4e90107c70 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e95b9d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9010dc60 con 0x7f4e90105340 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e8f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90102a10 0x7f4e901981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e8f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90102a10 0x7f4e901981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56134/0 (socket says 172.21.15.66:56134) 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e8f7fe640 1 -- 172.21.15.66:0/332226793 learned_addr learned my addr 172.21.15.66:0/332226793 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.833+0000 7f4e8ffff640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e90107c70 0x7f4e9019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8effd640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e90105340 0x7f4e901986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8f7fe640 1 -- 172.21.15.66:0/332226793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e90105340 msgr2=0x7f4e901986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8f7fe640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e90105340 0x7f4e901986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8f7fe640 1 -- 172.21.15.66:0/332226793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e90107c70 msgr2=0x7f4e9019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8f7fe640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e90107c70 0x7f4e9019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8f7fe640 1 -- 172.21.15.66:0/332226793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9019fdd0 con 0x7f4e90102a10 2024-04-21T09:07:33.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8ffff640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e90107c70 0x7f4e9019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:33.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8effd640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e90105340 0x7f4e901986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:33.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8f7fe640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90102a10 0x7f4e901981b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000d770 tx=0x7f4e8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:33.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8cff9640 1 -- 172.21.15.66:0/332226793 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80017070 con 0x7f4e90102a10 2024-04-21T09:07:33.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8cff9640 1 -- 172.21.15.66:0/332226793 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e800054d0 con 0x7f4e90102a10 2024-04-21T09:07:33.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e901a00c0 con 0x7f4e90102a10 2024-04-21T09:07:33.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e8cff9640 1 -- 172.21.15.66:0/332226793 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80004c30 con 0x7f4e90102a10 2024-04-21T09:07:33.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.834+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e901a05c0 con 0x7f4e90102a10 2024-04-21T09:07:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:33 smithi186 ceph-mon[31505]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:07:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:33 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3299934883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:33.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.835+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e900fdef0 con 0x7f4e90102a10 2024-04-21T09:07:33.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.836+0000 7f4e8cff9640 1 -- 172.21.15.66:0/332226793 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e80004e10 con 0x7f4e90102a10 2024-04-21T09:07:33.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.837+0000 7f4e8cff9640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e780a3c40 0x7f4e780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:33.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.837+0000 7f4e8effd640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e780a3c40 0x7f4e780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:33.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.837+0000 7f4e8cff9640 1 -- 172.21.15.66:0/332226793 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f4e800d0460 con 0x7f4e90102a10 2024-04-21T09:07:33.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.838+0000 7f4e8effd640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e780a3c40 0x7f4e780a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e901996c0 tx=0x7f4e84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:33.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:33.842+0000 7f4e8cff9640 1 -- 172.21.15.66:0/332226793 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e8009c980 con 0x7f4e90102a10 2024-04-21T09:07:33.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:33 smithi083 ceph-mon[31413]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:07:33.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:33 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3299934883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:34.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.027+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e90062b50 con 0x7f4e90102a10 2024-04-21T09:07:34.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.027+0000 7f4e8cff9640 1 -- 172.21.15.66:0/332226793 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f4e8009c280 con 0x7f4e90102a10 2024-04-21T09:07:34.028 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e780a3c40 msgr2=0x7f4e780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e780a3c40 0x7f4e780a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e901996c0 tx=0x7f4e84005f90 comp rx=0 tx=0).stop 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90102a10 msgr2=0x7f4e901981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90102a10 0x7f4e901981b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000d770 tx=0x7f4e8000dc30 comp rx=0 tx=0).stop 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 shutdown_connections 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f4e780a3c40 0x7f4e780a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e90107c70 0x7f4e9019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e90105340 0x7f4e901986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 --2- 172.21.15.66:0/332226793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e90102a10 0x7f4e901981b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 >> 172.21.15.66:0/332226793 conn(0x7f4e900fc400 msgr2=0x7f4e90103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:34.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 shutdown_connections 2024-04-21T09:07:34.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.029+0000 7f4e95b9d640 1 -- 172.21.15.66:0/332226793 wait complete. 2024-04-21T09:07:34.041 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.145+0000 7f8d84055640 1 -- 172.21.15.66:0/655657915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c105360 msgr2=0x7f8d7c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.145+0000 7f8d84055640 1 --2- 172.21.15.66:0/655657915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c105360 0x7f8d7c107750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009900 tx=0x7f8d6c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.145+0000 7f8d84055640 1 -- 172.21.15.66:0/655657915 shutdown_connections 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.145+0000 7f8d84055640 1 --2- 172.21.15.66:0/655657915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d7c107c90 0x7f8d7c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.145+0000 7f8d84055640 1 --2- 172.21.15.66:0/655657915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c105360 0x7f8d7c107750 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.145+0000 7f8d84055640 1 --2- 172.21.15.66:0/655657915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7c102a30 0x7f8d7c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.145+0000 7f8d84055640 1 -- 172.21.15.66:0/655657915 >> 172.21.15.66:0/655657915 conn(0x7f8d7c0fc440 msgr2=0x7f8d7c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 -- 172.21.15.66:0/655657915 shutdown_connections 2024-04-21T09:07:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 -- 172.21.15.66:0/655657915 wait complete. 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 Processor -- start 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 -- start start 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c102a30 0x7f8d7c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d7c105360 0x7f8d7c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7c107c90 0x7f8d7c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c10dab0 con 0x7f8d7c102a30 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c10d930 con 0x7f8d7c107c90 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d84055640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c10dc30 con 0x7f8d7c105360 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d81df1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c102a30 0x7f8d7c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d81df1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c102a30 0x7f8d7c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56144/0 (socket says 172.21.15.66:56144) 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.146+0000 7f8d81df1640 1 -- 172.21.15.66:0/2139835167 learned_addr learned my addr 172.21.15.66:0/2139835167 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d815f0640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d7c105360 0x7f8d7c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d825f2640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7c107c90 0x7f8d7c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d81df1640 1 -- 172.21.15.66:0/2139835167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d7c105360 msgr2=0x7f8d7c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d81df1640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d7c105360 0x7f8d7c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d81df1640 1 -- 172.21.15.66:0/2139835167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7c107c90 msgr2=0x7f8d7c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d81df1640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7c107c90 0x7f8d7c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d81df1640 1 -- 172.21.15.66:0/2139835167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c19fe90 con 0x7f8d7c102a30 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d815f0640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d7c105360 0x7f8d7c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d81df1640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c102a30 0x7f8d7c198270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c890 tx=0x7f8d7800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d72ffd640 1 -- 172.21.15.66:0/2139835167 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78004220 con 0x7f8d7c102a30 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d72ffd640 1 -- 172.21.15.66:0/2139835167 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d780043c0 con 0x7f8d7c102a30 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c1a0180 con 0x7f8d7c102a30 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d72ffd640 1 -- 172.21.15.66:0/2139835167 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78004a00 con 0x7f8d7c102a30 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.147+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c1a06d0 con 0x7f8d7c102a30 2024-04-21T09:07:34.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.148+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7c0fdf30 con 0x7f8d7c102a30 2024-04-21T09:07:34.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.149+0000 7f8d72ffd640 1 -- 172.21.15.66:0/2139835167 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d780079b0 con 0x7f8d7c102a30 2024-04-21T09:07:34.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.150+0000 7f8d72ffd640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8d680a3c40 0x7f8d680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.150+0000 7f8d815f0640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8d680a3c40 0x7f8d680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.150+0000 7f8d72ffd640 1 -- 172.21.15.66:0/2139835167 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f8d780d0740 con 0x7f8d7c102a30 2024-04-21T09:07:34.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.151+0000 7f8d815f0640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8d680a3c40 0x7f8d680a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c036040 tx=0x7f8d6c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:34.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.155+0000 7f8d72ffd640 1 -- 172.21.15.66:0/2139835167 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d78014030 con 0x7f8d7c102a30 2024-04-21T09:07:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.342+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d7c062b80 con 0x7f8d7c102a30 2024-04-21T09:07:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.342+0000 7f8d72ffd640 1 -- 172.21.15.66:0/2139835167 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f8d7809cc00 con 0x7f8d7c102a30 2024-04-21T09:07:34.343 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8d680a3c40 msgr2=0x7f8d680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8d680a3c40 0x7f8d680a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c036040 tx=0x7f8d6c000bc0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c102a30 msgr2=0x7f8d7c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c102a30 0x7f8d7c198270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c890 tx=0x7f8d7800cd50 comp rx=0 tx=0).stop 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 shutdown_connections 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8d680a3c40 0x7f8d680a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7c107c90 0x7f8d7c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d7c105360 0x7f8d7c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 --2- 172.21.15.66:0/2139835167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7c102a30 0x7f8d7c198270 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 >> 172.21.15.66:0/2139835167 conn(0x7f8d7c0fc440 msgr2=0x7f8d7c1034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 shutdown_connections 2024-04-21T09:07:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.344+0000 7f8d84055640 1 -- 172.21.15.66:0/2139835167 wait complete. 2024-04-21T09:07:34.356 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:07:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 -- 172.21.15.66:0/899749907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 msgr2=0x7fbe90101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 --2- 172.21.15.66:0/899749907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90101280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009900 tx=0x7fbe8c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 -- 172.21.15.66:0/899749907 shutdown_connections 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 --2- 172.21.15.66:0/899749907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe90102a00 0x7fbe90109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 --2- 172.21.15.66:0/899749907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90102040 0x7fbe901024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 --2- 172.21.15.66:0/899749907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90101280 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 -- 172.21.15.66:0/899749907 >> 172.21.15.66:0/899749907 conn(0x7fbe900fc420 msgr2=0x7fbe900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 -- 172.21.15.66:0/899749907 shutdown_connections 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 -- 172.21.15.66:0/899749907 wait complete. 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 Processor -- start 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 -- start start 2024-04-21T09:07:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90102040 0x7fbe90198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe90102a00 0x7fbe9019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.461+0000 7fbe97c14640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010c9f0 con 0x7fbe90100e60 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe97c14640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010c870 con 0x7fbe90102040 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe97c14640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010cb70 con 0x7fbe90102a00 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56160/0 (socket says 172.21.15.66:56160) 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 -- 172.21.15.66:0/3403494259 learned_addr learned my addr 172.21.15.66:0/3403494259 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe951af640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90102040 0x7fbe90198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe961b1640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe90102a00 0x7fbe9019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 -- 172.21.15.66:0/3403494259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe90102a00 msgr2=0x7fbe9019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe90102a00 0x7fbe9019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 -- 172.21.15.66:0/3403494259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90102040 msgr2=0x7fbe90198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90102040 0x7fbe90198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 -- 172.21.15.66:0/3403494259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9019fe60 con 0x7fbe90100e60 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe961b1640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe90102a00 0x7fbe9019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe959b0640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90198240 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c000c00 tx=0x7fbe8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe86ffd640 1 -- 172.21.15.66:0/3403494259 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c043020 con 0x7fbe90100e60 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe86ffd640 1 -- 172.21.15.66:0/3403494259 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe8c004aa0 con 0x7fbe90100e60 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe901a00f0 con 0x7fbe90100e60 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe86ffd640 1 -- 172.21.15.66:0/3403494259 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c003d00 con 0x7fbe90100e60 2024-04-21T09:07:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.462+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe901a0640 con 0x7fbe90100e60 2024-04-21T09:07:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.463+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe901a0840 con 0x7fbe90100e60 2024-04-21T09:07:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.464+0000 7fbe86ffd640 1 -- 172.21.15.66:0/3403494259 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe8c046020 con 0x7fbe90100e60 2024-04-21T09:07:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.465+0000 7fbe86ffd640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe7c0a3c40 0x7fbe7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.465+0000 7fbe951af640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe7c0a3c40 0x7fbe7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.465+0000 7fbe86ffd640 1 -- 172.21.15.66:0/3403494259 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fbe8c0f2d80 con 0x7fbe90100e60 2024-04-21T09:07:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.466+0000 7fbe951af640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe7c0a3c40 0x7fbe7c0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbe90199750 tx=0x7fbe8000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.470+0000 7fbe86ffd640 1 -- 172.21.15.66:0/3403494259 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe8c04d020 con 0x7fbe90100e60 2024-04-21T09:07:34.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:34 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3367287996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:34 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/332226793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:34 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2139835167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.657+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbe901a0af0 con 0x7fbe90100e60 2024-04-21T09:07:34.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.658+0000 7fbe86ffd640 1 -- 172.21.15.66:0/3403494259 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fbe8c0bf3b0 con 0x7fbe90100e60 2024-04-21T09:07:34.658 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:07:34.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe7c0a3c40 msgr2=0x7fbe7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe7c0a3c40 0x7fbe7c0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbe90199750 tx=0x7fbe8000b390 comp rx=0 tx=0).stop 2024-04-21T09:07:34.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 msgr2=0x7fbe90198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90198240 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c000c00 tx=0x7fbe8c004270 comp rx=0 tx=0).stop 2024-04-21T09:07:34.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 shutdown_connections 2024-04-21T09:07:34.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe7c0a3c40 0x7fbe7c0a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe90102a00 0x7fbe9019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90102040 0x7fbe90198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.659+0000 7fbe97c14640 1 --2- 172.21.15.66:0/3403494259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe90100e60 0x7fbe90198240 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.660+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 >> 172.21.15.66:0/3403494259 conn(0x7fbe900fc420 msgr2=0x7fbe9006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:34.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.660+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 shutdown_connections 2024-04-21T09:07:34.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.660+0000 7fbe97c14640 1 -- 172.21.15.66:0/3403494259 wait complete. 2024-04-21T09:07:34.673 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 -- 172.21.15.66:0/2042734914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514101fa0 msgr2=0x7f75141023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 --2- 172.21.15.66:0/2042734914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514101fa0 0x7f75141023c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7510009900 tx=0x7f751002bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 -- 172.21.15.66:0/2042734914 shutdown_connections 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 --2- 172.21.15.66:0/2042734914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7514103b50 0x7f751410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 --2- 172.21.15.66:0/2042734914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7514103190 0x7f7514103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 --2- 172.21.15.66:0/2042734914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514101fa0 0x7f75141023c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 -- 172.21.15.66:0/2042734914 >> 172.21.15.66:0/2042734914 conn(0x7f75140fd530 msgr2=0x7f75140ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 -- 172.21.15.66:0/2042734914 shutdown_connections 2024-04-21T09:07:34.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.774+0000 7f751ab40640 1 -- 172.21.15.66:0/2042734914 wait complete. 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 Processor -- start 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 -- start start 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7514101fa0 0x7f7514198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7514103190 0x7f7514198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514103b50 0x7f751419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f751410dae0 con 0x7f7514103b50 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f751410d960 con 0x7f7514103190 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f751ab40640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f751410dc60 con 0x7f7514101fa0 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514103b50 0x7f751419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514103b50 0x7f751419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43640/0 (socket says 172.21.15.66:43640) 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 -- 172.21.15.66:0/1562829109 learned_addr learned my addr 172.21.15.66:0/1562829109 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f750bfff640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7514103190 0x7f7514198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75188dc640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7514101fa0 0x7f7514198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 -- 172.21.15.66:0/1562829109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7514101fa0 msgr2=0x7f7514198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7514101fa0 0x7f7514198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 -- 172.21.15.66:0/1562829109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7514103190 msgr2=0x7f7514198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7514103190 0x7f7514198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.775+0000 7f75190dd640 1 -- 172.21.15.66:0/1562829109 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751419fd30 con 0x7f7514103b50 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.776+0000 7f75190dd640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514103b50 0x7f751419f6c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f750c00eaa0 tx=0x7f750c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.776+0000 7f7509ffb640 1 -- 172.21.15.66:0/1562829109 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c004d20 con 0x7f7514103b50 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.776+0000 7f7509ffb640 1 -- 172.21.15.66:0/1562829109 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c005020 con 0x7f7514103b50 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.776+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751419ffc0 con 0x7f7514103b50 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.776+0000 7f7509ffb640 1 -- 172.21.15.66:0/1562829109 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c0116b0 con 0x7f7514103b50 2024-04-21T09:07:34.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.776+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75141a0490 con 0x7f7514103b50 2024-04-21T09:07:34.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.777+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75141092c0 con 0x7f7514103b50 2024-04-21T09:07:34.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.778+0000 7f7509ffb640 1 -- 172.21.15.66:0/1562829109 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f750c0051d0 con 0x7f7514103b50 2024-04-21T09:07:34.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.779+0000 7f7509ffb640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74fc0a3c40 0x7f74fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:34.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.779+0000 7f75188dc640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74fc0a3c40 0x7f74fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:34.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.779+0000 7f7509ffb640 1 -- 172.21.15.66:0/1562829109 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f750c0d0a90 con 0x7f7514103b50 2024-04-21T09:07:34.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.780+0000 7f75188dc640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74fc0a3c40 0x7f74fc0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7510005ab0 tx=0x7f7510005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:34.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:34.784+0000 7f7509ffb640 1 -- 172.21.15.66:0/1562829109 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f750c016030 con 0x7f7514103b50 2024-04-21T09:07:34.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:34 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3367287996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:34 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/332226793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:34 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2139835167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:34 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3367287996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:34 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/332226793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:07:34.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:34 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2139835167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:07:35.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.010+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f751402d5c0 con 0x7f7514103b50 2024-04-21T09:07:35.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.011+0000 7f7509ffb640 1 -- 172.21.15.66:0/1562829109 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11867 (secure 0 0 0) 0x7f750c09cfb0 con 0x7f7514103b50 2024-04-21T09:07:35.013 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:35.013 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:06:13.097844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24476,24490]},"id":1}]} 2024-04-21T09:07:35.014 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-04-21T09:07:35.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74fc0a3c40 msgr2=0x7f74fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74fc0a3c40 0x7f74fc0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7510005ab0 tx=0x7f7510005fd0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514103b50 msgr2=0x7f751419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514103b50 0x7f751419f6c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f750c00eaa0 tx=0x7f750c00eea0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 shutdown_connections 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f74fc0a3c40 0x7f74fc0a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7514103b50 0x7f751419f6c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7514103190 0x7f7514198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 --2- 172.21.15.66:0/1562829109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7514101fa0 0x7f7514198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 >> 172.21.15.66:0/1562829109 conn(0x7f75140fd530 msgr2=0x7f7514106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.013+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 shutdown_connections 2024-04-21T09:07:35.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.014+0000 7f751ab40640 1 -- 172.21.15.66:0/1562829109 wait complete. 2024-04-21T09:07:35.026 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:07:35.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 -- 172.21.15.66:0/2687204792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 msgr2=0x7f59601090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 --2- 172.21.15.66:0/2687204792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601090d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5958009ca0 tx=0x7f595802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 -- 172.21.15.66:0/2687204792 shutdown_connections 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 --2- 172.21.15.66:0/2687204792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601090d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 --2- 172.21.15.66:0/2687204792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5960101e80 0x7f5960102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 --2- 172.21.15.66:0/2687204792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5960100c90 0x7f59601010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 -- 172.21.15.66:0/2687204792 >> 172.21.15.66:0/2687204792 conn(0x7f59600fc1e0 msgr2=0x7f59600fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 -- 172.21.15.66:0/2687204792 shutdown_connections 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 -- 172.21.15.66:0/2687204792 wait complete. 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 Processor -- start 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.128+0000 7f596679f640 1 -- start start 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f596679f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5960100c90 0x7f5960104c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f596679f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5960101e80 0x7f5960103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f596679f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f596679f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f596010c7a0 con 0x7f5960102840 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f596679f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f596010c620 con 0x7f5960101e80 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f596679f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f596010c920 con 0x7f5960100c90 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5964f9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5960101e80 0x7f5960103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43664/0 (socket says 172.21.15.66:43664) 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 -- 172.21.15.66:0/3294547431 learned_addr learned my addr 172.21.15.66:0/3294547431 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f596579d640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5960100c90 0x7f5960104c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 -- 172.21.15.66:0/3294547431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5960100c90 msgr2=0x7f5960104c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5960100c90 0x7f5960104c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 -- 172.21.15.66:0/3294547431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5960101e80 msgr2=0x7f5960103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5960101e80 0x7f5960103260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 -- 172.21.15.66:0/3294547431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5960104030 con 0x7f5960102840 2024-04-21T09:07:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.129+0000 7f5965f9e640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601037a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f595802c700 tx=0x7f5958034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.130+0000 7f59567fc640 1 -- 172.21.15.66:0/3294547431 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958039040 con 0x7f5960102840 2024-04-21T09:07:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.130+0000 7f59567fc640 1 -- 172.21.15.66:0/3294547431 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5958044070 con 0x7f5960102840 2024-04-21T09:07:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.130+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59601a1fd0 con 0x7f5960102840 2024-04-21T09:07:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.130+0000 7f59567fc640 1 -- 172.21.15.66:0/3294547431 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59580042a0 con 0x7f5960102840 2024-04-21T09:07:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.130+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59601a2430 con 0x7f5960102840 2024-04-21T09:07:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.131+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59601a26f0 con 0x7f5960102840 2024-04-21T09:07:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.132+0000 7f59567fc640 1 -- 172.21.15.66:0/3294547431 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5958003970 con 0x7f5960102840 2024-04-21T09:07:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.133+0000 7f59567fc640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f594c0a3c40 0x7f594c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.133+0000 7f59567fc640 1 -- 172.21.15.66:0/3294547431 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f59580f35a0 con 0x7f5960102840 2024-04-21T09:07:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.133+0000 7f596579d640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f594c0a3c40 0x7f594c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.133+0000 7f596579d640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f594c0a3c40 0x7f594c0a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f595c005fd0 tx=0x7f595c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.137+0000 7f59567fc640 1 -- 172.21.15.66:0/3294547431 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59580bfbd0 con 0x7f5960102840 2024-04-21T09:07:35.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.326+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59601a2a40 con 0x7f5960102840 2024-04-21T09:07:35.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.327+0000 7f59567fc640 1 -- 172.21.15.66:0/3294547431 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24616 (secure 0 0 0) 0x7f59580bf4d0 con 0x7f5960102840 2024-04-21T09:07:35.328 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:35.328 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:06:40.160656+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:07:35.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.329+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f594c0a3c40 msgr2=0x7f594c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.329+0000 7f596679f640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f594c0a3c40 0x7f594c0a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f595c005fd0 tx=0x7f595c005850 comp rx=0 tx=0).stop 2024-04-21T09:07:35.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.329+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 msgr2=0x7f59601037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.329+0000 7f596679f640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601037a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f595802c700 tx=0x7f5958034760 comp rx=0 tx=0).stop 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 shutdown_connections 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f594c0a3c40 0x7f594c0a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5960102840 0x7f59601037a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5960101e80 0x7f5960103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 --2- 172.21.15.66:0/3294547431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5960100c90 0x7f5960104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 >> 172.21.15.66:0/3294547431 conn(0x7f59600fc1e0 msgr2=0x7f59600fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 shutdown_connections 2024-04-21T09:07:35.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.330+0000 7f596679f640 1 -- 172.21.15.66:0/3294547431 wait complete. 2024-04-21T09:07:35.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:07:35.342 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.446+0000 7f9d76a77640 1 -- 172.21.15.66:0/517375152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 msgr2=0x7f9d701023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.446+0000 7f9d76a77640 1 --2- 172.21.15.66:0/517375152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d701023b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009900 tx=0x7f9d6402bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- 172.21.15.66:0/517375152 shutdown_connections 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 --2- 172.21.15.66:0/517375152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70103b40 0x7f9d7010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 --2- 172.21.15.66:0/517375152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d70103180 0x7f9d70103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 --2- 172.21.15.66:0/517375152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d701023b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- 172.21.15.66:0/517375152 >> 172.21.15.66:0/517375152 conn(0x7f9d700fd500 msgr2=0x7f9d700ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:35.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- 172.21.15.66:0/517375152 shutdown_connections 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- 172.21.15.66:0/517375152 wait complete. 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 Processor -- start 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- start start 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d70198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70103180 0x7f9d70198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d70103b40 0x7f9d7019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010dad0 con 0x7f9d70101f90 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010d950 con 0x7f9d70103b40 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d76a77640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010dc50 con 0x7f9d70103180 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d74813640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d70198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d74813640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d70198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43680/0 (socket says 172.21.15.66:43680) 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.447+0000 7f9d74813640 1 -- 172.21.15.66:0/234223399 learned_addr learned my addr 172.21.15.66:0/234223399 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:35.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d75014640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d70103b40 0x7f9d7019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d6ffff640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70103180 0x7f9d70198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d74813640 1 -- 172.21.15.66:0/234223399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70103180 msgr2=0x7f9d70198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d74813640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70103180 0x7f9d70198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d74813640 1 -- 172.21.15.66:0/234223399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d70103b40 msgr2=0x7f9d7019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d74813640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d70103b40 0x7f9d7019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d74813640 1 -- 172.21.15.66:0/234223399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7019fe30 con 0x7f9d70101f90 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d75014640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d70103b40 0x7f9d7019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d74813640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d70198210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9d64000c00 tx=0x7f9d64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d6dffb640 1 -- 172.21.15.66:0/234223399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64043020 con 0x7f9d70101f90 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d6dffb640 1 -- 172.21.15.66:0/234223399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d64004aa0 con 0x7f9d70101f90 2024-04-21T09:07:35.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701a00c0 con 0x7f9d70101f90 2024-04-21T09:07:35.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d6dffb640 1 -- 172.21.15.66:0/234223399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64003d00 con 0x7f9d70101f90 2024-04-21T09:07:35.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.448+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701a0610 con 0x7f9d70101f90 2024-04-21T09:07:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.449+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d701093a0 con 0x7f9d70101f90 2024-04-21T09:07:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.450+0000 7f9d6dffb640 1 -- 172.21.15.66:0/234223399 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9d64046020 con 0x7f9d70101f90 2024-04-21T09:07:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.451+0000 7f9d6dffb640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.451+0000 7f9d6ffff640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.451+0000 7f9d6dffb640 1 -- 172.21.15.66:0/234223399 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f9d640f2e20 con 0x7f9d70101f90 2024-04-21T09:07:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.452+0000 7f9d6ffff640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d600a3c40 0x7f9d600a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005d00 tx=0x7f9d5c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:35.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.456+0000 7f9d6dffb640 1 -- 172.21.15.66:0/234223399 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d6404d020 con 0x7f9d70101f90 2024-04-21T09:07:35.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:35 smithi066 ceph-mon[27953]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:07:35.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:35 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3403494259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:07:35.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:35 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1562829109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:35.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:35 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3294547431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:35.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.643+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d70062b50 con 0x7f9d70101f90 2024-04-21T09:07:35.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.643+0000 7f9d6dffb640 1 -- 172.21.15.66:0/234223399 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f9d640bf450 con 0x7f9d70101f90 2024-04-21T09:07:35.643 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d600a3c40 msgr2=0x7f9d600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d600a3c40 0x7f9d600a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005d00 tx=0x7f9d5c005c50 comp rx=0 tx=0).stop 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 msgr2=0x7f9d70198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d70198210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9d64000c00 tx=0x7f9d64004270 comp rx=0 tx=0).stop 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 shutdown_connections 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d70103b40 0x7f9d7019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d70103180 0x7f9d70198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 --2- 172.21.15.66:0/234223399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70101f90 0x7f9d70198210 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 >> 172.21.15.66:0/234223399 conn(0x7f9d700fd500 msgr2=0x7f9d70106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 shutdown_connections 2024-04-21T09:07:35.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.645+0000 7f9d76a77640 1 -- 172.21.15.66:0/234223399 wait complete. 2024-04-21T09:07:35.656 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 -- 172.21.15.66:0/4220797000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 msgr2=0x7f0b981092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 --2- 172.21.15.66:0/4220797000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b981092c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009ca0 tx=0x7f0b9002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 -- 172.21.15.66:0/4220797000 shutdown_connections 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 --2- 172.21.15.66:0/4220797000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b981092c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 --2- 172.21.15.66:0/4220797000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b98101fe0 0x7f0b98102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 --2- 172.21.15.66:0/4220797000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b98100e90 0x7f0b981012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 -- 172.21.15.66:0/4220797000 >> 172.21.15.66:0/4220797000 conn(0x7f0b980fc400 msgr2=0x7f0b980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 -- 172.21.15.66:0/4220797000 shutdown_connections 2024-04-21T09:07:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.761+0000 7f0b9f455640 1 -- 172.21.15.66:0/4220797000 wait complete. 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 Processor -- start 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 -- start start 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b98100e90 0x7f0b98198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b98101fe0 0x7f0b98198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b9819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9810c9d0 con 0x7f0b981029a0 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9810c850 con 0x7f0b98100e90 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9f455640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9810cb50 con 0x7f0b98101fe0 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b9819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b9819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43702/0 (socket says 172.21.15.66:43702) 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 -- 172.21.15.66:0/3518009762 learned_addr learned my addr 172.21.15.66:0/3518009762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d1f1640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b98100e90 0x7f0b98198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9c9f0640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b98101fe0 0x7f0b98198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 -- 172.21.15.66:0/3518009762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b98101fe0 msgr2=0x7f0b98198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b98101fe0 0x7f0b98198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 -- 172.21.15.66:0/3518009762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b98100e90 msgr2=0x7f0b98198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b98100e90 0x7f0b98198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 -- 172.21.15.66:0/3518009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9819fd20 con 0x7f0b981029a0 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d1f1640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b98100e90 0x7f0b98198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.762+0000 7f0b9d9f2640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b9819f6b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b9002c700 tx=0x7f0b90034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.763+0000 7f0b8e7fc640 1 -- 172.21.15.66:0/3518009762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90039040 con 0x7f0b981029a0 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.763+0000 7f0b8e7fc640 1 -- 172.21.15.66:0/3518009762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b90044070 con 0x7f0b981029a0 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.763+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9819ffb0 con 0x7f0b981029a0 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.763+0000 7f0b8e7fc640 1 -- 172.21.15.66:0/3518009762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900042a0 con 0x7f0b981029a0 2024-04-21T09:07:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.763+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b981a0410 con 0x7f0b981029a0 2024-04-21T09:07:35.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.764+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b981a0670 con 0x7f0b981029a0 2024-04-21T09:07:35.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.765+0000 7f0b8e7fc640 1 -- 172.21.15.66:0/3518009762 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b90003970 con 0x7f0b981029a0 2024-04-21T09:07:35.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.766+0000 7f0b8e7fc640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b840a3c40 0x7f0b840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:35.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.766+0000 7f0b9d1f1640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b840a3c40 0x7f0b840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:35.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.766+0000 7f0b8e7fc640 1 -- 172.21.15.66:0/3518009762 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f0b900f2c60 con 0x7f0b981029a0 2024-04-21T09:07:35.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.766+0000 7f0b9d1f1640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b840a3c40 0x7f0b840a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800abe0 tx=0x7f0b8800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:35.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.770+0000 7f0b8e7fc640 1 -- 172.21.15.66:0/3518009762 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b900bf290 con 0x7f0b981029a0 2024-04-21T09:07:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:35 smithi186 ceph-mon[31505]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:07:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:35 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3403494259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:07:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:35 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1562829109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:35 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3294547431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:35 smithi083 ceph-mon[31413]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:07:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:35 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3403494259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:07:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:35 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1562829109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:35.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:35 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3294547431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:35.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.955+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0b981a0a70 con 0x7f0b981029a0 2024-04-21T09:07:35.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.956+0000 7f0b8e7fc640 1 -- 172.21.15.66:0/3518009762 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f0b900beb90 con 0x7f0b981029a0 2024-04-21T09:07:35.956 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:07:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b840a3c40 msgr2=0x7f0b840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b840a3c40 0x7f0b840a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800abe0 tx=0x7f0b8800a5c0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 msgr2=0x7f0b9819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b9819f6b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b9002c700 tx=0x7f0b90034760 comp rx=0 tx=0).stop 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 shutdown_connections 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b840a3c40 0x7f0b840a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b981029a0 0x7f0b9819f6b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b98101fe0 0x7f0b98198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 --2- 172.21.15.66:0/3518009762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b98100e90 0x7f0b98198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 >> 172.21.15.66:0/3518009762 conn(0x7f0b980fc400 msgr2=0x7f0b980fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 shutdown_connections 2024-04-21T09:07:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:35.958+0000 7f0b9f455640 1 -- 172.21.15.66:0/3518009762 wait complete. 2024-04-21T09:07:35.970 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:07:36.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 -- 172.21.15.66:0/1451446446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103190 msgr2=0x7f9890103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 --2- 172.21.15.66:0/1451446446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103190 0x7f9890103610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f987802bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 -- 172.21.15.66:0/1451446446 shutdown_connections 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 --2- 172.21.15.66:0/1451446446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9890103b50 0x7f989010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 --2- 172.21.15.66:0/1451446446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103190 0x7f9890103610 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 --2- 172.21.15.66:0/1451446446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9890101fa0 0x7f98901023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 -- 172.21.15.66:0/1451446446 >> 172.21.15.66:0/1451446446 conn(0x7f98900fd530 msgr2=0x7f98900ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 -- 172.21.15.66:0/1451446446 shutdown_connections 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 -- 172.21.15.66:0/1451446446 wait complete. 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 Processor -- start 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.074+0000 7f989530e640 1 -- start start 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9890101fa0 0x7f9890075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9890103190 0x7f9890074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103b50 0x7f9890074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f989010dae0 con 0x7f9890103b50 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f989010d960 con 0x7f9890103190 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f989010dc60 con 0x7f9890101fa0 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103b50 0x7f9890074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103b50 0x7f9890074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43726/0 (socket says 172.21.15.66:43726) 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 -- 172.21.15.66:0/2796626334 learned_addr learned my addr 172.21.15.66:0/2796626334 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f9887fff640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9890103190 0x7f9890074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988effd640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9890101fa0 0x7f9890075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 -- 172.21.15.66:0/2796626334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9890101fa0 msgr2=0x7f9890075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9890101fa0 0x7f9890075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 -- 172.21.15.66:0/2796626334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9890103190 msgr2=0x7f9890074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9890103190 0x7f9890074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 -- 172.21.15.66:0/2796626334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9890075110 con 0x7f9890103b50 2024-04-21T09:07:36.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988f7fe640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103b50 0x7f9890074880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f988800d750 tx=0x7f988800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:36.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988cff9640 1 -- 172.21.15.66:0/2796626334 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888017070 con 0x7f9890103b50 2024-04-21T09:07:36.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988cff9640 1 -- 172.21.15.66:0/2796626334 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98880054d0 con 0x7f9890103b50 2024-04-21T09:07:36.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901a22e0 con 0x7f9890103b50 2024-04-21T09:07:36.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f988cff9640 1 -- 172.21.15.66:0/2796626334 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888004c60 con 0x7f9890103b50 2024-04-21T09:07:36.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.075+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a27b0 con 0x7f9890103b50 2024-04-21T09:07:36.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.076+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9890109390 con 0x7f9890103b50 2024-04-21T09:07:36.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.078+0000 7f988cff9640 1 -- 172.21.15.66:0/2796626334 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9888009bd0 con 0x7f9890103b50 2024-04-21T09:07:36.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.078+0000 7f988cff9640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f987c0a3c40 0x7f987c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.078+0000 7f988effd640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f987c0a3c40 0x7f987c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.078+0000 7f988cff9640 1 -- 172.21.15.66:0/2796626334 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f98880d01d0 con 0x7f9890103b50 2024-04-21T09:07:36.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.079+0000 7f988effd640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f987c0a3c40 0x7f987c0a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9890102ff0 tx=0x7f9880009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:36.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.083+0000 7f988cff9640 1 -- 172.21.15.66:0/2796626334 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f988809c0a0 con 0x7f9890103b50 2024-04-21T09:07:36.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.270+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9890062b50 con 0x7f9890103b50 2024-04-21T09:07:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.270+0000 7f988cff9640 1 -- 172.21.15.66:0/2796626334 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f98880bc030 con 0x7f9890103b50 2024-04-21T09:07:36.271 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f987c0a3c40 msgr2=0x7f987c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f987c0a3c40 0x7f987c0a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9890102ff0 tx=0x7f9880009500 comp rx=0 tx=0).stop 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103b50 msgr2=0x7f9890074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103b50 0x7f9890074880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f988800d750 tx=0x7f988800dc10 comp rx=0 tx=0).stop 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 shutdown_connections 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f987c0a3c40 0x7f987c0a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9890103b50 0x7f9890074880 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9890103190 0x7f9890074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 --2- 172.21.15.66:0/2796626334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9890101fa0 0x7f9890075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 >> 172.21.15.66:0/2796626334 conn(0x7f98900fd530 msgr2=0x7f9890106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:36.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 shutdown_connections 2024-04-21T09:07:36.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.272+0000 7f989530e640 1 -- 172.21.15.66:0/2796626334 wait complete. 2024-04-21T09:07:36.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:07:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 -- 172.21.15.66:0/554995509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c102990 msgr2=0x7f067c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 --2- 172.21.15.66:0/554995509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c102990 0x7f067c1092b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0674009ca0 tx=0x7f067402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 -- 172.21.15.66:0/554995509 shutdown_connections 2024-04-21T09:07:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 --2- 172.21.15.66:0/554995509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c102990 0x7f067c1092b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 --2- 172.21.15.66:0/554995509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f067c101fd0 0x7f067c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 --2- 172.21.15.66:0/554995509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067c100e80 0x7f067c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 -- 172.21.15.66:0/554995509 >> 172.21.15.66:0/554995509 conn(0x7f067c0fc3f0 msgr2=0x7f067c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 -- 172.21.15.66:0/554995509 shutdown_connections 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.388+0000 7f0682de2640 1 -- 172.21.15.66:0/554995509 wait complete. 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 Processor -- start 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 -- start start 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f067c100e80 0x7f067c104e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c101fd0 0x7f067c103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067c102990 0x7f067c103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f067c10c9c0 con 0x7f067c101fd0 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f067c10c840 con 0x7f067c102990 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f0682de2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f067c10cb40 con 0x7f067c100e80 2024-04-21T09:07:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f067bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c101fd0 0x7f067c103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f067bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c101fd0 0x7f067c103440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43748/0 (socket says 172.21.15.66:43748) 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.389+0000 7f067bfff640 1 -- 172.21.15.66:0/3471868710 learned_addr learned my addr 172.21.15.66:0/3471868710 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f0680b7e640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f067c100e80 0x7f067c104e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f068137f640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067c102990 0x7f067c103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f067bfff640 1 -- 172.21.15.66:0/3471868710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f067c100e80 msgr2=0x7f067c104e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f067bfff640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f067c100e80 0x7f067c104e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f067bfff640 1 -- 172.21.15.66:0/3471868710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067c102990 msgr2=0x7f067c103980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f067bfff640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067c102990 0x7f067c103980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f067bfff640 1 -- 172.21.15.66:0/3471868710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c104210 con 0x7f067c101fd0 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f067bfff640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c101fd0 0x7f067c103440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f066800d770 tx=0x7f066800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f0679ffb640 1 -- 172.21.15.66:0/3471868710 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668017070 con 0x7f067c101fd0 2024-04-21T09:07:36.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f0679ffb640 1 -- 172.21.15.66:0/3471868710 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0668009c30 con 0x7f067c101fd0 2024-04-21T09:07:36.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c1a22b0 con 0x7f067c101fd0 2024-04-21T09:07:36.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f0679ffb640 1 -- 172.21.15.66:0/3471868710 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668003cc0 con 0x7f067c101fd0 2024-04-21T09:07:36.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.390+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c1a2800 con 0x7f067c101fd0 2024-04-21T09:07:36.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.391+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c1a2ac0 con 0x7f067c101fd0 2024-04-21T09:07:36.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.392+0000 7f0679ffb640 1 -- 172.21.15.66:0/3471868710 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0668003e60 con 0x7f067c101fd0 2024-04-21T09:07:36.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.393+0000 7f0679ffb640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f066c0a3c40 0x7f066c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.393+0000 7f0680b7e640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f066c0a3c40 0x7f066c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.393+0000 7f0679ffb640 1 -- 172.21.15.66:0/3471868710 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f06680d01d0 con 0x7f067c101fd0 2024-04-21T09:07:36.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.393+0000 7f0680b7e640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f066c0a3c40 0x7f066c0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0670009bd0 tx=0x7f0670005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:36.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.397+0000 7f0679ffb640 1 -- 172.21.15.66:0/3471868710 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0668009de0 con 0x7f067c101fd0 2024-04-21T09:07:36.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.585+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f067c1a2c90 con 0x7f067c101fd0 2024-04-21T09:07:36.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.585+0000 7f0679ffb640 1 -- 172.21.15.66:0/3471868710 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f06680bc070 con 0x7f067c101fd0 2024-04-21T09:07:36.586 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:07:36.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f066c0a3c40 msgr2=0x7f066c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f066c0a3c40 0x7f066c0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0670009bd0 tx=0x7f0670005f90 comp rx=0 tx=0).stop 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c101fd0 msgr2=0x7f067c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c101fd0 0x7f067c103440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f066800d770 tx=0x7f066800dc30 comp rx=0 tx=0).stop 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 shutdown_connections 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f066c0a3c40 0x7f066c0a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067c102990 0x7f067c103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f067c101fd0 0x7f067c103440 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 --2- 172.21.15.66:0/3471868710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f067c100e80 0x7f067c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 >> 172.21.15.66:0/3471868710 conn(0x7f067c0fc3f0 msgr2=0x7f067c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 shutdown_connections 2024-04-21T09:07:36.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.588+0000 7f0682de2640 1 -- 172.21.15.66:0/3471868710 wait complete. 2024-04-21T09:07:36.598 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:36 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/234223399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.598 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:36 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3518009762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.598 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:36 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2796626334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.599 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:07:36.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 -- 172.21.15.66:0/1482146843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0100e70 msgr2=0x7f33b0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 --2- 172.21.15.66:0/1482146843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0100e70 0x7f33b0101290 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009ca0 tx=0x7f33a002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 -- 172.21.15.66:0/1482146843 shutdown_connections 2024-04-21T09:07:36.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 --2- 172.21.15.66:0/1482146843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33b0102980 0x7f33b01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 --2- 172.21.15.66:0/1482146843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b0101fc0 0x7f33b0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 --2- 172.21.15.66:0/1482146843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0100e70 0x7f33b0101290 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 -- 172.21.15.66:0/1482146843 >> 172.21.15.66:0/1482146843 conn(0x7f33b00fc3e0 msgr2=0x7f33b00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.700+0000 7f33b5c31640 1 -- 172.21.15.66:0/1482146843 shutdown_connections 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 -- 172.21.15.66:0/1482146843 wait complete. 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 Processor -- start 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 -- start start 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b0100e70 0x7f33b0069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33b0101fc0 0x7f33b019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0102980 0x7f33b019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33b010c9c0 con 0x7f33b0102980 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33b010c840 con 0x7f33b0100e70 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33b5c31640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f33b010cb40 con 0x7f33b0101fc0 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.701+0000 7f33affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0102980 0x7f33b019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0102980 0x7f33b019f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43760/0 (socket says 172.21.15.66:43760) 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 -- 172.21.15.66:0/2292094599 learned_addr learned my addr 172.21.15.66:0/2292094599 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33af7fe640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b0100e70 0x7f33b0069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 -- 172.21.15.66:0/2292094599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33b0101fc0 msgr2=0x7f33b019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33b0101fc0 0x7f33b019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 -- 172.21.15.66:0/2292094599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b0100e70 msgr2=0x7f33b0069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b0100e70 0x7f33b0069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 -- 172.21.15.66:0/2292094599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b019fd60 con 0x7f33b0102980 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33af7fe640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b0100e70 0x7f33b0069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33affff640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0102980 0x7f33b019f4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33a800d750 tx=0x7f33a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33acff9640 1 -- 172.21.15.66:0/2292094599 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8017070 con 0x7f33b0102980 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33acff9640 1 -- 172.21.15.66:0/2292094599 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a80054d0 con 0x7f33b0102980 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b01a0050 con 0x7f33b0102980 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33acff9640 1 -- 172.21.15.66:0/2292094599 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8004c60 con 0x7f33b0102980 2024-04-21T09:07:36.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.702+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b01a0570 con 0x7f33b0102980 2024-04-21T09:07:36.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.703+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b0108180 con 0x7f33b0102980 2024-04-21T09:07:36.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.704+0000 7f33acff9640 1 -- 172.21.15.66:0/2292094599 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f33a8009bd0 con 0x7f33b0102980 2024-04-21T09:07:36.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.705+0000 7f33acff9640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33980a3c40 0x7f33980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:36.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.705+0000 7f33acff9640 1 -- 172.21.15.66:0/2292094599 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f33a80d0340 con 0x7f33b0102980 2024-04-21T09:07:36.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.705+0000 7f33af7fe640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33980a3c40 0x7f33980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:36.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.705+0000 7f33af7fe640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33980a3c40 0x7f33980a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33a0000f80 tx=0x7f33a00324c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:36.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.710+0000 7f33acff9640 1 -- 172.21.15.66:0/2292094599 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f33a809c860 con 0x7f33b0102980 2024-04-21T09:07:36.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:36 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/234223399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:36 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3518009762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:36 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2796626334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:36 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/234223399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:36 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3518009762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:36 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2796626334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:07:36.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.933+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33b002d810 con 0x7f33b0102980 2024-04-21T09:07:36.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.934+0000 7f33acff9640 1 -- 172.21.15.66:0/2292094599 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11867 (secure 0 0 0) 0x7f33a809c160 con 0x7f33b0102980 2024-04-21T09:07:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:36.935 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:06:13.097844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24476,24490]},"id":1}]} 2024-04-21T09:07:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33980a3c40 msgr2=0x7f33980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33980a3c40 0x7f33980a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33a0000f80 tx=0x7f33a00324c0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0102980 msgr2=0x7f33b019f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0102980 0x7f33b019f4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33a800d750 tx=0x7f33a800dc10 comp rx=0 tx=0).stop 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 shutdown_connections 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f33980a3c40 0x7f33980a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33b0102980 0x7f33b019f4a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33b0101fc0 0x7f33b019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 --2- 172.21.15.66:0/2292094599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b0100e70 0x7f33b0069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 >> 172.21.15.66:0/2292094599 conn(0x7f33b00fc3e0 msgr2=0x7f33b0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 shutdown_connections 2024-04-21T09:07:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:36.936+0000 7f33b5c31640 1 -- 172.21.15.66:0/2292094599 wait complete. 2024-04-21T09:07:36.948 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:07:37.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.049+0000 7ffaa242c640 1 -- 172.21.15.66:0/3045206831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 msgr2=0x7ffa9c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.049+0000 7ffaa242c640 1 --2- 172.21.15.66:0/3045206831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c101290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009900 tx=0x7ffa8c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:37.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.049+0000 7ffaa242c640 1 -- 172.21.15.66:0/3045206831 shutdown_connections 2024-04-21T09:07:37.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.049+0000 7ffaa242c640 1 --2- 172.21.15.66:0/3045206831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa9c102980 0x7ffa9c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.049+0000 7ffaa242c640 1 --2- 172.21.15.66:0/3045206831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa9c101fc0 0x7ffa9c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.049+0000 7ffaa242c640 1 --2- 172.21.15.66:0/3045206831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c101290 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.049+0000 7ffaa242c640 1 -- 172.21.15.66:0/3045206831 >> 172.21.15.66:0/3045206831 conn(0x7ffa9c0fc3e0 msgr2=0x7ffa9c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 -- 172.21.15.66:0/3045206831 shutdown_connections 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 -- 172.21.15.66:0/3045206831 wait complete. 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 Processor -- start 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 -- start start 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa9c101fc0 0x7ffa9c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa9c102980 0x7ffa9c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c10c9c0 con 0x7ffa9c100e70 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c10c840 con 0x7ffa9c101fc0 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffaa242c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c10cb40 con 0x7ffa9c102980 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.050+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43780/0 (socket says 172.21.15.66:43780) 2024-04-21T09:07:37.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 -- 172.21.15.66:0/1429901583 learned_addr learned my addr 172.21.15.66:0/1429901583 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 -- 172.21.15.66:0/1429901583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa9c102980 msgr2=0x7ffa9c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa9c102980 0x7ffa9c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 -- 172.21.15.66:0/1429901583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa9c101fc0 msgr2=0x7ffa9c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9b7fe640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa9c101fc0 0x7ffa9c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa9c101fc0 0x7ffa9c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 -- 172.21.15.66:0/1429901583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c19fdc0 con 0x7ffa9c100e70 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa9bfff640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c1981a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa997fa640 1 -- 172.21.15.66:0/1429901583 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c043020 con 0x7ffa9c100e70 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa997fa640 1 -- 172.21.15.66:0/1429901583 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa8c004440 con 0x7ffa9c100e70 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c1a0050 con 0x7ffa9c100e70 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffa997fa640 1 -- 172.21.15.66:0/1429901583 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c003bf0 con 0x7ffa9c100e70 2024-04-21T09:07:37.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.051+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c1a05a0 con 0x7ffa9c100e70 2024-04-21T09:07:37.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.052+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9c1a08c0 con 0x7ffa9c100e70 2024-04-21T09:07:37.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.054+0000 7ffa997fa640 1 -- 172.21.15.66:0/1429901583 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffa8c046020 con 0x7ffa9c100e70 2024-04-21T09:07:37.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.054+0000 7ffa997fa640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ffa840a3c40 0x7ffa840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.054+0000 7ffa9b7fe640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ffa840a3c40 0x7ffa840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.055+0000 7ffa997fa640 1 -- 172.21.15.66:0/1429901583 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7ffa8c0f2c40 con 0x7ffa9c100e70 2024-04-21T09:07:37.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.055+0000 7ffa9b7fe640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ffa840a3c40 0x7ffa840a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c1996b0 tx=0x7ffa90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.059+0000 7ffa997fa640 1 -- 172.21.15.66:0/1429901583 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa8c0f9080 con 0x7ffa9c100e70 2024-04-21T09:07:37.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.247+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa9c1a0bb0 con 0x7ffa9c100e70 2024-04-21T09:07:37.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.248+0000 7ffa997fa640 1 -- 172.21.15.66:0/1429901583 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24616 (secure 0 0 0) 0x7ffa8c04d020 con 0x7ffa9c100e70 2024-04-21T09:07:37.249 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:37.249 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:06:40.160656+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:07:37.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.250+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ffa840a3c40 msgr2=0x7ffa840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.250+0000 7ffaa242c640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ffa840a3c40 0x7ffa840a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c1996b0 tx=0x7ffa90005f90 comp rx=0 tx=0).stop 2024-04-21T09:07:37.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.250+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 msgr2=0x7ffa9c1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.250+0000 7ffaa242c640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c1981a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c004040 comp rx=0 tx=0).stop 2024-04-21T09:07:37.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 shutdown_connections 2024-04-21T09:07:37.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ffa840a3c40 0x7ffa840a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa9c102980 0x7ffa9c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa9c101fc0 0x7ffa9c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 --2- 172.21.15.66:0/1429901583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa9c100e70 0x7ffa9c1981a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 >> 172.21.15.66:0/1429901583 conn(0x7ffa9c0fc3e0 msgr2=0x7ffa9c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 shutdown_connections 2024-04-21T09:07:37.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.251+0000 7ffaa242c640 1 -- 172.21.15.66:0/1429901583 wait complete. 2024-04-21T09:07:37.263 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 -- 172.21.15.66:0/802234470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 msgr2=0x7fbe7c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 --2- 172.21.15.66:0/802234470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c109330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009ca0 tx=0x7fbe7402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 -- 172.21.15.66:0/802234470 shutdown_connections 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 --2- 172.21.15.66:0/802234470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c109330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 --2- 172.21.15.66:0/802234470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7c102050 0x7fbe7c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 --2- 172.21.15.66:0/802234470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7c100e70 0x7fbe7c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 -- 172.21.15.66:0/802234470 >> 172.21.15.66:0/802234470 conn(0x7fbe7c0fc470 msgr2=0x7fbe7c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 -- 172.21.15.66:0/802234470 shutdown_connections 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.364+0000 7fbe81367640 1 -- 172.21.15.66:0/802234470 wait complete. 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 Processor -- start 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 -- start start 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7c100e70 0x7fbe7c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7c102050 0x7fbe7c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c10ca20 con 0x7fbe7c102a10 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c10c8a0 con 0x7fbe7c102050 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe81367640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c10cba0 con 0x7fbe7c100e70 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43800/0 (socket says 172.21.15.66:43800) 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 -- 172.21.15.66:0/1292435563 learned_addr learned my addr 172.21.15.66:0/1292435563 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7a7fc640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7c102050 0x7fbe7c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7affd640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7c100e70 0x7fbe7c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 -- 172.21.15.66:0/1292435563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7c100e70 msgr2=0x7fbe7c198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7c100e70 0x7fbe7c198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 -- 172.21.15.66:0/1292435563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7c102050 msgr2=0x7fbe7c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7c102050 0x7fbe7c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.365+0000 7fbe7b7fe640 1 -- 172.21.15.66:0/1292435563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c19fd70 con 0x7fbe7c102a10 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.366+0000 7fbe7b7fe640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c19f700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe7402c700 tx=0x7fbe740346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.366+0000 7fbe63fff640 1 -- 172.21.15.66:0/1292435563 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74041020 con 0x7fbe7c102a10 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.366+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7c1a0000 con 0x7fbe7c102a10 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.366+0000 7fbe63fff640 1 -- 172.21.15.66:0/1292435563 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe74034ca0 con 0x7fbe7c102a10 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.366+0000 7fbe63fff640 1 -- 172.21.15.66:0/1292435563 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74039040 con 0x7fbe7c102a10 2024-04-21T09:07:37.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.366+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7c1a0540 con 0x7fbe7c102a10 2024-04-21T09:07:37.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.367+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7c1a07a0 con 0x7fbe7c102a10 2024-04-21T09:07:37.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.368+0000 7fbe63fff640 1 -- 172.21.15.66:0/1292435563 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe74044020 con 0x7fbe7c102a10 2024-04-21T09:07:37.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.369+0000 7fbe63fff640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe640a3c40 0x7fbe640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.369+0000 7fbe7affd640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe640a3c40 0x7fbe640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.369+0000 7fbe63fff640 1 -- 172.21.15.66:0/1292435563 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fbe740f2960 con 0x7fbe7c102a10 2024-04-21T09:07:37.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.369+0000 7fbe7affd640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe640a3c40 0x7fbe640a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c101bc0 tx=0x7fbe6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.372+0000 7fbe63fff640 1 -- 172.21.15.66:0/1292435563 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe7404b020 con 0x7fbe7c102a10 2024-04-21T09:07:37.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.599+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe7c1a0970 con 0x7fbe7c102a10 2024-04-21T09:07:37.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.600+0000 7fbe63fff640 1 -- 172.21.15.66:0/1292435563 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11867 (secure 0 0 0) 0x7fbe740bf0b0 con 0x7fbe7c102a10 2024-04-21T09:07:37.600 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:37.601 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:06:13.097844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24476,24490]},"id":1}]} 2024-04-21T09:07:37.601 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-04-21T09:07:37.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe640a3c40 msgr2=0x7fbe640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe640a3c40 0x7fbe640a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c101bc0 tx=0x7fbe6c005f90 comp rx=0 tx=0).stop 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 msgr2=0x7fbe7c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c19f700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe7402c700 tx=0x7fbe740346d0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 shutdown_connections 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbe640a3c40 0x7fbe640a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe7c102a10 0x7fbe7c19f700 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7c102050 0x7fbe7c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 --2- 172.21.15.66:0/1292435563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7c100e70 0x7fbe7c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 >> 172.21.15.66:0/1292435563 conn(0x7fbe7c0fc470 msgr2=0x7fbe7c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 shutdown_connections 2024-04-21T09:07:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.602+0000 7fbe81367640 1 -- 172.21.15.66:0/1292435563 wait complete. 2024-04-21T09:07:37.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:37 smithi066 ceph-mon[27953]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:37.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:37 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3471868710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:07:37.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:37 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2292094599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:37.613 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:37 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1429901583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:37.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T09:05:46.330734+0000', 'modified': '2024-04-21T09:06:13.097844+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24490, 'mds_2': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/2851815427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 2851815427}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 2851815427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/1405629184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 1405629184}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 1405629184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/1780100152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1780100152}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1780100152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.83:6837/1568837633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1568837633}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1568837633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.186:6839/599772329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 599772329}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 599772329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/4140026829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 4140026829}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 4140026829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24512, 24476, 24490]} 2024-04-21T09:07:37.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-21T09:07:37.614 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 -- 172.21.15.66:0/1663856619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 msgr2=0x7f8ff4107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/1663856619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4107750 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009900 tx=0x7f8fe802bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 -- 172.21.15.66:0/1663856619 shutdown_connections 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/1663856619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff4107c90 0x7f8ff410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/1663856619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4107750 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/1663856619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff4102a30 0x7f8ff4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 -- 172.21.15.66:0/1663856619 >> 172.21.15.66:0/1663856619 conn(0x7f8ff40fc440 msgr2=0x7f8ff40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 -- 172.21.15.66:0/1663856619 shutdown_connections 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.716+0000 7f8ff9d54640 1 -- 172.21.15.66:0/1663856619 wait complete. 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 Processor -- start 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 -- start start 2024-04-21T09:07:37.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff4102a30 0x7f8ff4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff4107c90 0x7f8ff419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410dab0 con 0x7f8ff4105360 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410d930 con 0x7f8ff4107c90 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff9d54640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410dc30 con 0x7f8ff4102a30 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43824/0 (socket says 172.21.15.66:43824) 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff2ffd640 1 -- 172.21.15.66:0/4071287271 learned_addr learned my addr 172.21.15.66:0/4071287271 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff4102a30 0x7f8ff4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.717+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff4107c90 0x7f8ff419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff2ffd640 1 -- 172.21.15.66:0/4071287271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff4102a30 msgr2=0x7f8ff4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff4102a30 0x7f8ff4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff2ffd640 1 -- 172.21.15.66:0/4071287271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff4107c90 msgr2=0x7f8ff419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff4107c90 0x7f8ff419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff2ffd640 1 -- 172.21.15.66:0/4071287271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff419fdf0 con 0x7f8ff4105360 2024-04-21T09:07:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4198760 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff0ff9640 1 -- 172.21.15.66:0/4071287271 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8043020 con 0x7f8ff4105360 2024-04-21T09:07:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff41a0080 con 0x7f8ff4105360 2024-04-21T09:07:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff0ff9640 1 -- 172.21.15.66:0/4071287271 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe8004aa0 con 0x7f8ff4105360 2024-04-21T09:07:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff0ff9640 1 -- 172.21.15.66:0/4071287271 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8003d00 con 0x7f8ff4105360 2024-04-21T09:07:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.718+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41a05d0 con 0x7f8ff4105360 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ff0ff9640 1 -- 172.21.15.66:0/4071287271 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8fe8046020 con 0x7f8ff4105360 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 Processor -- start 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 -- start start 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0ce40 0x555c0cd0d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0d830 0x555c0cd0dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0e2d0 0x555c0cd14eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555c0cc71b10 con 0x555c0cd0d830 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c0cc0f240 con 0x555c0cd0e2d0 2024-04-21T09:07:37.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ffca16740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555c0cc6dad0 con 0x555c0cd0ce40 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0d830 0x555c0cd0dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0d830 0x555c0cd0dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43828/0 (socket says 172.21.15.66:43828) 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ff37fe640 1 -- 172.21.15.66:0/4091126575 learned_addr learned my addr 172.21.15.66:0/4091126575 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.721+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0e2d0 0x555c0cd14eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff0ff9640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fdc0a3c40 0x7f8fdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0ce40 0x555c0cd0d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fdc0a3c40 0x7f8fdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff37fe640 1 -- 172.21.15.66:0/4091126575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0ce40 msgr2=0x555c0cd0d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0ce40 0x555c0cd0d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff37fe640 1 -- 172.21.15.66:0/4091126575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0e2d0 msgr2=0x555c0cd14eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0e2d0 0x555c0cd14eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff37fe640 1 -- 172.21.15.66:0/4091126575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0c9842d0 con 0x555c0cd0d830 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0e2d0 0x555c0cd14eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff0ff9640 1 -- 172.21.15.66:0/4071287271 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f8fe80f2cd0 con 0x7f8ff4105360 2024-04-21T09:07:37.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0d830 0x555c0cd0dcb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400bd10 tx=0x7f8fe40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a474c0864846ee9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8fd9ffb640 1 -- 172.21.15.66:0/4091126575 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004030 con 0x555c0cd0d830 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fdc0a3c40 0x7f8fdc0a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fec005c40 tx=0x7f8fec007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8fd9ffb640 1 -- 172.21.15.66:0/4091126575 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe40035f0 con 0x555c0cd0d830 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.722+0000 7f8fd9ffb640 1 -- 172.21.15.66:0/4091126575 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4011770 con 0x555c0cd0d830 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 -- 172.21.15.66:0/4091126575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0d830 msgr2=0x555c0cd0dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0d830 0x555c0cd0dcb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400bd10 tx=0x7f8fe40107a0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 -- 172.21.15.66:0/4091126575 shutdown_connections 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0e2d0 0x555c0cd14eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0d830 0x555c0cd0dcb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 --2- 172.21.15.66:0/4091126575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0ce40 0x555c0cd0d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 -- 172.21.15.66:0/4091126575 >> 172.21.15.66:0/4091126575 conn(0x555c0cd08f40 msgr2=0x555c0cd07490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 -- 172.21.15.66:0/4091126575 shutdown_connections 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 -- 172.21.15.66:0/4091126575 wait complete. 2024-04-21T09:07:37.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 -- start start 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.723+0000 7f8ffca16740 1 Processor -- start 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ffca16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd154b0 con 0x555c0cd0e2d0 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ffca16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c0cbfa190 con 0x555c0cd0ce40 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ffca16740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd15630 con 0x555c0cd0d830 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ffca16740 10 client.0 _ll_register_callbacks cb 0x555c0cbf58b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43834/0 (socket says 172.21.15.66:43834) 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 -- 172.21.15.66:0/713667075 learned_addr learned my addr 172.21.15.66:0/713667075 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 -- 172.21.15.66:0/713667075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 msgr2=0x555c0cd173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 -- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 msgr2=0x555c0cd16eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 -- 172.21.15.66:0/713667075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0ca4e6b0 con 0x555c0cd0e2d0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0043c0 tx=0x7f8fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badfa678f6d8d6da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8fd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/713667075 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec01d040 con 0x555c0cd0e2d0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.724+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/713667075 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f8fec011d20 con 0x555c0cd0e2d0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe40035f0 con 0x555c0cd0e2d0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/713667075 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8fec02f050 con 0x555c0cd0e2d0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 10 client.15216 fetch_fsmap learned FSMap version 24 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c0cd21860 con 0x555c0cd0e2d0 2024-04-21T09:07:37.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/713667075 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe40035f0 con 0x555c0cd0e2d0 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/713667075 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8fec021440 con 0x555c0cd0e2d0 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 10 client.15216 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 10 client.15216 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 20 client.15216 populate_metadata read hostname 'smithi066' 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd23fd0 0x555c0cd25440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 4 client.15216 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c0cc7f530 con 0x555c0cd23fd0 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.725+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd23fd0 0x555c0cd25440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.726+0000 7f8fd2ffd640 10 client.15216 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:07:37.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.726+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd23fd0 0x555c0cd25440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f73d7a59ed653bc2 server_cookie=3a059a8223b0300a in_seq=0 out_seq=0 2024-04-21T09:07:37.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/713667075 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c0cc7f530 con 0x555c0cd23fd0 2024-04-21T09:07:37.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8fd2ffd640 10 client.15216 handle_command_reply: tid=0 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 client.15216 shutdown 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 msgr2=0x555c0cd1e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0043c0 tx=0x7f8fec004a40 comp rx=0 tx=0).stop 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 shutdown_connections 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd23fd0 0x555c0cd25440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e430 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8fd2ffd640 0 client.15216 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 --2- 172.21.15.66:0/713667075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 >> 172.21.15.66:0/713667075 conn(0x555c0cd08f40 msgr2=0x555c0cd06a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 shutdown_connections 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 1 -- 172.21.15.66:0/713667075 wait complete. 2024-04-21T09:07:37.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.727+0000 7f8ffca16740 20 client.15216 trim_cache size 0 max 16384 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 Processor -- start 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 -- start start 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd23780 con 0x555c0cd0e2d0 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd23e50 con 0x555c0cd0ce40 2024-04-21T09:07:37.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.745+0000 7f8ffca16740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd24520 con 0x555c0cd0d830 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43848/0 (socket says 172.21.15.66:43848) 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 -- 172.21.15.66:0/1265530850 learned_addr learned my addr 172.21.15.66:0/1265530850 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 -- 172.21.15.66:0/1265530850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 msgr2=0x555c0cd173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 -- 172.21.15.66:0/1265530850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 msgr2=0x555c0cd16eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 -- 172.21.15.66:0/1265530850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0cd235b0 con 0x555c0cd0e2d0 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8fec007390 tx=0x7f8fec005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16f3b9235a6b7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8fd97fa640 1 -- 172.21.15.66:0/1265530850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec037070 con 0x555c0cd0e2d0 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8fd97fa640 1 -- 172.21.15.66:0/1265530850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fec026070 con 0x555c0cd0e2d0 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.746+0000 7f8fd97fa640 1 -- 172.21.15.66:0/1265530850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec018070 con 0x555c0cd0e2d0 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- 172.21.15.66:0/1265530850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 msgr2=0x555c0cd1e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8fec007390 tx=0x7f8fec005ef0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- 172.21.15.66:0/1265530850 shutdown_connections 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd1e350 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 --2- 172.21.15.66:0/1265530850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd16eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- 172.21.15.66:0/1265530850 >> 172.21.15.66:0/1265530850 conn(0x555c0cd08f40 msgr2=0x555c0cd11840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- 172.21.15.66:0/1265530850 shutdown_connections 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- 172.21.15.66:0/1265530850 wait complete. 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- start start 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 Processor -- start 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd252c0 con 0x555c0cd0e2d0 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd24bf0 con 0x555c0cd0ce40 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.747+0000 7f8ffca16740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555c0cd25990 con 0x555c0cd0d830 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ffca16740 10 client.0 _ll_register_callbacks cb 0x7f8fec01a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd12cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43852/0 (socket says 172.21.15.66:43852) 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 -- 172.21.15.66:0/3997661865 learned_addr learned my addr 172.21.15.66:0/3997661865 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff2ffd640 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 -- 172.21.15.66:0/3997661865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 msgr2=0x555c0cd1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd1e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 -- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 msgr2=0x555c0cd127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 -- 172.21.15.66:0/3997661865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0cd13230 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ff3fff640 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd12cf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0219d0 tx=0x7f8fec014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fd446c633b5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8fd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3997661865 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec037070 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3997661865 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f8fec004a40 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fec026070 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3997661865 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8fec04b050 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ffca16740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.748+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c0cd14530 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3997661865 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fec026070 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3997661865 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8fec043020 con 0x555c0cd0e2d0 2024-04-21T09:07:37.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ffca16740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ffca16740 10 client.15222 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ffca16740 20 client.15222 populate_metadata read hostname 'smithi066' 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ffca16740 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd8e690 0x555c0cd8d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ffca16740 4 client.15222 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "status"}) -- 0x555c0cc793a0 con 0x555c0cd8e690 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd8e690 0x555c0cd8d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8fd2ffd640 10 client.15222 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.749+0000 7f8ff37fe640 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd8e690 0x555c0cd8d3a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=489c9dd7a20c37df server_cookie=626849b44fde1737 in_seq=0 out_seq=0 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3997661865 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555c0cc793a0 con 0x555c0cd8e690 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8fd2ffd640 10 client.15222 handle_command_reply: tid=0 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 client.15222 shutdown 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 msgr2=0x555c0cd12cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd12cf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0219d0 tx=0x7f8fec014040 comp rx=0 tx=0).stop 2024-04-21T09:07:37.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 shutdown_connections 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x555c0cd8e690 0x555c0cd8d3a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555c0cd0e2d0 0x555c0cd12cf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c0cd0d830 0x555c0cd1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 --2- 172.21.15.66:0/3997661865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0cd0ce40 0x555c0cd127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 >> 172.21.15.66:0/3997661865 conn(0x555c0cd08f40 msgr2=0x555c0cd11840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8fd2ffd640 0 client.15222 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 shutdown_connections 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 1 -- 172.21.15.66:0/3997661865 wait complete. 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ffca16740 20 client.15222 trim_cache size 0 max 16384 2024-04-21T09:07:37.757 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 2, 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24476, 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 85.660528364000001, 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 24, 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 93, 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 91, 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 128.15886633599999 2024-04-21T09:07:37.758 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:07:37.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fdc0a3c40 msgr2=0x7f8fdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fdc0a3c40 0x7f8fdc0a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fec005c40 tx=0x7f8fec007520 comp rx=0 tx=0).stop 2024-04-21T09:07:37.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 msgr2=0x7f8ff4198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.750+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4198760 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8004270 comp rx=0 tx=0).stop 2024-04-21T09:07:37.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 shutdown_connections 2024-04-21T09:07:37.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8fdc0a3c40 0x7f8fdc0a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff4107c90 0x7f8ff419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff4105360 0x7f8ff4198760 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 --2- 172.21.15.66:0/4071287271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff4102a30 0x7f8ff4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 >> 172.21.15.66:0/4071287271 conn(0x7f8ff40fc440 msgr2=0x7f8ff4103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 shutdown_connections 2024-04-21T09:07:37.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.751+0000 7f8ff9d54640 1 -- 172.21.15.66:0/4071287271 wait complete. 2024-04-21T09:07:37.762 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T09:07:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:37 smithi186 ceph-mon[31505]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:37 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3471868710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:07:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:37 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2292094599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:37 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1429901583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:37.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:37 smithi083 ceph-mon[31413]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:37.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:37 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3471868710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:07:37.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:37 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2292094599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:37.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:37 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1429901583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 -- 172.21.15.66:0/151037829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114107c70 msgr2=0x7ff11410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 --2- 172.21.15.66:0/151037829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114107c70 0x7ff11410a0c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009ca0 tx=0x7ff10c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 -- 172.21.15.66:0/151037829 shutdown_connections 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 --2- 172.21.15.66:0/151037829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114107c70 0x7ff11410a0c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 --2- 172.21.15.66:0/151037829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114105340 0x7ff114107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 --2- 172.21.15.66:0/151037829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff114102a10 0x7ff114104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 -- 172.21.15.66:0/151037829 >> 172.21.15.66:0/151037829 conn(0x7ff1140fc400 msgr2=0x7ff1140fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 -- 172.21.15.66:0/151037829 shutdown_connections 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.864+0000 7ff11b681640 1 -- 172.21.15.66:0/151037829 wait complete. 2024-04-21T09:07:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 Processor -- start 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 -- start start 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114102a10 0x7ff1141981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114105340 0x7ff1141986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff114107c70 0x7ff11419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff11410dae0 con 0x7ff114105340 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff11410d960 con 0x7ff114102a10 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11b681640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff11410dc60 con 0x7ff114107c70 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff118c1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114105340 0x7ff1141986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff118c1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114105340 0x7ff1141986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43864/0 (socket says 172.21.15.66:43864) 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff118c1c640 1 -- 172.21.15.66:0/949875610 learned_addr learned my addr 172.21.15.66:0/949875610 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff11941d640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114102a10 0x7ff1141981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff118c1c640 1 -- 172.21.15.66:0/949875610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff114107c70 msgr2=0x7ff11419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.865+0000 7ff119c1e640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff114107c70 0x7ff11419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff118c1c640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff114107c70 0x7ff11419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff118c1c640 1 -- 172.21.15.66:0/949875610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114102a10 msgr2=0x7ff1141981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff118c1c640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114102a10 0x7ff1141981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff118c1c640 1 -- 172.21.15.66:0/949875610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11419fdd0 con 0x7ff114105340 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff119c1e640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff114107c70 0x7ff11419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff118c1c640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114105340 0x7ff1141986f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d770 tx=0x7ff10800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff1067fc640 1 -- 172.21.15.66:0/949875610 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108017070 con 0x7ff114105340 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff1067fc640 1 -- 172.21.15.66:0/949875610 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff108009c30 con 0x7ff114105340 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1141a00c0 con 0x7ff114105340 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff1067fc640 1 -- 172.21.15.66:0/949875610 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10800f620 con 0x7ff114105340 2024-04-21T09:07:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.866+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141a0660 con 0x7ff114105340 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.868+0000 7ff1067fc640 1 -- 172.21.15.66:0/949875610 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff10801a070 con 0x7ff114105340 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 Processor -- start 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 -- start start 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1793e40 0x55d2b1794260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1794cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b17952d0 0x55d2b179beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2b16f8b10 con 0x55d2b1793e40 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2b1696240 con 0x55d2b1794830 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11e276740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d2b16f4ad0 con 0x55d2b17952d0 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff1067fc640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1000a3cc0 0x7ff1000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1794cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff119c1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1793e40 0x55d2b1794260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff118c1c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b17952d0 0x55d2b179beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1794cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:43424/0 (socket says 172.21.15.66:43424) 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 -- 172.21.15.66:0/4079342283 learned_addr learned my addr 172.21.15.66:0/4079342283 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff119c1e640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1000a3cc0 0x7ff1000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff1067fc640 1 -- 172.21.15.66:0/949875610 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7ff1080d0330 con 0x7ff114105340 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 -- 172.21.15.66:0/4079342283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b17952d0 msgr2=0x55d2b179beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 --2- 172.21.15.66:0/4079342283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b17952d0 0x55d2b179beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 -- 172.21.15.66:0/4079342283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1793e40 msgr2=0x55d2b1794260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 --2- 172.21.15.66:0/4079342283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1793e40 0x55d2b1794260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.869+0000 7ff11941d640 1 -- 172.21.15.66:0/4079342283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b140b2d0 con 0x55d2b1794830 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.870+0000 7ff11941d640 1 --2- 172.21.15.66:0/4079342283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1794cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff110009d20 tx=0x7ff110011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111631a14bd0505d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.870+0000 7ff119c1e640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1000a3cc0 0x7ff1000a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55d2b178f200 tx=0x7ff10c02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.870+0000 7ff0ff7fe640 1 -- 172.21.15.66:0/4079342283 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11001e070 con 0x55d2b1794830 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.870+0000 7ff0ff7fe640 1 -- 172.21.15.66:0/4079342283 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff110019ae0 con 0x55d2b1794830 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.870+0000 7ff0ff7fe640 1 -- 172.21.15.66:0/4079342283 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11000ea80 con 0x55d2b1794830 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- 172.21.15.66:0/4079342283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 msgr2=0x55d2b1794cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 --2- 172.21.15.66:0/4079342283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1794cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff110009d20 tx=0x7ff110011810 comp rx=0 tx=0).stop 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- 172.21.15.66:0/4079342283 shutdown_connections 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 --2- 172.21.15.66:0/4079342283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b17952d0 0x55d2b179beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 --2- 172.21.15.66:0/4079342283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1794cb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 --2- 172.21.15.66:0/4079342283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1793e40 0x55d2b1794260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- 172.21.15.66:0/4079342283 >> 172.21.15.66:0/4079342283 conn(0x55d2b178ff40 msgr2=0x55d2b178e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- 172.21.15.66:0/4079342283 shutdown_connections 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- 172.21.15.66:0/4079342283 wait complete. 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- start start 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 Processor -- start 2024-04-21T09:07:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b179de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1794830 0x55d2b179e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b17952d0 0x55d2b17a5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2b179c4b0 con 0x55d2b1794830 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2b1681190 con 0x55d2b17952d0 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.871+0000 7ff11e276740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d2b179c630 con 0x55d2b1793e40 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff11e276740 10 client.0 _ll_register_callbacks cb 0x55d2b167c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1794830 0x55d2b179e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1794830 0x55d2b179e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43888/0 (socket says 172.21.15.66:43888) 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 -- 172.21.15.66:0/2196389488 learned_addr learned my addr 172.21.15.66:0/2196389488 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff119c1e640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b17952d0 0x55d2b17a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff11941d640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b179de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 -- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 msgr2=0x55d2b179de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b179de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 -- 172.21.15.66:0/2196389488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b17952d0 msgr2=0x55d2b17a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b17952d0 0x55d2b17a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 -- 172.21.15.66:0/2196389488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b14d56b0 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff11941d640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b179de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff118c1c640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1794830 0x55d2b179e3d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff108098bc0 tx=0x7ff10809c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af58c5471c9f1c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff0fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff0fcff9640 1 -- 172.21.15.66:0/2196389488 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108021040 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff0fcff9640 1 -- 172.21.15.66:0/2196389488 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7ff1080a0120 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff110019ae0 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff0fcff9640 1 -- 172.21.15.66:0/2196389488 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff1080d0160 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff11e276740 10 client.15234 fetch_fsmap learned FSMap version 24 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.872+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2b17a8840 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff0fcff9640 1 -- 172.21.15.66:0/2196389488 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff110019ae0 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff0fcff9640 1 -- 172.21.15.66:0/2196389488 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff108097f40 con 0x55d2b1794830 2024-04-21T09:07:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff11e276740 10 client.15234 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff11e276740 10 client.15234 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff11e276740 20 client.15234 populate_metadata read hostname 'smithi066' 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff11e276740 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b17aaf30 0x55d2b17ac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff11e276740 4 client.15234 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2b1706530 con 0x55d2b17aaf30 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.873+0000 7ff11941d640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b17aaf30 0x55d2b17ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.874+0000 7ff11941d640 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b17aaf30 0x55d2b17ac300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbdc6f995ad740e2 server_cookie=71ee7f6efa286032 in_seq=0 out_seq=0 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.874+0000 7ff0fcff9640 10 client.15234 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff0fcff9640 1 -- 172.21.15.66:0/2196389488 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2b1706530 con 0x55d2b17aaf30 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff0fcff9640 10 client.15234 handle_command_reply: tid=0 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 client.15234 shutdown 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1794830 msgr2=0x55d2b179e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1794830 0x55d2b179e3d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff108098bc0 tx=0x7ff10809c0c0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 shutdown_connections 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b17952d0 0x55d2b17a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b17aaf30 0x55d2b17ac300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b1794830 0x55d2b179e3d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 --2- 172.21.15.66:0/2196389488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b179de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff0fcff9640 0 client.15234 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 >> 172.21.15.66:0/2196389488 conn(0x55d2b178ff40 msgr2=0x55d2b178da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 shutdown_connections 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 1 -- 172.21.15.66:0/2196389488 wait complete. 2024-04-21T09:07:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.875+0000 7ff11e276740 20 client.15234 trim_cache size 0 max 16384 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 Processor -- start 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 -- start start 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b1798840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1798d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2b17aa860 con 0x55d2b17952d0 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2b17aadb0 con 0x55d2b1794830 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff11e276740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d2b17ab3e0 con 0x55d2b1793e40 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff119c1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff119c1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43894/0 (socket says 172.21.15.66:43894) 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.893+0000 7ff119c1e640 1 -- 172.21.15.66:0/3032127166 learned_addr learned my addr 172.21.15.66:0/3032127166 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff118c1c640 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1798d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff11941d640 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b1798840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff119c1e640 1 -- 172.21.15.66:0/3032127166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 msgr2=0x55d2b1798840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff119c1e640 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b1798840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff119c1e640 1 -- 172.21.15.66:0/3032127166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 msgr2=0x55d2b1798d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff119c1e640 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1798d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff119c1e640 1 -- 172.21.15.66:0/3032127166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b179bb10 con 0x55d2b17952d0 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff11941d640 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b1798840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff118c1c640 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1798d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff119c1e640 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a53d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009ca0 tx=0x7ff10c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f4466e40a8acc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff0feffd640 1 -- 172.21.15.66:0/3032127166 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c039030 con 0x55d2b17952d0 2024-04-21T09:07:37.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff0feffd640 1 -- 172.21.15.66:0/3032127166 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff10c054070 con 0x55d2b17952d0 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.894+0000 7ff0feffd640 1 -- 172.21.15.66:0/3032127166 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c04f6a0 con 0x55d2b17952d0 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- 172.21.15.66:0/3032127166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 msgr2=0x55d2b17a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a53d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009ca0 tx=0x7ff10c004a40 comp rx=0 tx=0).stop 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- 172.21.15.66:0/3032127166 shutdown_connections 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a53d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1794830 0x55d2b1798d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 --2- 172.21.15.66:0/3032127166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1793e40 0x55d2b1798840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- 172.21.15.66:0/3032127166 >> 172.21.15.66:0/3032127166 conn(0x55d2b178ff40 msgr2=0x55d2b179be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- 172.21.15.66:0/3032127166 shutdown_connections 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- 172.21.15.66:0/3032127166 wait complete. 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- start start 2024-04-21T09:07:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 Processor -- start 2024-04-21T09:07:37.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1793e40 0x55d2b17a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1794830 0x55d2b17a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2b17ac180 con 0x55d2b17952d0 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2b17abab0 con 0x55d2b1793e40 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.895+0000 7ff11e276740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d2b17ac850 con 0x55d2b1794830 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff11e276740 10 client.0 _ll_register_callbacks cb 0x7ff10800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43910/0 (socket says 172.21.15.66:43910) 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 -- 172.21.15.66:0/4010776118 learned_addr learned my addr 172.21.15.66:0/4010776118 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff118c1c640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1794830 0x55d2b17a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff11941d640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1793e40 0x55d2b17a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 -- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1794830 msgr2=0x55d2b17a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1794830 0x55d2b17a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 -- 172.21.15.66:0/4010776118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1793e40 msgr2=0x55d2b17a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1793e40 0x55d2b17a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 -- 172.21.15.66:0/4010776118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b17a4380 con 0x55d2b17952d0 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff118c1c640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1794830 0x55d2b17a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:37.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff11941d640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1793e40 0x55d2b17a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff119c1e640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a3e40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff10c004af0 tx=0x7ff10c0325c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682d3993c18ad23d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff0fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff0fcff9640 1 -- 172.21.15.66:0/4010776118 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c039070 con 0x55d2b17952d0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff10c054070 con 0x55d2b17952d0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff0fcff9640 1 -- 172.21.15.66:0/4010776118 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7ff10c04f850 con 0x55d2b17952d0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff0fcff9640 1 -- 172.21.15.66:0/4010776118 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff10c066050 con 0x55d2b17952d0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff11e276740 10 client.15240 fetch_fsmap learned FSMap version 24 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.896+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2b17a55f0 con 0x55d2b17952d0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff0fcff9640 1 -- 172.21.15.66:0/4010776118 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff10c054070 con 0x55d2b17952d0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff0fcff9640 1 -- 172.21.15.66:0/4010776118 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff10c0587f0 con 0x55d2b17952d0 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff11e276740 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff11e276740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff11e276740 20 client.15240 populate_metadata read hostname 'smithi066' 2024-04-21T09:07:37.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff11e276740 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b1815060 0x55d2b1813d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff11e276740 4 client.15240 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "status"}) -- 0x55d2b17003a0 con 0x55d2b1815060 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.897+0000 7ff11941d640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b1815060 0x55d2b1813d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff0fcff9640 10 client.15240 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11941d640 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b1815060 0x55d2b1813d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a633fcdc256e6344 server_cookie=9fa34e76365dc10c in_seq=0 out_seq=0 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff0fcff9640 1 -- 172.21.15.66:0/4010776118 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d2b17003a0 con 0x55d2b1815060 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff0fcff9640 10 client.15240 handle_command_reply: tid=0 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 client.15240 shutdown 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 msgr2=0x55d2b17a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a3e40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff10c004af0 tx=0x7ff10c0325c0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 shutdown_connections 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55d2b1815060 0x55d2b1813d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2b17952d0 0x55d2b17a3e40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b1794830 0x55d2b17a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 --2- 172.21.15.66:0/4010776118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2b1793e40 0x55d2b17a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff0fcff9640 0 client.15240 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 >> 172.21.15.66:0/4010776118 conn(0x55d2b178ff40 msgr2=0x55d2b179bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 shutdown_connections 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 1 -- 172.21.15.66:0/4010776118 wait complete. 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.898+0000 7ff11e276740 20 client.15240 trim_cache size 0 max 16384 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 1, 2024-04-21T09:07:37.906 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24490, 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 86.817633884000003, 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 24, 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 93, 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 91, 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 125.592950022 2024-04-21T09:07:37.907 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1000a3cc0 msgr2=0x7ff1000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1000a3cc0 0x7ff1000a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55d2b178f200 tx=0x7ff10c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114105340 msgr2=0x7ff1141986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114105340 0x7ff1141986f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d770 tx=0x7ff10800dc30 comp rx=0 tx=0).stop 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 shutdown_connections 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff1000a3cc0 0x7ff1000a6180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff114107c70 0x7ff11419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff114105340 0x7ff1141986f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 --2- 172.21.15.66:0/949875610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114102a10 0x7ff1141981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:37.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 >> 172.21.15.66:0/949875610 conn(0x7ff1140fc400 msgr2=0x7ff114103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:37.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 shutdown_connections 2024-04-21T09:07:37.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:37.899+0000 7ff11b681640 1 -- 172.21.15.66:0/949875610 wait complete. 2024-04-21T09:07:37.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/4221881572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 msgr2=0x7f2d681077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/4221881572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d681077b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009900 tx=0x7f2d5c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/4221881572 shutdown_connections 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/4221881572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d68107cf0 0x7f2d6810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/4221881572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d681077b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/4221881572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d68102a90 0x7f2d68104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/4221881572 >> 172.21.15.66:0/4221881572 conn(0x7f2d680fc440 msgr2=0x7f2d680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:38.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/4221881572 shutdown_connections 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.014+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/4221881572 wait complete. 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 Processor -- start 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 -- start start 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d68102a90 0x7f2d681982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d68198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d68107cf0 0x7f2d6819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6810db80 con 0x7f2d681053c0 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6810da00 con 0x7f2d68107cf0 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6cb9f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6810dd00 con 0x7f2d68102a90 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d68198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d68198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43926/0 (socket says 172.21.15.66:43926) 2024-04-21T09:07:38.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 -- 172.21.15.66:0/2606008798 learned_addr learned my addr 172.21.15.66:0/2606008798 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d6659c640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d68102a90 0x7f2d681982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d66d9d640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d68107cf0 0x7f2d6819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 -- 172.21.15.66:0/2606008798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d68102a90 msgr2=0x7f2d681982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d68102a90 0x7f2d681982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 -- 172.21.15.66:0/2606008798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d68107cf0 msgr2=0x7f2d6819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d68107cf0 0x7f2d6819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 -- 172.21.15.66:0/2606008798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6819fee0 con 0x7f2d681053c0 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.015+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d68198800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.016+0000 7f2d577fe640 1 -- 172.21.15.66:0/2606008798 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c043020 con 0x7f2d681053c0 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.016+0000 7f2d577fe640 1 -- 172.21.15.66:0/2606008798 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d5c0044c0 con 0x7f2d681053c0 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.016+0000 7f2d577fe640 1 -- 172.21.15.66:0/2606008798 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c003c90 con 0x7f2d681053c0 2024-04-21T09:07:38.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.016+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681a0170 con 0x7f2d681053c0 2024-04-21T09:07:38.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.016+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a0740 con 0x7f2d681053c0 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.018+0000 7f2d577fe640 1 -- 172.21.15.66:0/2606008798 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2d5c046020 con 0x7f2d681053c0 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.018+0000 7f2d6f78b740 1 Processor -- start 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.018+0000 7f2d6f78b740 1 -- start start 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f320e40 0x55932f321260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f321830 0x55932f321cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f3222d0 0x55932f328eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55932f285b10 con 0x55932f3222d0 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55932f223240 con 0x55932f321830 2024-04-21T09:07:38.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55932f281ad0 con 0x55932f320e40 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d65d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f3222d0 0x55932f328eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d65d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f3222d0 0x55932f328eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43938/0 (socket says 172.21.15.66:43938) 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d65d9b640 1 -- 172.21.15.66:0/4145892893 learned_addr learned my addr 172.21.15.66:0/4145892893 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6659c640 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f321830 0x55932f321cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d577fe640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2d500a3c40 0x7f2d500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d66d9d640 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f320e40 0x55932f321260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d66d9d640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2d500a3c40 0x7f2d500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d577fe640 1 -- 172.21.15.66:0/2606008798 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2d5c0f29d0 con 0x7f2d681053c0 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6659c640 1 -- 172.21.15.66:0/4145892893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f320e40 msgr2=0x55932f321260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6659c640 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f320e40 0x55932f321260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6659c640 1 -- 172.21.15.66:0/4145892893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f3222d0 msgr2=0x55932f328eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6659c640 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f3222d0 0x55932f328eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6659c640 1 -- 172.21.15.66:0/4145892893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55932ef982d0 con 0x55932f321830 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f3222d0 0x55932f328eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d6659c640 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f321830 0x55932f321cb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800a430 tx=0x7f2d580096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d13baba404331f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.019+0000 7f2d66d9d640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2d500a3c40 0x7f2d500a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d600058c0 tx=0x7f2d60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d54ff9640 1 -- 172.21.15.66:0/4145892893 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58013790 con 0x55932f321830 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d54ff9640 1 -- 172.21.15.66:0/4145892893 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d58019070 con 0x55932f321830 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d54ff9640 1 -- 172.21.15.66:0/4145892893 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58014c80 con 0x55932f321830 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 -- 172.21.15.66:0/4145892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f321830 msgr2=0x55932f321cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f321830 0x55932f321cb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800a430 tx=0x7f2d580096d0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 -- 172.21.15.66:0/4145892893 shutdown_connections 2024-04-21T09:07:38.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f3222d0 0x55932f328eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f321830 0x55932f321cb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/4145892893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f320e40 0x55932f321260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 -- 172.21.15.66:0/4145892893 >> 172.21.15.66:0/4145892893 conn(0x55932f31cf40 msgr2=0x55932f31b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.020+0000 7f2d6f78b740 1 -- 172.21.15.66:0/4145892893 shutdown_connections 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 -- 172.21.15.66:0/4145892893 wait complete. 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 -- start start 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 Processor -- start 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f32aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f32b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f332430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55932f3294b0 con 0x55932f320e40 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55932f20e190 con 0x55932f3222d0 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55932f329630 con 0x55932f321830 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6f78b740 10 client.0 _ll_register_callbacks cb 0x55932f2098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f32aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f32aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43946/0 (socket says 172.21.15.66:43946) 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d6659c640 1 -- 172.21.15.66:0/3620829846 learned_addr learned my addr 172.21.15.66:0/3620829846 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d66d9d640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f332430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.021+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f32b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6659c640 1 -- 172.21.15.66:0/3620829846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 msgr2=0x55932f32b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6659c640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f32b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6659c640 1 -- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 msgr2=0x55932f332430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6659c640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f332430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6659c640 1 -- 172.21.15.66:0/3620829846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55932f0626b0 con 0x55932f320e40 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d66d9d640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f332430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6659c640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f32aeb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009d80 tx=0x7f2d580146d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3fd933f69c1c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d4dffb640 1 -- 172.21.15.66:0/3620829846 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58013590 con 0x55932f320e40 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d4dffb640 1 -- 172.21.15.66:0/3620829846 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2d58013730 con 0x55932f320e40 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d58019070 con 0x55932f320e40 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d4dffb640 1 -- 172.21.15.66:0/3620829846 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2d58013930 con 0x55932f320e40 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6f78b740 10 client.15252 fetch_fsmap learned FSMap version 24 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55932f3357a0 con 0x55932f320e40 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d4dffb640 1 -- 172.21.15.66:0/3620829846 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d58019070 con 0x55932f320e40 2024-04-21T09:07:38.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d4dffb640 1 -- 172.21.15.66:0/3620829846 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2d5801eb90 con 0x55932f320e40 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6f78b740 10 client.15252 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.022+0000 7f2d6f78b740 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.023+0000 7f2d6f78b740 20 client.15252 populate_metadata read hostname 'smithi066' 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.023+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f337ad0 0x55932f338f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.023+0000 7f2d6f78b740 4 client.15252 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.023+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55932f293530 con 0x55932f337ad0 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.023+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f337ad0 0x55932f338f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.023+0000 7f2d4dffb640 10 client.15252 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.023+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f337ad0 0x55932f338f40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18649c8d89320413 server_cookie=b7ddd17828520020 in_seq=0 out_seq=0 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d4dffb640 1 -- 172.21.15.66:0/3620829846 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55932f293530 con 0x55932f337ad0 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d4dffb640 10 client.15252 handle_command_reply: tid=0 2024-04-21T09:07:38.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 client.15252 shutdown 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 msgr2=0x55932f32aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f32aeb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009d80 tx=0x7f2d580146d0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 shutdown_connections 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f337ad0 0x55932f338f40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f332430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f32b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/3620829846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f32aeb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 >> 172.21.15.66:0/3620829846 conn(0x55932f31cf40 msgr2=0x55932f31aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d4dffb640 0 client.15252 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 shutdown_connections 2024-04-21T09:07:38.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 1 -- 172.21.15.66:0/3620829846 wait complete. 2024-04-21T09:07:38.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.024+0000 7f2d6f78b740 20 client.15252 trim_cache size 0 max 16384 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 Processor -- start 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 -- start start 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f31a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f31a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f340210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55932f337280 con 0x55932f320e40 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55932f337950 con 0x55932f3222d0 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55932f338020 con 0x55932f321830 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f31a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f31a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43958/0 (socket says 172.21.15.66:43958) 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 -- 172.21.15.66:0/2385822742 learned_addr learned my addr 172.21.15.66:0/2385822742 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:38.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 -- 172.21.15.66:0/2385822742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 msgr2=0x55932f31a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 --2- 172.21.15.66:0/2385822742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f31a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 -- 172.21.15.66:0/2385822742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 msgr2=0x55932f340210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 --2- 172.21.15.66:0/2385822742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f340210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 -- 172.21.15.66:0/2385822742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55932f31c230 con 0x55932f320e40 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d6659c640 1 --2- 172.21.15.66:0/2385822742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f31a170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2d5801c500 tx=0x7f2d580094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d4df173db3e887 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d4ffff640 1 -- 172.21.15.66:0/2385822742 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58014770 con 0x55932f320e40 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d4ffff640 1 -- 172.21.15.66:0/2385822742 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d58019070 con 0x55932f320e40 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.043+0000 7f2d4ffff640 1 -- 172.21.15.66:0/2385822742 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5801cc80 con 0x55932f320e40 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- 172.21.15.66:0/2385822742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 msgr2=0x55932f31a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/2385822742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f31a170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2d5801c500 tx=0x7f2d580094a0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- 172.21.15.66:0/2385822742 shutdown_connections 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/2385822742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f340210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/2385822742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f31a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/2385822742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f31a170 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- 172.21.15.66:0/2385822742 >> 172.21.15.66:0/2385822742 conn(0x55932f31cf40 msgr2=0x55932f31b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- 172.21.15.66:0/2385822742 shutdown_connections 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- 172.21.15.66:0/2385822742 wait complete. 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- start start 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 Processor -- start 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f3230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f323610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f3244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55932f338dc0 con 0x55932f320e40 2024-04-21T09:07:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55932f3386f0 con 0x55932f3222d0 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.044+0000 7f2d6f78b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55932f339490 con 0x55932f321830 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6f78b740 10 client.0 _ll_register_callbacks cb 0x7f2d5c0bac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f3230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f3230d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43972/0 (socket says 172.21.15.66:43972) 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 -- 172.21.15.66:0/815851325 learned_addr learned my addr 172.21.15.66:0/815851325 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f323610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 -- 172.21.15.66:0/815851325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 msgr2=0x55932f323610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f323610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 -- 172.21.15.66:0/815851325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 msgr2=0x55932f3244d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f3244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 -- 172.21.15.66:0/815851325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55932f324ac0 con 0x55932f320e40 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f323610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:07:38.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6659c640 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f3230d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2d580070b0 tx=0x7f2d5800ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bbe1f1f8f1e1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d4dffb640 1 -- 172.21.15.66:0/815851325 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58041060 con 0x55932f320e40 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d58019070 con 0x55932f320e40 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d4dffb640 1 -- 172.21.15.66:0/815851325 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2d58013cd0 con 0x55932f320e40 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d4dffb640 1 -- 172.21.15.66:0/815851325 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2d58048050 con 0x55932f320e40 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6f78b740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.045+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55932f3a2380 con 0x55932f320e40 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d4dffb640 1 -- 172.21.15.66:0/815851325 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d58019070 con 0x55932f320e40 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d4dffb640 1 -- 172.21.15.66:0/815851325 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2d5801e850 con 0x55932f320e40 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d6f78b740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d6f78b740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:07:38.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d6f78b740 20 client.15258 populate_metadata read hostname 'smithi066' 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f3a6c10 0x55932f3a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d6f78b740 4 client.15258 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "status"}) -- 0x55932f28d3a0 con 0x55932f3a6c10 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.046+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f3a6c10 0x55932f3a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d65d9b640 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f3a6c10 0x55932f3a3050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2689290b53a9364 server_cookie=5eb3ed0f7252dcb8 in_seq=0 out_seq=0 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d4dffb640 10 client.15258 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d4dffb640 1 -- 172.21.15.66:0/815851325 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55932f28d3a0 con 0x55932f3a6c10 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d4dffb640 10 client.15258 handle_command_reply: tid=0 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 client.15258 shutdown 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 msgr2=0x55932f3230d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f3230d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2d580070b0 tx=0x7f2d5800ad80 comp rx=0 tx=0).stop 2024-04-21T09:07:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 shutdown_connections 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55932f3a6c10 0x55932f3a3050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55932f3222d0 0x55932f3244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d4dffb640 0 client.15258 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55932f321830 0x55932f323610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 --2- 172.21.15.66:0/815851325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55932f320e40 0x55932f3230d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 >> 172.21.15.66:0/815851325 conn(0x55932f31cf40 msgr2=0x55932f31aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 shutdown_connections 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 1 -- 172.21.15.66:0/815851325 wait complete. 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6f78b740 20 client.15258 trim_cache size 0 max 16384 2024-04-21T09:07:38.055 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24512, 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 111.082959751, 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 24, 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 93, 2024-04-21T09:07:38.056 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 91, 2024-04-21T09:07:38.057 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 117.182823766 2024-04-21T09:07:38.057 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:07:38.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2d500a3c40 msgr2=0x7f2d500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2d500a3c40 0x7f2d500a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d600058c0 tx=0x7f2d60005f90 comp rx=0 tx=0).stop 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 msgr2=0x7f2d68198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.047+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d68198800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c0040c0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 shutdown_connections 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2d500a3c40 0x7f2d500a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d68107cf0 0x7f2d6819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d681053c0 0x7f2d68198800 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 --2- 172.21.15.66:0/2606008798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d68102a90 0x7f2d681982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 >> 172.21.15.66:0/2606008798 conn(0x7f2d680fc440 msgr2=0x7f2d681034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 shutdown_connections 2024-04-21T09:07:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:38.048+0000 7f2d6cb9f640 1 -- 172.21.15.66:0/2606008798 wait complete. 2024-04-21T09:07:38.060 INFO:tasks.cephadm:Setup complete, yielding 2024-04-21T09:07:38.060 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T09:07:38.069 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:07:38.069 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-21T09:07:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:38.801 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:38 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1292435563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:38.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:38 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1292435563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:38.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:38 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1292435563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:07:39.769 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:39 smithi066 ceph-mon[27953]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:39.770 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:39 smithi066 ceph-mon[27953]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T09:07:39.770 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-21T09:07:39.770 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:39 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-21T09:07:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:39 smithi186 ceph-mon[31505]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:39 smithi186 ceph-mon[31505]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T09:07:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-21T09:07:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:39 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-21T09:07:39.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:39 smithi083 ceph-mon[31413]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:39 smithi083 ceph-mon[31413]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T09:07:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-21T09:07:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:39 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-21T09:07:40.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 -- 172.21.15.66:0/3849605817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8801015a0 msgr2=0x7fa88010b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 --2- 172.21.15.66:0/3849605817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8801015a0 0x7fa88010b170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa870009990 tx=0x7fa870009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 -- 172.21.15.66:0/3849605817 shutdown_connections 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 --2- 172.21.15.66:0/3849605817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880104180 0x7fa8801045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 --2- 172.21.15.66:0/3849605817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8801015a0 0x7fa88010b170 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 --2- 172.21.15.66:0/3849605817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa880100c00 0x7fa880101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 -- 172.21.15.66:0/3849605817 >> 172.21.15.66:0/3849605817 conn(0x7fa8800fc780 msgr2=0x7fa8800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 -- 172.21.15.66:0/3849605817 shutdown_connections 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.206+0000 7fa88754c700 1 -- 172.21.15.66:0/3849605817 wait complete. 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 Processor -- start 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 -- start start 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa880100c00 0x7fa88019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:40.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8801015a0 0x7fa88019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880104180 0x7fa8801a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010e5f0 con 0x7fa880100c00 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010e470 con 0x7fa880104180 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa88754c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010e770 con 0x7fa8801015a0 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa8852e8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa880100c00 0x7fa88019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa8852e8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa880100c00 0x7fa88019aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43986/0 (socket says 172.21.15.66:43986) 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa8852e8700 1 -- 172.21.15.66:0/3030836344 learned_addr learned my addr 172.21.15.66:0/3030836344 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa885ae9700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880104180 0x7fa8801a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa884ae7700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8801015a0 0x7fa88019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa8852e8700 1 -- 172.21.15.66:0/3030836344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8801015a0 msgr2=0x7fa88019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa8852e8700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8801015a0 0x7fa88019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa8852e8700 1 -- 172.21.15.66:0/3030836344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880104180 msgr2=0x7fa8801a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:40.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.207+0000 7fa8852e8700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880104180 0x7fa8801a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.208+0000 7fa8852e8700 1 -- 172.21.15.66:0/3030836344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8801a2ad0 con 0x7fa880100c00 2024-04-21T09:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.208+0000 7fa8852e8700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa880100c00 0x7fa88019aec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa878009f40 tx=0x7fa87800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.208+0000 7fa87e7fc700 1 -- 172.21.15.66:0/3030836344 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780092e0 con 0x7fa880100c00 2024-04-21T09:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.208+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8801a2d60 con 0x7fa880100c00 2024-04-21T09:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.208+0000 7fa87e7fc700 1 -- 172.21.15.66:0/3030836344 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa878003a00 con 0x7fa880100c00 2024-04-21T09:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.208+0000 7fa87e7fc700 1 -- 172.21.15.66:0/3030836344 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878004560 con 0x7fa880100c00 2024-04-21T09:07:40.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.208+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8801a31b0 con 0x7fa880100c00 2024-04-21T09:07:40.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.209+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8801a33e0 con 0x7fa880100c00 2024-04-21T09:07:40.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.210+0000 7fa87e7fc700 1 -- 172.21.15.66:0/3030836344 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa87801b030 con 0x7fa880100c00 2024-04-21T09:07:40.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.211+0000 7fa87e7fc700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa8680a3c40 0x7fa8680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:40.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.211+0000 7fa884ae7700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa8680a3c40 0x7fa8680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:40.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.211+0000 7fa87e7fc700 1 -- 172.21.15.66:0/3030836344 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8780d0f90 con 0x7fa880100c00 2024-04-21T09:07:40.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.211+0000 7fa884ae7700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa8680a3c40 0x7fa8680a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa874005950 tx=0x7fa8740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:40.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.215+0000 7fa87e7fc700 1 -- 172.21.15.66:0/3030836344 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa878022020 con 0x7fa880100c00 2024-04-21T09:07:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.445+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa880103430 con 0x7fa8680a3c40 2024-04-21T09:07:40.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.446+0000 7fa87e7fc700 1 -- 172.21.15.66:0/3030836344 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa880103430 con 0x7fa8680a3c40 2024-04-21T09:07:40.447 INFO:teuthology.orchestra.run.smithi066.stdout:Backend: cephadm 2024-04-21T09:07:40.447 INFO:teuthology.orchestra.run.smithi066.stdout:Available: Yes 2024-04-21T09:07:40.447 INFO:teuthology.orchestra.run.smithi066.stdout:Paused: No 2024-04-21T09:07:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.448+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa8680a3c40 msgr2=0x7fa8680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.448+0000 7fa88754c700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa8680a3c40 0x7fa8680a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa874005950 tx=0x7fa8740058e0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa880100c00 msgr2=0x7fa88019aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa880100c00 0x7fa88019aec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa878009f40 tx=0x7fa87800c3a0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 shutdown_connections 2024-04-21T09:07:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa8680a3c40 0x7fa8680a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880104180 0x7fa8801a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa880100c00 0x7fa88019aec0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 --2- 172.21.15.66:0/3030836344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8801015a0 0x7fa88019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 >> 172.21.15.66:0/3030836344 conn(0x7fa8800fc780 msgr2=0x7fa880101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 shutdown_connections 2024-04-21T09:07:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:40.449+0000 7fa88754c700 1 -- 172.21.15.66:0/3030836344 wait complete. 2024-04-21T09:07:40.757 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-21T09:07:40.757 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:40 smithi066 ceph-mon[27953]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-21T09:07:40.757 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:40 smithi066 ceph-mon[27953]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T09:07:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-21T09:07:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:40 smithi186 ceph-mon[31505]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-21T09:07:40.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:40 smithi186 ceph-mon[31505]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T09:07:40.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-21T09:07:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:40 smithi083 ceph-mon[31413]: from='mgr.14152 172.21.15.66:0/3862824916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-21T09:07:40.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:40 smithi083 ceph-mon[31413]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T09:07:41.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-21T09:07:41.343 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:41.786 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:41 smithi066 ceph-mon[27953]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-21T09:07:41.786 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:41 smithi066 ceph-mon[27953]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:41.786 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:41 smithi066 ceph-mon[27953]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T09:07:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:41 smithi186 ceph-mon[31505]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-21T09:07:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:41 smithi186 ceph-mon[31505]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:41 smithi186 ceph-mon[31505]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T09:07:41.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:41 smithi083 ceph-mon[31413]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-21T09:07:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:41 smithi083 ceph-mon[31413]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:41 smithi083 ceph-mon[31413]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T09:07:43.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.052+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1353974864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4105f20 msgr2=0x7f2dc4108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.052+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1353974864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4105f20 0x7f2dc4108310 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009990 tx=0x7f2db8009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.052+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1353974864 shutdown_connections 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.052+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1353974864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc41035f0 0x7f2dc41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.052+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1353974864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc4108850 0x7f2dc410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.052+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1353974864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4105f20 0x7f2dc4108310 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.052+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1353974864 >> 172.21.15.66:0/1353974864 conn(0x7f2dc40fcf60 msgr2=0x7f2dc40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.053+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1353974864 shutdown_connections 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.053+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1353974864 wait complete. 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.053+0000 7f2dca9a1700 1 Processor -- start 2024-04-21T09:07:43.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.053+0000 7f2dca9a1700 1 -- start start 2024-04-21T09:07:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.053+0000 7f2dca9a1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc41035f0 0x7f2dc4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.053+0000 7f2dca9a1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc4105f20 0x7f2dc41990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.053+0000 7f2dca9a1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4108850 0x7f2dc41a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dca9a1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc410e570 con 0x7f2dc4108850 2024-04-21T09:07:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dca9a1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc410e3f0 con 0x7f2dc4105f20 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dca9a1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc410e6f0 con 0x7f2dc41035f0 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4108850 0x7f2dc41a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4108850 0x7f2dc41a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43998/0 (socket says 172.21.15.66:43998) 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 -- 172.21.15.66:0/1719367917 learned_addr learned my addr 172.21.15.66:0/1719367917 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc37fe700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc4105f20 0x7f2dc41990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc3fff700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc41035f0 0x7f2dc4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 -- 172.21.15.66:0/1719367917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc41035f0 msgr2=0x7f2dc4198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc41035f0 0x7f2dc4198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 -- 172.21.15.66:0/1719367917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc4105f20 msgr2=0x7f2dc41990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc4105f20 0x7f2dc41990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 -- 172.21.15.66:0/1719367917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc41a07c0 con 0x7f2dc4108850 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc3fff700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc41035f0 0x7f2dc4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc8f3e700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4108850 0x7f2dc41a0120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc007d90 tx=0x7f2dbc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc17fa700 1 -- 172.21.15.66:0/1719367917 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc00ad00 con 0x7f2dc4108850 2024-04-21T09:07:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc17fa700 1 -- 172.21.15.66:0/1719367917 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc004030 con 0x7f2dc4108850 2024-04-21T09:07:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc41a0ab0 con 0x7f2dc4108850 2024-04-21T09:07:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.054+0000 7f2dc17fa700 1 -- 172.21.15.66:0/1719367917 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc012680 con 0x7f2dc4108850 2024-04-21T09:07:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.055+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc41a0fa0 con 0x7f2dc4108850 2024-04-21T09:07:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.055+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc4069730 con 0x7f2dc4108850 2024-04-21T09:07:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.057+0000 7f2dc17fa700 1 -- 172.21.15.66:0/1719367917 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2dbc003a00 con 0x7f2dc4108850 2024-04-21T09:07:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.057+0000 7f2dc17fa700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2dac0a3c40 0x7f2dac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.057+0000 7f2dc3fff700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2dac0a3c40 0x7f2dac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.057+0000 7f2dc17fa700 1 -- 172.21.15.66:0/1719367917 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f2dbc0d0da0 con 0x7f2dc4108850 2024-04-21T09:07:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.058+0000 7f2dc3fff700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2dac0a3c40 0x7f2dac0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2db4005950 tx=0x7f2db40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:43.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.061+0000 7f2dc17fa700 1 -- 172.21.15.66:0/1719367917 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2dbc09d3f0 con 0x7f2dc4108850 2024-04-21T09:07:43.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.280+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2dc4062e80 con 0x7f2dac0a3c40 2024-04-21T09:07:43.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.294+0000 7f2dc17fa700 1 -- 172.21.15.66:0/1719367917 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f2dc4062e80 con 0x7f2dac0a3c40 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.a smithi066 running (2m) 20s ago 2m 13.7M - 19.0.0-3154-ga630765c b45a994450e2 c81f341de805 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.b smithi083 running (2m) 55s ago 2m 28.2M - 19.0.0-3154-ga630765c b45a994450e2 9079479c6bb0 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.c smithi186 running (2m) 55s ago 2m 20.6M - 19.0.0-3154-ga630765c b45a994450e2 a7641693372a 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.d smithi066 running (2m) 20s ago 2m 16.4M - 19.0.0-3154-ga630765c b45a994450e2 5e59e9398166 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.e smithi083 running (2m) 55s ago 2m 15.8M - 19.0.0-3154-ga630765c b45a994450e2 e42708ebc0bf 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.f smithi186 running (2m) 55s ago 2m 17.4M - 19.0.0-3154-ga630765c b45a994450e2 deac478869e6 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.g smithi066 running (2m) 20s ago 2m 12.9M - 19.0.0-3154-ga630765c b45a994450e2 7422db858321 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.h smithi083 running (2m) 55s ago 2m 17.0M - 19.0.0-3154-ga630765c b45a994450e2 66486971edd0 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.i smithi186 running (2m) 55s ago 2m 19.5M - 19.0.0-3154-ga630765c b45a994450e2 51d59c974614 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.j smithi066 running (2m) 20s ago 2m 14.0M - 19.0.0-3154-ga630765c b45a994450e2 5ef9adaba20a 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.k smithi083 running (2m) 55s ago 2m 12.9M - 19.0.0-3154-ga630765c b45a994450e2 7c2f6bee9ced 2024-04-21T09:07:43.295 INFO:teuthology.orchestra.run.smithi066.stdout:mds.l smithi186 running (2m) 55s ago 2m 15.5M - 19.0.0-3154-ga630765c b45a994450e2 1bfc3d587b3c 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.x smithi066 *:9283,8765 running (9m) 20s ago 9m 516M - 19.0.0-3154-ga630765c b45a994450e2 d5f0a88d9d49 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.y smithi083 *:8443,8765 running (7m) 55s ago 7m 452M - 19.0.0-3154-ga630765c b45a994450e2 60da962a5603 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.z smithi186 *:8443,8765 running (7m) 55s ago 7m 452M - 19.0.0-3154-ga630765c b45a994450e2 26a6274ad30b 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:mon.a smithi066 running (9m) 20s ago 10m 50.5M 2048M 19.0.0-3154-ga630765c b45a994450e2 8ad485c04e09 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:mon.b smithi083 running (8m) 55s ago 8m 40.4M 2048M 19.0.0-3154-ga630765c b45a994450e2 a844b996d036 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:mon.c smithi186 running (8m) 55s ago 8m 39.1M 2048M 19.0.0-3154-ga630765c b45a994450e2 9bbbd0ab3cbc 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 smithi066 running (7m) 20s ago 7m 56.9M 4272M 19.0.0-3154-ga630765c b45a994450e2 99d4fe73114a 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 smithi083 running (7m) 55s ago 7m 47.1M 4272M 19.0.0-3154-ga630765c b45a994450e2 ce37e4c099c8 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 smithi186 running (6m) 55s ago 6m 46.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 c3317b071730 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 smithi066 running (6m) 20s ago 6m 44.3M 4272M 19.0.0-3154-ga630765c b45a994450e2 0aba80cbf42b 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 smithi083 running (5m) 55s ago 5m 43.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 b5417f2d446a 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 smithi186 running (5m) 55s ago 5m 58.2M 4272M 19.0.0-3154-ga630765c b45a994450e2 86ffb474f4af 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 smithi066 running (5m) 20s ago 5m 42.0M 4272M 19.0.0-3154-ga630765c b45a994450e2 3c8242b5e539 2024-04-21T09:07:43.296 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 smithi083 running (4m) 55s ago 4m 55.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 4fc4388edaf6 2024-04-21T09:07:43.297 INFO:teuthology.orchestra.run.smithi066.stdout:osd.8 smithi186 running (4m) 55s ago 4m 53.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 f0c5a6450c2d 2024-04-21T09:07:43.297 INFO:teuthology.orchestra.run.smithi066.stdout:osd.9 smithi066 running (3m) 20s ago 3m 45.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 aecfc98c6892 2024-04-21T09:07:43.297 INFO:teuthology.orchestra.run.smithi066.stdout:osd.10 smithi083 running (3m) 55s ago 3m 52.8M 4272M 19.0.0-3154-ga630765c b45a994450e2 7b37fef0a6cc 2024-04-21T09:07:43.297 INFO:teuthology.orchestra.run.smithi066.stdout:osd.11 smithi186 running (2m) 55s ago 2m 44.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 3cb7e3ddc6a5 2024-04-21T09:07:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2dac0a3c40 msgr2=0x7f2dac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2dac0a3c40 0x7f2dac0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2db4005950 tx=0x7f2db40058e0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4108850 msgr2=0x7f2dc41a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4108850 0x7f2dc41a0120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc007d90 tx=0x7f2dbc00a340 comp rx=0 tx=0).stop 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 shutdown_connections 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f2dac0a3c40 0x7f2dac0a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc4108850 0x7f2dc41a0120 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc41035f0 0x7f2dc4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 --2- 172.21.15.66:0/1719367917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc4105f20 0x7f2dc41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 >> 172.21.15.66:0/1719367917 conn(0x7f2dc40fcf60 msgr2=0x7f2dc4103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 shutdown_connections 2024-04-21T09:07:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:43.297+0000 7f2dca9a1700 1 -- 172.21.15.66:0/1719367917 wait complete. 2024-04-21T09:07:43.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:43 smithi186 ceph-mon[31505]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-21T09:07:43.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:43 smithi083 ceph-mon[31413]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-21T09:07:43.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:43 smithi066 ceph-mon[27953]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-21T09:07:44.112 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-21T09:07:44.363 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:44.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:44 smithi066 ceph-mon[27953]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:44.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:44 smithi186 ceph-mon[31505]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:44.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:44 smithi083 ceph-mon[31413]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:45.800 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:45 smithi066 ceph-mon[27953]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-21T09:07:45.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:45 smithi186 ceph-mon[31505]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-21T09:07:45.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:45 smithi083 ceph-mon[31413]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-21T09:07:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 -- 172.21.15.66:0/3874551601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 msgr2=0x7f32bc105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 --2- 172.21.15.66:0/3874551601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc105960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009990 tx=0x7f32b8009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 -- 172.21.15.66:0/3874551601 shutdown_connections 2024-04-21T09:07:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 --2- 172.21.15.66:0/3874551601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32bc1087d0 0x7f32bc10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 --2- 172.21.15.66:0/3874551601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc105960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 --2- 172.21.15.66:0/3874551601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32bc105ea0 0x7f32bc108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 -- 172.21.15.66:0/3874551601 >> 172.21.15.66:0/3874551601 conn(0x7f32bc0fcfa0 msgr2=0x7f32bc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 -- 172.21.15.66:0/3874551601 shutdown_connections 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.193+0000 7f32c447f700 1 -- 172.21.15.66:0/3874551601 wait complete. 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 Processor -- start 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 -- start start 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32bc105ea0 0x7f32bc199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32bc1087d0 0x7f32bc1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10e630 con 0x7f32bc103570 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10e4b0 con 0x7f32bc105ea0 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c447f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10e7b0 con 0x7f32bc1087d0 2024-04-21T09:07:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c221b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c221b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41218/0 (socket says 172.21.15.66:41218) 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c221b700 1 -- 172.21.15.66:0/3271107831 learned_addr learned my addr 172.21.15.66:0/3271107831 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c2a1c700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32bc1087d0 0x7f32bc1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.194+0000 7f32c1a1a700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32bc105ea0 0x7f32bc199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c221b700 1 -- 172.21.15.66:0/3271107831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32bc1087d0 msgr2=0x7f32bc1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c221b700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32bc1087d0 0x7f32bc1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c221b700 1 -- 172.21.15.66:0/3271107831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32bc105ea0 msgr2=0x7f32bc199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c221b700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32bc105ea0 0x7f32bc199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c221b700 1 -- 172.21.15.66:0/3271107831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc1a0940 con 0x7f32bc103570 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c2a1c700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32bc1087d0 0x7f32bc1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c221b700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc198cf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f32b80036f0 tx=0x7f32b8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32b37fe700 1 -- 172.21.15.66:0/3271107831 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8027070 con 0x7f32bc103570 2024-04-21T09:07:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32b37fe700 1 -- 172.21.15.66:0/3271107831 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b8004100 con 0x7f32bc103570 2024-04-21T09:07:46.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32bc1a0bd0 con 0x7f32bc103570 2024-04-21T09:07:46.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32b37fe700 1 -- 172.21.15.66:0/3271107831 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8022da0 con 0x7f32bc103570 2024-04-21T09:07:46.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.195+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32bc1a1020 con 0x7f32bc103570 2024-04-21T09:07:46.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.196+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32bc02df80 con 0x7f32bc103570 2024-04-21T09:07:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.198+0000 7f32b37fe700 1 -- 172.21.15.66:0/3271107831 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f32b802a070 con 0x7f32bc103570 2024-04-21T09:07:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.199+0000 7f32b37fe700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f32a80a3c40 0x7f32a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.199+0000 7f32c1a1a700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f32a80a3c40 0x7f32a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.199+0000 7f32b37fe700 1 -- 172.21.15.66:0/3271107831 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f32b80d1d40 con 0x7f32bc103570 2024-04-21T09:07:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.199+0000 7f32c1a1a700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f32a80a3c40 0x7f32a80a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32ac006fd0 tx=0x7f32ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:46.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.203+0000 7f32b37fe700 1 -- 172.21.15.66:0/3271107831 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32b8031020 con 0x7f32bc103570 2024-04-21T09:07:46.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.420+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f32bc062e80 con 0x7f32a80a3c40 2024-04-21T09:07:46.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.426+0000 7f32b37fe700 1 -- 172.21.15.66:0/3271107831 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f32bc062e80 con 0x7f32a80a3c40 2024-04-21T09:07:46.427 INFO:teuthology.orchestra.run.smithi066.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-21T09:07:46.427 INFO:teuthology.orchestra.run.smithi066.stdout:mds.all 12/12 59s ago 2m smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-21T09:07:46.427 INFO:teuthology.orchestra.run.smithi066.stdout:mgr 3/3 59s ago 8m smithi066=x;smithi083=y;smithi186=z;count:3 2024-04-21T09:07:46.427 INFO:teuthology.orchestra.run.smithi066.stdout:mon 3/3 59s ago 8m smithi066:172.21.15.66=a;smithi083:172.21.15.83=b;smithi186:172.21.15.186=c;count:3 2024-04-21T09:07:46.427 INFO:teuthology.orchestra.run.smithi066.stdout:osd 12 59s ago - 2024-04-21T09:07:46.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f32a80a3c40 msgr2=0x7f32a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f32a80a3c40 0x7f32a80a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32ac006fd0 tx=0x7f32ac008040 comp rx=0 tx=0).stop 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 msgr2=0x7f32bc198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc198cf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f32b80036f0 tx=0x7f32b8005e80 comp rx=0 tx=0).stop 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 shutdown_connections 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f32a80a3c40 0x7f32a80a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32bc1087d0 0x7f32bc1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32bc103570 0x7f32bc198cf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 --2- 172.21.15.66:0/3271107831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32bc105ea0 0x7f32bc199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.428+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 >> 172.21.15.66:0/3271107831 conn(0x7f32bc0fcfa0 msgr2=0x7f32bc104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.429+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 shutdown_connections 2024-04-21T09:07:46.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:46.429+0000 7f32c447f700 1 -- 172.21.15.66:0/3271107831 wait complete. 2024-04-21T09:07:47.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-21T09:07:47.463 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:47.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:47 smithi186 ceph-mon[31505]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-21T09:07:47.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:47 smithi186 ceph-mon[31505]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:47.854 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:47 smithi066 ceph-mon[27953]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-21T09:07:47.855 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:47 smithi066 ceph-mon[27953]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:47.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:47 smithi083 ceph-mon[31413]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-21T09:07:47.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:47 smithi083 ceph-mon[31413]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:49.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 -- 172.21.15.66:0/1369266857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78381019a0 msgr2=0x7f7838101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:49.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 --2- 172.21.15.66:0/1369266857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78381019a0 0x7f7838101dc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f78280099c0 tx=0x7f7828009cd0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 -- 172.21.15.66:0/1369266857 shutdown_connections 2024-04-21T09:07:49.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 --2- 172.21.15.66:0/1369266857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7838103540 0x7f7838109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 --2- 172.21.15.66:0/1369266857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78381019a0 0x7f7838101dc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 --2- 172.21.15.66:0/1369266857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7838102ba0 0x7f7838103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 -- 172.21.15.66:0/1369266857 >> 172.21.15.66:0/1369266857 conn(0x7f78380fcf60 msgr2=0x7f78380ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 -- 172.21.15.66:0/1369266857 shutdown_connections 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.246+0000 7f783ca84700 1 -- 172.21.15.66:0/1369266857 wait complete. 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 Processor -- start 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 -- start start 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78381019a0 0x7f7838198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7838102ba0 0x7f78381991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7838103540 0x7f78381a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d410 con 0x7f7838103540 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d290 con 0x7f7838102ba0 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f783ca84700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d590 con 0x7f78381019a0 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.247+0000 7f7836d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7838103540 0x7f78381a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7838103540 0x7f78381a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41236/0 (socket says 172.21.15.66:41236) 2024-04-21T09:07:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 -- 172.21.15.66:0/2979954658 learned_addr learned my addr 172.21.15.66:0/2979954658 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7835d9b700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7838102ba0 0x7f78381991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f783659c700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78381019a0 0x7f7838198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 -- 172.21.15.66:0/2979954658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78381019a0 msgr2=0x7f7838198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78381019a0 0x7f7838198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 -- 172.21.15.66:0/2979954658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7838102ba0 msgr2=0x7f78381991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7838102ba0 0x7f78381991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 -- 172.21.15.66:0/2979954658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78381a0870 con 0x7f7838103540 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7835d9b700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7838102ba0 0x7f78381991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f7836d9d700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7838103540 0x7f78381a01a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7830007ee0 tx=0x7f783000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f78277fe700 1 -- 172.21.15.66:0/2979954658 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783000ddd0 con 0x7f7838103540 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78381a0b60 con 0x7f7838103540 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f78277fe700 1 -- 172.21.15.66:0/2979954658 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7830004020 con 0x7f7838103540 2024-04-21T09:07:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f78277fe700 1 -- 172.21.15.66:0/2979954658 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830015660 con 0x7f7838103540 2024-04-21T09:07:49.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.248+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78381a10a0 con 0x7f7838103540 2024-04-21T09:07:49.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.249+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7838069730 con 0x7f7838103540 2024-04-21T09:07:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.251+0000 7f78277fe700 1 -- 172.21.15.66:0/2979954658 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f78300041d0 con 0x7f7838103540 2024-04-21T09:07:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.251+0000 7f78277fe700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f78200a3c40 0x7f78200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.252+0000 7f783659c700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f78200a3c40 0x7f78200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.252+0000 7f78277fe700 1 -- 172.21.15.66:0/2979954658 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f78300d0a20 con 0x7f7838103540 2024-04-21T09:07:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.252+0000 7f783659c700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f78200a3c40 0x7f78200a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7828009990 tx=0x7f7828005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:49.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.255+0000 7f78277fe700 1 -- 172.21.15.66:0/2979954658 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7830010070 con 0x7f7838103540 2024-04-21T09:07:49.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.468+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f783806b820 con 0x7f78200a3c40 2024-04-21T09:07:49.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.469+0000 7f78277fe700 1 -- 172.21.15.66:0/2979954658 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f783806b820 con 0x7f78200a3c40 2024-04-21T09:07:49.470 INFO:teuthology.orchestra.run.smithi066.stdout:HOST ADDR LABELS STATUS 2024-04-21T09:07:49.470 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 172.21.15.66 2024-04-21T09:07:49.470 INFO:teuthology.orchestra.run.smithi066.stdout:smithi083 172.21.15.83 2024-04-21T09:07:49.470 INFO:teuthology.orchestra.run.smithi066.stdout:smithi186 172.21.15.186 2024-04-21T09:07:49.470 INFO:teuthology.orchestra.run.smithi066.stdout:3 hosts in cluster 2024-04-21T09:07:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.471+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f78200a3c40 msgr2=0x7f78200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.471+0000 7f783ca84700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f78200a3c40 0x7f78200a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7828009990 tx=0x7f7828005fb0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.471+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7838103540 msgr2=0x7f78381a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.471+0000 7f783ca84700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7838103540 0x7f78381a01a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7830007ee0 tx=0x7f783000d3b0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 shutdown_connections 2024-04-21T09:07:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f78200a3c40 0x7f78200a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7838103540 0x7f78381a01a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78381019a0 0x7f7838198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 --2- 172.21.15.66:0/2979954658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7838102ba0 0x7f78381991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:49.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 >> 172.21.15.66:0/2979954658 conn(0x7f78380fcf60 msgr2=0x7f783806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:49.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 shutdown_connections 2024-04-21T09:07:49.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:49.472+0000 7f783ca84700 1 -- 172.21.15.66:0/2979954658 wait complete. 2024-04-21T09:07:49.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:49 smithi186 ceph-mon[31505]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 348 B/s wr, 7 op/s 2024-04-21T09:07:49.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:49 smithi083 ceph-mon[31413]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 348 B/s wr, 7 op/s 2024-04-21T09:07:49.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:49 smithi066 ceph-mon[27953]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 348 B/s wr, 7 op/s 2024-04-21T09:07:50.174 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-21T09:07:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:50.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:50 smithi186 ceph-mon[31505]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:50.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:50 smithi083 ceph-mon[31413]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:50.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:50 smithi066 ceph-mon[27953]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:51 smithi066 ceph-mon[27953]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T09:07:51.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:51 smithi186 ceph-mon[31505]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T09:07:51.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:51 smithi083 ceph-mon[31413]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T09:07:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 -- 172.21.15.66:0/713393806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc103540 msgr2=0x7fcfdc109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 --2- 172.21.15.66:0/713393806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc103540 0x7fcfdc109df0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40099c0 tx=0x7fcfd4009cd0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 -- 172.21.15.66:0/713393806 shutdown_connections 2024-04-21T09:07:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 --2- 172.21.15.66:0/713393806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc1019a0 0x7fcfdc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 --2- 172.21.15.66:0/713393806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc103540 0x7fcfdc109df0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 --2- 172.21.15.66:0/713393806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfdc102ba0 0x7fcfdc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 -- 172.21.15.66:0/713393806 >> 172.21.15.66:0/713393806 conn(0x7fcfdc0fcf40 msgr2=0x7fcfdc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 -- 172.21.15.66:0/713393806 shutdown_connections 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.204+0000 7fcfe4451700 1 -- 172.21.15.66:0/713393806 wait complete. 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 Processor -- start 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 -- start start 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc1019a0 0x7fcfdc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc102ba0 0x7fcfdc199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfdc103540 0x7fcfdc1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d420 con 0x7fcfdc102ba0 2024-04-21T09:07:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d2a0 con 0x7fcfdc1019a0 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe4451700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d5a0 con 0x7fcfdc103540 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe19ec700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc102ba0 0x7fcfdc199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe19ec700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc102ba0 0x7fcfdc199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41248/0 (socket says 172.21.15.66:41248) 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.205+0000 7fcfe19ec700 1 -- 172.21.15.66:0/216765058 learned_addr learned my addr 172.21.15.66:0/216765058 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe29ee700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfdc103540 0x7fcfdc1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe21ed700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc1019a0 0x7fcfdc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe19ec700 1 -- 172.21.15.66:0/216765058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfdc103540 msgr2=0x7fcfdc1a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe19ec700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfdc103540 0x7fcfdc1a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe19ec700 1 -- 172.21.15.66:0/216765058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc1019a0 msgr2=0x7fcfdc198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe19ec700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc1019a0 0x7fcfdc198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe19ec700 1 -- 172.21.15.66:0/216765058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc1a0750 con 0x7fcfdc102ba0 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe21ed700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc1019a0 0x7fcfdc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe19ec700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc102ba0 0x7fcfdc199030 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c9e0 tx=0x7fcfd000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfdb7fe700 1 -- 172.21.15.66:0/216765058 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0007ac0 con 0x7fcfdc102ba0 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfdb7fe700 1 -- 172.21.15.66:0/216765058 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd0007c60 con 0x7fcfdc102ba0 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc1a0a40 con 0x7fcfdc102ba0 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfdb7fe700 1 -- 172.21.15.66:0/216765058 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0018660 con 0x7fcfdc102ba0 2024-04-21T09:07:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.206+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a0fd0 con 0x7fcfdc102ba0 2024-04-21T09:07:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.207+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc069730 con 0x7fcfdc102ba0 2024-04-21T09:07:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.209+0000 7fcfdb7fe700 1 -- 172.21.15.66:0/216765058 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcfd000f450 con 0x7fcfdc102ba0 2024-04-21T09:07:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.209+0000 7fcfdb7fe700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcfc40a3cc0 0x7fcfc40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.209+0000 7fcfe21ed700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcfc40a3cc0 0x7fcfc40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.209+0000 7fcfdb7fe700 1 -- 172.21.15.66:0/216765058 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fcfd00d06a0 con 0x7fcfdc102ba0 2024-04-21T09:07:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.210+0000 7fcfe21ed700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcfc40a3cc0 0x7fcfc40a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc102a00 tx=0x7fcfcc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.214+0000 7fcfdb7fe700 1 -- 172.21.15.66:0/216765058 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfd0014030 con 0x7fcfdc102ba0 2024-04-21T09:07:52.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.432+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 --> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fcfdc0ff030 con 0x7fcfc40a3cc0 2024-04-21T09:07:52.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.434+0000 7fcfdb7fe700 1 -- 172.21.15.66:0/216765058 <== mgr.14152 v2:172.21.15.66:6800/3032531159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fcfdc0ff030 con 0x7fcfc40a3cc0 2024-04-21T09:07:52.434 INFO:teuthology.orchestra.run.smithi066.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-21T09:07:52.435 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T09:07:52.435 INFO:teuthology.orchestra.run.smithi066.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T09:07:52.435 INFO:teuthology.orchestra.run.smithi066.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-21T09:07:52.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcfc40a3cc0 msgr2=0x7fcfc40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:52.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcfc40a3cc0 0x7fcfc40a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc102a00 tx=0x7fcfcc009040 comp rx=0 tx=0).stop 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc102ba0 msgr2=0x7fcfdc199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc102ba0 0x7fcfdc199030 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c9e0 tx=0x7fcfd000ccf0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 shutdown_connections 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fcfc40a3cc0 0x7fcfc40a6180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc1019a0 0x7fcfdc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfdc103540 0x7fcfdc1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 --2- 172.21.15.66:0/216765058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfdc102ba0 0x7fcfdc199030 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:52.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 >> 172.21.15.66:0/216765058 conn(0x7fcfdc0fcf40 msgr2=0x7fcfdc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:52.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 shutdown_connections 2024-04-21T09:07:52.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:52.436+0000 7fcfe4451700 1 -- 172.21.15.66:0/216765058 wait complete. 2024-04-21T09:07:53.116 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T09:07:53.188 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:07:53.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-21T09:07:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:53.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:53 smithi186 ceph-mon[31505]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-21T09:07:53.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:53 smithi186 ceph-mon[31505]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:53.847 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:53 smithi066 ceph-mon[27953]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-21T09:07:53.847 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:53 smithi066 ceph-mon[27953]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:53.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:53 smithi083 ceph-mon[31413]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-21T09:07:53.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:53 smithi083 ceph-mon[31413]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:07:55.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 -- 172.21.15.66:0/2281759116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754103530 msgr2=0x7f8754105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:55.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 --2- 172.21.15.66:0/2281759116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754103530 0x7f8754105920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8744009990 tx=0x7f8744009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 -- 172.21.15.66:0/2281759116 shutdown_connections 2024-04-21T09:07:55.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 --2- 172.21.15.66:0/2281759116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8754108790 0x7f875410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 --2- 172.21.15.66:0/2281759116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754103530 0x7f8754105920 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 --2- 172.21.15.66:0/2281759116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8754105e60 0x7f8754108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 -- 172.21.15.66:0/2281759116 >> 172.21.15.66:0/2281759116 conn(0x7f87540fcf40 msgr2=0x7f87540ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 -- 172.21.15.66:0/2281759116 shutdown_connections 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.152+0000 7f8759e49700 1 -- 172.21.15.66:0/2281759116 wait complete. 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 Processor -- start 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 -- start start 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8754103530 0x7f875419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754105e60 0x7f875419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8754108790 0x7f87541a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f875410e540 con 0x7f8754105e60 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f875410e3c0 con 0x7f8754108790 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8759e49700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f875410e6c0 con 0x7f8754103530 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8752ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754105e60 0x7f875419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:55.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8752ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754105e60 0x7f875419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55068/0 (socket says 172.21.15.66:55068) 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.153+0000 7f8752ffd700 1 -- 172.21.15.66:0/3136573772 learned_addr learned my addr 172.21.15.66:0/3136573772 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8753fff700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8754108790 0x7f87541a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f87537fe700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8754103530 0x7f875419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8752ffd700 1 -- 172.21.15.66:0/3136573772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8754103530 msgr2=0x7f875419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8752ffd700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8754103530 0x7f875419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8752ffd700 1 -- 172.21.15.66:0/3136573772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8754108790 msgr2=0x7f87541a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8752ffd700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8754108790 0x7f87541a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8752ffd700 1 -- 172.21.15.66:0/3136573772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87541a2a80 con 0x7f8754105e60 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f87537fe700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8754103530 0x7f875419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8752ffd700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754105e60 0x7f875419b360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f874800d9b0 tx=0x7f874800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8750ff9700 1 -- 172.21.15.66:0/3136573772 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748017070 con 0x7f8754105e60 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8750ff9700 1 -- 172.21.15.66:0/3136573772 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8748004830 con 0x7f8754105e60 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87541a2d70 con 0x7f8754105e60 2024-04-21T09:07:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8750ff9700 1 -- 172.21.15.66:0/3136573772 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87480053a0 con 0x7f8754105e60 2024-04-21T09:07:55.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.154+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87541a3250 con 0x7f8754105e60 2024-04-21T09:07:55.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.155+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8754069730 con 0x7f8754105e60 2024-04-21T09:07:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.156+0000 7f8750ff9700 1 -- 172.21.15.66:0/3136573772 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f87480049e0 con 0x7f8754105e60 2024-04-21T09:07:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.157+0000 7f8750ff9700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f873c0a3cc0 0x7f873c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.157+0000 7f87537fe700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f873c0a3cc0 0x7f873c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.157+0000 7f8750ff9700 1 -- 172.21.15.66:0/3136573772 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f87480d0690 con 0x7f8754105e60 2024-04-21T09:07:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.158+0000 7f87537fe700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f873c0a3cc0 0x7f873c0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f874400b5c0 tx=0x7f8744005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.162+0000 7f8750ff9700 1 -- 172.21.15.66:0/3136573772 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8748020020 con 0x7f8754105e60 2024-04-21T09:07:55.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.421+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f875402df80 con 0x7f8754105e60 2024-04-21T09:07:55.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.421+0000 7f8750ff9700 1 -- 172.21.15.66:0/3136573772 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3048 (secure 0 0 0) 0x7f875402df80 con 0x7f8754105e60 2024-04-21T09:07:55.422 INFO:teuthology.orchestra.run.smithi066.stdout:e24 2024-04-21T09:07:55.422 INFO:teuthology.orchestra.run.smithi066.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:legacy client fscid: 1 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:Filesystem 'cephfs' (1) 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:fs_name cephfs 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 24 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:created 2024-04-21T09:05:46.330734+0000 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:modified 2024-04-21T09:06:13.097844+0000 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:tableserver 0 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:root 0 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:session_timeout 300 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:session_autoclose 300 2024-04-21T09:07:55.423 INFO:teuthology.orchestra.run.smithi066.stdout:max_file_size 1099511627776 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:max_xattr_size 65536 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:required_client_features {} 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:last_failure 0 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:last_failure_osd_epoch 0 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:max_mds 3 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:in 0,1,2 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:up {0=24512,1=24490,2=24476} 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:failed 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:damaged 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:stopped 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:data_pools [3,4] 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:metadata_pool 2 2024-04-21T09:07:55.424 INFO:teuthology.orchestra.run.smithi066.stdout:inline_data disabled 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:balancer 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:bal_rank_mask -1 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:standby_count_wanted 0 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:qdb_cluster leader: 24512 members: 24512,24476,24490 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b{0:24512} state up:active seq 3 addr [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c{0:24469} state up:standby-replay seq 1 addr [v2:172.21.15.186:6836/1780100152,v1:172.21.15.186:6837/1780100152] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.i{1:24490} state up:active seq 11 addr [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.f{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/1405629184,v1:172.21.15.186:6835/1405629184] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.h{2:24476} state up:active seq 12 addr [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.e{2:24440} state up:standby-replay seq 1 addr [v2:172.21.15.83:6834/2851815427,v1:172.21.15.83:6835/2851815427] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout:Standby daemons: 2024-04-21T09:07:55.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:55.426 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.66:6834/3512926718,v1:172.21.15.66:6835/3512926718] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.426 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.66:6836/2533275448,v1:172.21.15.66:6837/2533275448] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.426 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.g{-1:14610} state up:standby seq 1 addr [v2:172.21.15.66:6838/2337695414,v1:172.21.15.66:6839/2337695414] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.426 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a{-1:14622} state up:standby seq 1 addr [v2:172.21.15.66:6840/1874916254,v1:172.21.15.66:6841/1874916254] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.426 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.l{-1:24403} state up:standby seq 1 addr [v2:172.21.15.186:6832/446664633,v1:172.21.15.186:6833/446664633] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.426 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.83:6832/325280073,v1:172.21.15.83:6833/325280073] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f873c0a3cc0 msgr2=0x7f873c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f873c0a3cc0 0x7f873c0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f874400b5c0 tx=0x7f8744005fd0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754105e60 msgr2=0x7f875419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754105e60 0x7f875419b360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f874800d9b0 tx=0x7f874800dd70 comp rx=0 tx=0).stop 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 shutdown_connections 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f873c0a3cc0 0x7f873c0a6180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8754108790 0x7f87541a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8754103530 0x7f875419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 --2- 172.21.15.66:0/3136573772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8754105e60 0x7f875419b360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.424+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 >> 172.21.15.66:0/3136573772 conn(0x7f87540fcf40 msgr2=0x7f8754102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.425+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 shutdown_connections 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:55.425+0000 7f8759e49700 1 -- 172.21.15.66:0/3136573772 wait complete. 2024-04-21T09:07:55.427 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-04-21T09:07:55.743 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:55 smithi066 ceph-mon[27953]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:07:55.743 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:55 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3136573772' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T09:07:55.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:55 smithi186 ceph-mon[31505]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:07:55.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:55 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3136573772' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T09:07:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:55 smithi083 ceph-mon[31413]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:07:55.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:55 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3136573772' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T09:07:56.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-21T09:07:56.306 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:07:57.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:57 smithi066 ceph-mon[27953]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:57.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:57 smithi186 ceph-mon[31505]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:57.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:57 smithi083 ceph-mon[31413]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 -- 172.21.15.66:0/1501981345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054103d30 msgr2=0x7f2054104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 --2- 172.21.15.66:0/1501981345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054103d30 0x7f2054104190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2048009990 tx=0x7f2048009ca0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 -- 172.21.15.66:0/1501981345 shutdown_connections 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 --2- 172.21.15.66:0/1501981345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20541046d0 0x7f205410af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 --2- 172.21.15.66:0/1501981345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2054100d50 0x7f2054101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 --2- 172.21.15.66:0/1501981345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054103d30 0x7f2054104190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 -- 172.21.15.66:0/1501981345 >> 172.21.15.66:0/1501981345 conn(0x7f20540fc6d0 msgr2=0x7f20540feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 -- 172.21.15.66:0/1501981345 shutdown_connections 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.097+0000 7f205bad7700 1 -- 172.21.15.66:0/1501981345 wait complete. 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 Processor -- start 2024-04-21T09:07:58.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 -- start start 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054100d50 0x7f2054103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054103d30 0x7f2054101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20541046d0 0x7f2054102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f205410e610 con 0x7f2054100d50 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f205410e490 con 0x7f2054103d30 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205bad7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f205410e790 con 0x7f20541046d0 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f2059873700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054100d50 0x7f2054103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f2059873700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054100d50 0x7f2054103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55096/0 (socket says 172.21.15.66:55096) 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f2059873700 1 -- 172.21.15.66:0/3454106588 learned_addr learned my addr 172.21.15.66:0/3454106588 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f2059072700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054103d30 0x7f2054101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.098+0000 7f205a074700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20541046d0 0x7f2054102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2059873700 1 -- 172.21.15.66:0/3454106588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20541046d0 msgr2=0x7f2054102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2059873700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20541046d0 0x7f2054102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2059873700 1 -- 172.21.15.66:0/3454106588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054103d30 msgr2=0x7f2054101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2059873700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054103d30 0x7f2054101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2059873700 1 -- 172.21.15.66:0/3454106588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2054102ad0 con 0x7f2054100d50 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f205a074700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20541046d0 0x7f2054102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2059873700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054100d50 0x7f2054103690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f204c00b9c0 tx=0x7f204c00bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2052ffd700 1 -- 172.21.15.66:0/3454106588 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c019070 con 0x7f2054100d50 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2052ffd700 1 -- 172.21.15.66:0/3454106588 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f204c00ce00 con 0x7f2054100d50 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f2052ffd700 1 -- 172.21.15.66:0/3454106588 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c012340 con 0x7f2054100d50 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20541a2e90 con 0x7f2054100d50 2024-04-21T09:07:58.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.099+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20541a3310 con 0x7f2054100d50 2024-04-21T09:07:58.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.100+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205402df80 con 0x7f2054100d50 2024-04-21T09:07:58.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.102+0000 7f2052ffd700 1 -- 172.21.15.66:0/3454106588 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f204c01c070 con 0x7f2054100d50 2024-04-21T09:07:58.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.103+0000 7f2052ffd700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f203c0a3c40 0x7f203c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:58.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.103+0000 7f2059072700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f203c0a3c40 0x7f203c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:58.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.103+0000 7f2052ffd700 1 -- 172.21.15.66:0/3454106588 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f204c0cffb0 con 0x7f2054100d50 2024-04-21T09:07:58.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.103+0000 7f2059072700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f203c0a3c40 0x7f203c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f204800b5c0 tx=0x7f2048005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:58.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.106+0000 7f2052ffd700 1 -- 172.21.15.66:0/3454106588 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f204c09c600 con 0x7f2054100d50 2024-04-21T09:07:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.315+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f205402e1a0 con 0x7f2054100d50 2024-04-21T09:07:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.316+0000 7f2052ffd700 1 -- 172.21.15.66:0/3454106588 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) ==== 56+0+5639 (secure 0 0 0) 0x7f204c09bf00 con 0x7f2054100d50 2024-04-21T09:07:58.316 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 95 2024-04-21T09:07:58.316 INFO:teuthology.orchestra.run.smithi066.stdout:fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:created 2024-04-21T08:57:34.912480+0000 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:modified 2024-04-21T09:07:40.521019+0000 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:crush_version 27 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:full_ratio 0.95 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:backfillfull_ratio 0.9 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:nearfull_ratio 0.85 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:require_min_compat_client luminous 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:min_compat_client luminous 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:require_osd_release squid 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:stretch_mode_enabled false 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-21T09:07:58.317 INFO:teuthology.orchestra.run.smithi066.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:max_osd 12 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6802/1993480786,v1:172.21.15.66:6803/1993480786] [v2:172.21.15.66:6804/1993480786,v1:172.21.15.66:6805/1993480786] exists,up c5f4d872-e0ff-41aa-bd69-ac73da467920 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 up in weight 1 up_from 13 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6800/2284873950,v1:172.21.15.83:6801/2284873950] [v2:172.21.15.83:6802/2284873950,v1:172.21.15.83:6803/2284873950] exists,up 49f582ca-69a5-425c-a239-3f46fb57c283 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/993640486,v1:172.21.15.186:6801/993640486] [v2:172.21.15.186:6802/993640486,v1:172.21.15.186:6803/993640486] exists,up c0462467-e4ef-4fc1-8f53-5ce7742ff9af 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6810/4132171023,v1:172.21.15.66:6811/4132171023] [v2:172.21.15.66:6812/4132171023,v1:172.21.15.66:6813/4132171023] exists,up e6c91e9b-d431-49b2-8b56-562e85e16010 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 up in weight 1 up_from 34 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6808/2820280622,v1:172.21.15.83:6809/2820280622] [v2:172.21.15.83:6810/2820280622,v1:172.21.15.83:6811/2820280622] exists,up 0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 up in weight 1 up_from 39 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/3723805891,v1:172.21.15.186:6809/3723805891] [v2:172.21.15.186:6810/3723805891,v1:172.21.15.186:6811/3723805891] exists,up c32caead-4fab-4b5e-9bc5-7d1bb3877747 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6818/455614173,v1:172.21.15.66:6819/455614173] [v2:172.21.15.66:6820/455614173,v1:172.21.15.66:6821/455614173] exists,up 1ffb4890-5238-4ba0-9197-82b350fbdcd2 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 up in weight 1 up_from 52 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6816/1973931443,v1:172.21.15.83:6817/1973931443] [v2:172.21.15.83:6818/1973931443,v1:172.21.15.83:6819/1973931443] exists,up 8a6d8f44-b895-4c32-a527-c129c044aa7d 2024-04-21T09:07:58.318 INFO:teuthology.orchestra.run.smithi066.stdout:osd.8 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/2023460936,v1:172.21.15.186:6817/2023460936] [v2:172.21.15.186:6818/2023460936,v1:172.21.15.186:6819/2023460936] exists,up b2c7530c-00b0-49d6-9157-9c3c86f9b4a6 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:osd.9 up in weight 1 up_from 68 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6826/1483038287,v1:172.21.15.66:6827/1483038287] [v2:172.21.15.66:6828/1483038287,v1:172.21.15.66:6829/1483038287] exists,up 7b58b67c-416e-4de6-b41c-334ef88ea3ab 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:osd.10 up in weight 1 up_from 73 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6824/2551875994,v1:172.21.15.83:6825/2551875994] [v2:172.21.15.83:6826/2551875994,v1:172.21.15.83:6827/2551875994] exists,up 0a1a9369-e557-41ed-99a8-942cc6a1bd91 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:osd.11 up in weight 1 up_from 81 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/3472608146,v1:172.21.15.186:6825/3472608146] [v2:172.21.15.186:6826/3472608146,v1:172.21.15.186:6827/3472608146] exists,up 6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.b [8,11] 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.c [2,1] 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.f [8,11] 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.1b [2,4] 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.1 [4,9] 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.14 [4,6] 2024-04-21T09:07:58.319 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.25 [5,9] 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.2c [4,9] 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.6 [9,11] 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.9 [5,10] 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.13 [9,8] 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.1b [9,8] 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/612283217 expires 2024-04-22T08:58:38.211749+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6801/763565357 expires 2024-04-22T08:58:38.211749+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6800/3624224927 expires 2024-04-22T08:58:12.038096+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/2187684654 expires 2024-04-22T08:58:12.038096+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/3374606173 expires 2024-04-22T08:58:38.211749+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6800/763565357 expires 2024-04-22T08:58:38.211749+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/502604018 expires 2024-04-22T08:58:12.038096+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6801/3624224927 expires 2024-04-22T08:58:12.038096+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/2365699589 expires 2024-04-22T08:58:38.211749+0000 2024-04-21T09:07:58.320 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/3868723687 expires 2024-04-22T08:58:12.038096+0000 2024-04-21T09:07:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f203c0a3c40 msgr2=0x7f203c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f203c0a3c40 0x7f203c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f204800b5c0 tx=0x7f2048005fb0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054100d50 msgr2=0x7f2054103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054100d50 0x7f2054103690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f204c00b9c0 tx=0x7f204c00bcd0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 shutdown_connections 2024-04-21T09:07:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f203c0a3c40 0x7f203c0a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20541046d0 0x7f2054102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2054100d50 0x7f2054103690 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 --2- 172.21.15.66:0/3454106588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054103d30 0x7f2054101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 >> 172.21.15.66:0/3454106588 conn(0x7f20540fc6d0 msgr2=0x7f2054107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 shutdown_connections 2024-04-21T09:07:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:58.318+0000 7f205bad7700 1 -- 172.21.15.66:0/3454106588 wait complete. 2024-04-21T09:07:58.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:58 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3454106588' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T09:07:58.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:58 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3454106588' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T09:07:58.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:58 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3454106588' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T09:07:59.000 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-21T09:07:59.011 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:07:59.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 -- 172.21.15.66:0/3111906289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 msgr2=0x7fe3801092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 --2- 172.21.15.66:0/3111906289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe3801092a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe378009900 tx=0x7fe37802bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 -- 172.21.15.66:0/3111906289 shutdown_connections 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 --2- 172.21.15.66:0/3111906289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe3801092a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 --2- 172.21.15.66:0/3111906289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe380101fc0 0x7fe380102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 --2- 172.21.15.66:0/3111906289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380100e70 0x7fe380101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 -- 172.21.15.66:0/3111906289 >> 172.21.15.66:0/3111906289 conn(0x7fe3800fc3e0 msgr2=0x7fe3800fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 -- 172.21.15.66:0/3111906289 shutdown_connections 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 -- 172.21.15.66:0/3111906289 wait complete. 2024-04-21T09:07:59.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 Processor -- start 2024-04-21T09:07:59.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.121+0000 7fe3852d2640 1 -- start start 2024-04-21T09:07:59.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe3852d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380100e70 0x7fe3801980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe3852d2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe380101fc0 0x7fe380198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe3852d2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe38019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe3852d2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe38010c9d0 con 0x7fe380102980 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe3852d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe38010c850 con 0x7fe380100e70 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe3852d2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe38010cb50 con 0x7fe380101fc0 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe38019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe38019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55120/0 (socket says 172.21.15.66:55120) 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 -- 172.21.15.66:0/109328456 learned_addr learned my addr 172.21.15.66:0/109328456 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37e7fc640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe380101fc0 0x7fe380198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37effd640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380100e70 0x7fe3801980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 -- 172.21.15.66:0/109328456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe380101fc0 msgr2=0x7fe380198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe380101fc0 0x7fe380198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 -- 172.21.15.66:0/109328456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380100e70 msgr2=0x7fe3801980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380100e70 0x7fe3801980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 -- 172.21.15.66:0/109328456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38019fd00 con 0x7fe380102980 2024-04-21T09:07:59.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37effd640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380100e70 0x7fe3801980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37e7fc640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe380101fc0 0x7fe380198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe37f7fe640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe38019f690 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe378009230 tx=0x7fe3780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe367fff640 1 -- 172.21.15.66:0/109328456 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378043020 con 0x7fe380102980 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe367fff640 1 -- 172.21.15.66:0/109328456 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe378004aa0 con 0x7fe380102980 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38019ff90 con 0x7fe380102980 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.122+0000 7fe367fff640 1 -- 172.21.15.66:0/109328456 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378003c70 con 0x7fe380102980 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.123+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3801a0450 con 0x7fe380102980 2024-04-21T09:07:59.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.123+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3801a0770 con 0x7fe380102980 2024-04-21T09:07:59.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.125+0000 7fe367fff640 1 -- 172.21.15.66:0/109328456 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe378046020 con 0x7fe380102980 2024-04-21T09:07:59.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.125+0000 7fe367fff640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3680a3bb0 0x7fe3680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.125+0000 7fe37effd640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3680a3bb0 0x7fe3680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.125+0000 7fe367fff640 1 -- 172.21.15.66:0/109328456 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fe3780f2fa0 con 0x7fe380102980 2024-04-21T09:07:59.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.126+0000 7fe37effd640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3680a3bb0 0x7fe3680a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe370007e70 tx=0x7fe370007850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:59.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.129+0000 7fe367fff640 1 -- 172.21.15.66:0/109328456 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe37804d020 con 0x7fe380102980 2024-04-21T09:07:59.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.320+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3801a0a60 con 0x7fe380102980 2024-04-21T09:07:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.322+0000 7fe367fff640 1 -- 172.21.15.66:0/109328456 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24710 (secure 0 0 0) 0x7fe3780bf5f0 con 0x7fe380102980 2024-04-21T09:07:59.322 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:07:59.323 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:07:40.521019+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3680a3bb0 msgr2=0x7fe3680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3680a3bb0 0x7fe3680a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe370007e70 tx=0x7fe370007850 comp rx=0 tx=0).stop 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 msgr2=0x7fe38019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe38019f690 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe378009230 tx=0x7fe3780042f0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 shutdown_connections 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe3680a3bb0 0x7fe3680a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe380102980 0x7fe38019f690 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe380101fc0 0x7fe380198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 --2- 172.21.15.66:0/109328456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380100e70 0x7fe3801980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 >> 172.21.15.66:0/109328456 conn(0x7fe3800fc3e0 msgr2=0x7fe3800fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 shutdown_connections 2024-04-21T09:07:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.324+0000 7fe3852d2640 1 -- 172.21.15.66:0/109328456 wait complete. 2024-04-21T09:07:59.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:07:59.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:07:59.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 -- 172.21.15.66:0/1480041039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478102040 msgr2=0x7fa4781024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 --2- 172.21.15.66:0/1480041039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478102040 0x7fa4781024c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa474009900 tx=0x7fa47402bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:59.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 -- 172.21.15.66:0/1480041039 shutdown_connections 2024-04-21T09:07:59.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 --2- 172.21.15.66:0/1480041039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa478102a00 0x7fa478109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 --2- 172.21.15.66:0/1480041039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478102040 0x7fa4781024c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 --2- 172.21.15.66:0/1480041039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa478100e60 0x7fa478101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 -- 172.21.15.66:0/1480041039 >> 172.21.15.66:0/1480041039 conn(0x7fa4780fc440 msgr2=0x7fa4780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 -- 172.21.15.66:0/1480041039 shutdown_connections 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.442+0000 7fa47f352640 1 -- 172.21.15.66:0/1480041039 wait complete. 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 Processor -- start 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 -- start start 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478100e60 0x7fa478198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa478102040 0x7fa4781987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa478102a00 0x7fa47819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810ca10 con 0x7fa478100e60 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810c890 con 0x7fa478102040 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47f352640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810cb90 con 0x7fa478102a00 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478100e60 0x7fa478198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478100e60 0x7fa478198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55142/0 (socket says 172.21.15.66:55142) 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 -- 172.21.15.66:0/3964747635 learned_addr learned my addr 172.21.15.66:0/3964747635 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d8ef640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa478102a00 0x7fa47819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47c8ed640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa478102040 0x7fa4781987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 -- 172.21.15.66:0/3964747635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa478102a00 msgr2=0x7fa47819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa478102a00 0x7fa47819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 -- 172.21.15.66:0/3964747635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa478102040 msgr2=0x7fa4781987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa478102040 0x7fa4781987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 -- 172.21.15.66:0/3964747635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47819fe80 con 0x7fa478100e60 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d8ef640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa478102a00 0x7fa47819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa47d0ee640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478100e60 0x7fa478198260 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00d950 tx=0x7fa46c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:59.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.443+0000 7fa46a7fc640 1 -- 172.21.15.66:0/3964747635 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c017070 con 0x7fa478100e60 2024-04-21T09:07:59.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.444+0000 7fa46a7fc640 1 -- 172.21.15.66:0/3964747635 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c009c30 con 0x7fa478100e60 2024-04-21T09:07:59.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.444+0000 7fa46a7fc640 1 -- 172.21.15.66:0/3964747635 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00f620 con 0x7fa478100e60 2024-04-21T09:07:59.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.444+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781a0170 con 0x7fa478100e60 2024-04-21T09:07:59.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.444+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781a0670 con 0x7fa478100e60 2024-04-21T09:07:59.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.445+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4781a0930 con 0x7fa478100e60 2024-04-21T09:07:59.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.446+0000 7fa46a7fc640 1 -- 172.21.15.66:0/3964747635 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa46c01a070 con 0x7fa478100e60 2024-04-21T09:07:59.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.446+0000 7fa46a7fc640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa4640a3bb0 0x7fa4640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.447+0000 7fa47c8ed640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa4640a3bb0 0x7fa4640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.447+0000 7fa46a7fc640 1 -- 172.21.15.66:0/3964747635 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa46c0d02d0 con 0x7fa478100e60 2024-04-21T09:07:59.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.447+0000 7fa47c8ed640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa4640a3bb0 0x7fa4640a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa478199770 tx=0x7fa474005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:59.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.450+0000 7fa46a7fc640 1 -- 172.21.15.66:0/3964747635 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa46c009de0 con 0x7fa478100e60 2024-04-21T09:07:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:59 smithi066 ceph-mon[27953]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:59.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:07:59 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/109328456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:59.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.639+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa4781a0b00 con 0x7fa478100e60 2024-04-21T09:07:59.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.639+0000 7fa46a7fc640 1 -- 172.21.15.66:0/3964747635 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fa46c09c920 con 0x7fa478100e60 2024-04-21T09:07:59.640 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa4640a3bb0 msgr2=0x7fa4640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa4640a3bb0 0x7fa4640a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa478199770 tx=0x7fa474005be0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478100e60 msgr2=0x7fa478198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478100e60 0x7fa478198260 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00d950 tx=0x7fa46c00de10 comp rx=0 tx=0).stop 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 shutdown_connections 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa4640a3bb0 0x7fa4640a6070 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa478102a00 0x7fa47819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa478102040 0x7fa4781987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 --2- 172.21.15.66:0/3964747635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa478100e60 0x7fa478198260 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 >> 172.21.15.66:0/3964747635 conn(0x7fa4780fc440 msgr2=0x7fa47806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:59.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 shutdown_connections 2024-04-21T09:07:59.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.641+0000 7fa47f352640 1 -- 172.21.15.66:0/3964747635 wait complete. 2024-04-21T09:07:59.653 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:07:59.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 -- 172.21.15.66:0/2758721132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102040 msgr2=0x7f94641024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 --2- 172.21.15.66:0/2758721132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102040 0x7f94641024c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9458009900 tx=0x7f945802bc30 comp rx=0 tx=0).stop 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 -- 172.21.15.66:0/2758721132 shutdown_connections 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 --2- 172.21.15.66:0/2758721132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9464102a00 0x7f9464109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 --2- 172.21.15.66:0/2758721132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102040 0x7f94641024c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 --2- 172.21.15.66:0/2758721132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9464100e60 0x7f9464101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 -- 172.21.15.66:0/2758721132 >> 172.21.15.66:0/2758721132 conn(0x7f94640fc440 msgr2=0x7f94640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 -- 172.21.15.66:0/2758721132 shutdown_connections 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.758+0000 7f9468c3f640 1 -- 172.21.15.66:0/2758721132 wait complete. 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 Processor -- start 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 -- start start 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9464100e60 0x7f9464197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9464102040 0x7f94641976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102a00 0x7f946419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f946410ca60 con 0x7f9464102a00 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f946410c8e0 con 0x7f9464100e60 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9468c3f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f946410cbe0 con 0x7f9464102040 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102a00 0x7f946419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102a00 0x7f946419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55154/0 (socket says 172.21.15.66:55154) 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 -- 172.21.15.66:0/3061823382 learned_addr learned my addr 172.21.15.66:0/3061823382 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f946259c640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9464100e60 0x7f9464197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9461d9b640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9464102040 0x7f94641976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 -- 172.21.15.66:0/3061823382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9464102040 msgr2=0x7f94641976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9464102040 0x7f94641976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 -- 172.21.15.66:0/3061823382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9464100e60 msgr2=0x7f9464197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9464100e60 0x7f9464197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.759+0000 7f9462d9d640 1 -- 172.21.15.66:0/3061823382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946406a650 con 0x7f9464102a00 2024-04-21T09:07:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.760+0000 7f9462d9d640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102a00 0x7f946419e710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b970 tx=0x7f945c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.760+0000 7f94537fe640 1 -- 172.21.15.66:0/3061823382 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c019070 con 0x7f9464102a00 2024-04-21T09:07:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.760+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946406a940 con 0x7f9464102a00 2024-04-21T09:07:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.760+0000 7f94537fe640 1 -- 172.21.15.66:0/3061823382 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945c014440 con 0x7f9464102a00 2024-04-21T09:07:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.760+0000 7f94537fe640 1 -- 172.21.15.66:0/3061823382 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0075d0 con 0x7f9464102a00 2024-04-21T09:07:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.760+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946406af10 con 0x7f9464102a00 2024-04-21T09:07:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.761+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9464108250 con 0x7f9464102a00 2024-04-21T09:07:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.762+0000 7f94537fe640 1 -- 172.21.15.66:0/3061823382 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f945c01c070 con 0x7f9464102a00 2024-04-21T09:07:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.763+0000 7f94537fe640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f944c0a3bb0 0x7f944c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:07:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.763+0000 7f946259c640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f944c0a3bb0 0x7f944c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:07:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.763+0000 7f94537fe640 1 -- 172.21.15.66:0/3061823382 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f945c0cfd20 con 0x7f9464102a00 2024-04-21T09:07:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.763+0000 7f946259c640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f944c0a3bb0 0x7f944c0a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9464101bb0 tx=0x7f945400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:07:59.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.768+0000 7f94537fe640 1 -- 172.21.15.66:0/3061823382 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f945c09bd20 con 0x7f9464102a00 2024-04-21T09:07:59.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:59 smithi186 ceph-mon[31505]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:59.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:07:59 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/109328456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:59.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:59 smithi083 ceph-mon[31413]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-21T09:07:59.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:07:59 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/109328456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:07:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.953+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f946402d660 con 0x7f9464102a00 2024-04-21T09:07:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.954+0000 7f94537fe640 1 -- 172.21.15.66:0/3061823382 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f945c0a0020 con 0x7f9464102a00 2024-04-21T09:07:59.954 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:07:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f944c0a3bb0 msgr2=0x7f944c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f944c0a3bb0 0x7f944c0a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9464101bb0 tx=0x7f945400b390 comp rx=0 tx=0).stop 2024-04-21T09:07:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102a00 msgr2=0x7f946419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:07:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102a00 0x7f946419e710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b970 tx=0x7f945c00be30 comp rx=0 tx=0).stop 2024-04-21T09:07:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 shutdown_connections 2024-04-21T09:07:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f944c0a3bb0 0x7f944c0a6070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9464102a00 0x7f946419e710 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9464102040 0x7f94641976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 --2- 172.21.15.66:0/3061823382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9464100e60 0x7f9464197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:07:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.955+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 >> 172.21.15.66:0/3061823382 conn(0x7f94640fc440 msgr2=0x7f9464105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:07:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.956+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 shutdown_connections 2024-04-21T09:07:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:07:59.956+0000 7f9468c3f640 1 -- 172.21.15.66:0/3061823382 wait complete. 2024-04-21T09:07:59.967 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:00.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 -- 172.21.15.66:0/3715471517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 msgr2=0x7f82101023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 --2- 172.21.15.66:0/3715471517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f82101023d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8208009ca0 tx=0x7f820802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 -- 172.21.15.66:0/3715471517 shutdown_connections 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 --2- 172.21.15.66:0/3715471517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8210103b60 0x7f821010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 --2- 172.21.15.66:0/3715471517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82101031a0 0x7f8210103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 --2- 172.21.15.66:0/3715471517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f82101023d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 -- 172.21.15.66:0/3715471517 >> 172.21.15.66:0/3715471517 conn(0x7f82100fd520 msgr2=0x7f82100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 -- 172.21.15.66:0/3715471517 shutdown_connections 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 -- 172.21.15.66:0/3715471517 wait complete. 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.071+0000 7f8216fd5640 1 Processor -- start 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 -- start start 2024-04-21T09:08:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f8210075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82101031a0 0x7f8210074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8210103b60 0x7f8210074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f821010daf0 con 0x7f8210101fb0 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f821010d970 con 0x7f82101031a0 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f821010dc70 con 0x7f8210103b60 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f8210075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f8210075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55170/0 (socket says 172.21.15.66:55170) 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 -- 172.21.15.66:0/2953163769 learned_addr learned my addr 172.21.15.66:0/2953163769 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8215572640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8210103b60 0x7f8210074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 -- 172.21.15.66:0/2953163769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8210103b60 msgr2=0x7f8210074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8210103b60 0x7f8210074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 -- 172.21.15.66:0/2953163769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82101031a0 msgr2=0x7f8210074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f820ffff640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82101031a0 0x7f8210074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82101031a0 0x7f8210074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 -- 172.21.15.66:0/2953163769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8210075110 con 0x7f8210101fb0 2024-04-21T09:08:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8214d71640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f8210075d00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8208032380 tx=0x7f82080347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f820dffb640 1 -- 172.21.15.66:0/2953163769 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208039040 con 0x7f8210101fb0 2024-04-21T09:08:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f820dffb640 1 -- 172.21.15.66:0/2953163769 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8208044070 con 0x7f8210101fb0 2024-04-21T09:08:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82101a22a0 con 0x7f8210101fb0 2024-04-21T09:08:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.072+0000 7f820dffb640 1 -- 172.21.15.66:0/2953163769 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82080042a0 con 0x7f8210101fb0 2024-04-21T09:08:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.073+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82101a2840 con 0x7f8210101fb0 2024-04-21T09:08:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.074+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8210109350 con 0x7f8210101fb0 2024-04-21T09:08:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.075+0000 7f820dffb640 1 -- 172.21.15.66:0/2953163769 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8208003a00 con 0x7f8210101fb0 2024-04-21T09:08:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.075+0000 7f820dffb640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81f80a3c30 0x7f81f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.076+0000 7f820ffff640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81f80a3c30 0x7f81f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.076+0000 7f820dffb640 1 -- 172.21.15.66:0/2953163769 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f82080f2b80 con 0x7f8210101fb0 2024-04-21T09:08:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.076+0000 7f820ffff640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81f80a3c30 0x7f81f80a60f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8204005fd0 tx=0x7f8204005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:00.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.080+0000 7f820dffb640 1 -- 172.21.15.66:0/2953163769 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f820803e3a0 con 0x7f8210101fb0 2024-04-21T09:08:00.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.267+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8210062b50 con 0x7f8210101fb0 2024-04-21T09:08:00.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.268+0000 7f820dffb640 1 -- 172.21.15.66:0/2953163769 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f82080bf1d0 con 0x7f8210101fb0 2024-04-21T09:08:00.268 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81f80a3c30 msgr2=0x7f81f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81f80a3c30 0x7f81f80a60f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8204005fd0 tx=0x7f8204005c70 comp rx=0 tx=0).stop 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 msgr2=0x7f8210075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f8210075d00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8208032380 tx=0x7f82080347b0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 shutdown_connections 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f81f80a3c30 0x7f81f80a60f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8210103b60 0x7f8210074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82101031a0 0x7f8210074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 --2- 172.21.15.66:0/2953163769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8210101fb0 0x7f8210075d00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 >> 172.21.15.66:0/2953163769 conn(0x7f82100fd520 msgr2=0x7f8210106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 shutdown_connections 2024-04-21T09:08:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.270+0000 7f8216fd5640 1 -- 172.21.15.66:0/2953163769 wait complete. 2024-04-21T09:08:00.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:00.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.387+0000 7f6e9489a640 1 -- 172.21.15.66:0/2640367440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90102a10 msgr2=0x7f6e90104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.387+0000 7f6e9489a640 1 --2- 172.21.15.66:0/2640367440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90102a10 0x7f6e90104e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009900 tx=0x7f6e8802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:00.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 -- 172.21.15.66:0/2640367440 shutdown_connections 2024-04-21T09:08:00.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 --2- 172.21.15.66:0/2640367440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e90107c70 0x7f6e9010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 --2- 172.21.15.66:0/2640367440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105340 0x7f6e90107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 --2- 172.21.15.66:0/2640367440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90102a10 0x7f6e90104e00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 -- 172.21.15.66:0/2640367440 >> 172.21.15.66:0/2640367440 conn(0x7f6e900fc400 msgr2=0x7f6e900fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:00.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 -- 172.21.15.66:0/2640367440 shutdown_connections 2024-04-21T09:08:00.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 -- 172.21.15.66:0/2640367440 wait complete. 2024-04-21T09:08:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 Processor -- start 2024-04-21T09:08:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 -- start start 2024-04-21T09:08:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.388+0000 7f6e9489a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e90102a10 0x7f6e901982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e9489a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105340 0x7f6e90198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e9489a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90107c70 0x7f6e9019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e9489a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9010db30 con 0x7f6e90107c70 2024-04-21T09:08:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e9489a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9010d9b0 con 0x7f6e90102a10 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e9489a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9010dcb0 con 0x7f6e90105340 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90107c70 0x7f6e9019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90107c70 0x7f6e9019f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55194/0 (socket says 172.21.15.66:55194) 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 -- 172.21.15.66:0/3609583142 learned_addr learned my addr 172.21.15.66:0/3609583142 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8e59c640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e90102a10 0x7f6e901982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8dd9b640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105340 0x7f6e90198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 -- 172.21.15.66:0/3609583142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105340 msgr2=0x7f6e90198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105340 0x7f6e90198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 -- 172.21.15.66:0/3609583142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e90102a10 msgr2=0x7f6e901982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e90102a10 0x7f6e901982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 -- 172.21.15.66:0/3609583142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9019ff00 con 0x7f6e90107c70 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8dd9b640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105340 0x7f6e90198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e8ed9d640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90107c70 0x7f6e9019f890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009fd0 tx=0x7f6e8000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e877fe640 1 -- 172.21.15.66:0/3609583142 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80017070 con 0x7f6e90107c70 2024-04-21T09:08:00.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e901a01f0 con 0x7f6e90107c70 2024-04-21T09:08:00.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e877fe640 1 -- 172.21.15.66:0/3609583142 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e80003e20 con 0x7f6e90107c70 2024-04-21T09:08:00.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e877fe640 1 -- 172.21.15.66:0/3609583142 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80010460 con 0x7f6e90107c70 2024-04-21T09:08:00.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.389+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e901a06f0 con 0x7f6e90107c70 2024-04-21T09:08:00.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.390+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e900fdef0 con 0x7f6e90107c70 2024-04-21T09:08:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.391+0000 7f6e877fe640 1 -- 172.21.15.66:0/3609583142 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6e80003970 con 0x7f6e90107c70 2024-04-21T09:08:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.392+0000 7f6e877fe640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6e7c0a3bb0 0x7f6e7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.392+0000 7f6e8e59c640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6e7c0a3bb0 0x7f6e7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.392+0000 7f6e877fe640 1 -- 172.21.15.66:0/3609583142 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f6e800d05e0 con 0x7f6e90107c70 2024-04-21T09:08:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.393+0000 7f6e8e59c640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6e7c0a3bb0 0x7f6e7c0a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009230 tx=0x7f6e88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.397+0000 7f6e877fe640 1 -- 172.21.15.66:0/3609583142 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e8009cc30 con 0x7f6e90107c70 2024-04-21T09:08:00.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.581+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6e90062b50 con 0x7f6e90107c70 2024-04-21T09:08:00.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.582+0000 7f6e877fe640 1 -- 172.21.15.66:0/3609583142 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f6e8009c530 con 0x7f6e90107c70 2024-04-21T09:08:00.582 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:00.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6e7c0a3bb0 msgr2=0x7f6e7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6e7c0a3bb0 0x7f6e7c0a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009230 tx=0x7f6e88005fd0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90107c70 msgr2=0x7f6e9019f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90107c70 0x7f6e9019f890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009fd0 tx=0x7f6e8000b260 comp rx=0 tx=0).stop 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 shutdown_connections 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f6e7c0a3bb0 0x7f6e7c0a6070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e90107c70 0x7f6e9019f890 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105340 0x7f6e90198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 --2- 172.21.15.66:0/3609583142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e90102a10 0x7f6e901982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 >> 172.21.15.66:0/3609583142 conn(0x7f6e900fc400 msgr2=0x7f6e901034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 shutdown_connections 2024-04-21T09:08:00.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.584+0000 7f6e9489a640 1 -- 172.21.15.66:0/3609583142 wait complete. 2024-04-21T09:08:00.595 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 -- 172.21.15.66:0/1869775138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 msgr2=0x7fbb701024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/1869775138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701024d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009900 tx=0x7fbb6402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 -- 172.21.15.66:0/1869775138 shutdown_connections 2024-04-21T09:08:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/1869775138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb70102a10 0x7fbb70109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/1869775138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701024d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/1869775138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb70100e70 0x7fbb70101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 -- 172.21.15.66:0/1869775138 >> 172.21.15.66:0/1869775138 conn(0x7fbb700fc470 msgr2=0x7fbb700fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 -- 172.21.15.66:0/1869775138 shutdown_connections 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.696+0000 7fbb74b0d640 1 -- 172.21.15.66:0/1869775138 wait complete. 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 Processor -- start 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 -- start start 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb70100e70 0x7fbb70198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb70102a10 0x7fbb7019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7010ca10 con 0x7fbb70102050 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7010c890 con 0x7fbb70100e70 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb74b0d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7010cb90 con 0x7fbb70102a10 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb6dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb6dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55210/0 (socket says 172.21.15.66:55210) 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb6dd9b640 1 -- 172.21.15.66:0/529725995 learned_addr learned my addr 172.21.15.66:0/529725995 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.697+0000 7fbb6e59c640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb70100e70 0x7fbb70198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6ed9d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb70102a10 0x7fbb7019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6dd9b640 1 -- 172.21.15.66:0/529725995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb70102a10 msgr2=0x7fbb7019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6dd9b640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb70102a10 0x7fbb7019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6dd9b640 1 -- 172.21.15.66:0/529725995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb70100e70 msgr2=0x7fbb70198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6dd9b640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb70100e70 0x7fbb70198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6dd9b640 1 -- 172.21.15.66:0/529725995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7019fda0 con 0x7fbb70102050 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6ed9d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb70102a10 0x7fbb7019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6e59c640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb70100e70 0x7fbb70198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb6dd9b640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701986c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbb64006010 tx=0x7fbb640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb5f7fe640 1 -- 172.21.15.66:0/529725995 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64043020 con 0x7fbb70102050 2024-04-21T09:08:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb5f7fe640 1 -- 172.21.15.66:0/529725995 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb64004aa0 con 0x7fbb70102050 2024-04-21T09:08:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb701a0030 con 0x7fbb70102050 2024-04-21T09:08:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb5f7fe640 1 -- 172.21.15.66:0/529725995 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64003c70 con 0x7fbb70102050 2024-04-21T09:08:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.698+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb701a0580 con 0x7fbb70102050 2024-04-21T09:08:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.699+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb701a0780 con 0x7fbb70102050 2024-04-21T09:08:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.700+0000 7fbb5f7fe640 1 -- 172.21.15.66:0/529725995 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbb64046020 con 0x7fbb70102050 2024-04-21T09:08:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.701+0000 7fbb5f7fe640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbb580a3bb0 0x7fbb580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.701+0000 7fbb6e59c640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbb580a3bb0 0x7fbb580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.701+0000 7fbb5f7fe640 1 -- 172.21.15.66:0/529725995 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fbb640f2c10 con 0x7fbb70102050 2024-04-21T09:08:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.701+0000 7fbb6e59c640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbb580a3bb0 0x7fbb580a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb60005d00 tx=0x7fbb60005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.706+0000 7fbb5f7fe640 1 -- 172.21.15.66:0/529725995 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb6404d020 con 0x7fbb70102050 2024-04-21T09:08:00.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:00 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3964747635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:00 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3061823382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:00 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2953163769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:00 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3964747635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:00 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3061823382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:00 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2953163769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:00 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3964747635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:00 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3061823382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:00 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2953163769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:00.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.932+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb701a0950 con 0x7fbb70102050 2024-04-21T09:08:00.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.933+0000 7fbb5f7fe640 1 -- 172.21.15.66:0/529725995 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11867 (secure 0 0 0) 0x7fbb640bf260 con 0x7fbb70102050 2024-04-21T09:08:00.934 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:00.934 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:06:13.097844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24476,24490]},"id":1}]} 2024-04-21T09:08:00.934 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbb580a3bb0 msgr2=0x7fbb580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbb580a3bb0 0x7fbb580a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb60005d00 tx=0x7fbb60005c90 comp rx=0 tx=0).stop 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 msgr2=0x7fbb701986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701986c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbb64006010 tx=0x7fbb640042f0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 shutdown_connections 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fbb580a3bb0 0x7fbb580a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb70102a10 0x7fbb7019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb70102050 0x7fbb701986c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 --2- 172.21.15.66:0/529725995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb70100e70 0x7fbb70198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 >> 172.21.15.66:0/529725995 conn(0x7fbb700fc470 msgr2=0x7fbb7006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 shutdown_connections 2024-04-21T09:08:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:00.935+0000 7fbb74b0d640 1 -- 172.21.15.66:0/529725995 wait complete. 2024-04-21T09:08:00.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:01.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 -- 172.21.15.66:0/526669830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8100e80 msgr2=0x7fdfd81012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/526669830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8100e80 0x7fdfd81012a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009ca0 tx=0x7fdfd002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 -- 172.21.15.66:0/526669830 shutdown_connections 2024-04-21T09:08:01.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/526669830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd8102990 0x7fdfd81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/526669830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd8101fd0 0x7fdfd8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/526669830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8100e80 0x7fdfd81012a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 -- 172.21.15.66:0/526669830 >> 172.21.15.66:0/526669830 conn(0x7fdfd80fc410 msgr2=0x7fdfd80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 -- 172.21.15.66:0/526669830 shutdown_connections 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.048+0000 7fdfdf96d640 1 -- 172.21.15.66:0/526669830 wait complete. 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 Processor -- start 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 -- start start 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd8100e80 0x7fdfd819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8101fd0 0x7fdfd819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd8102990 0x7fdfd81a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd810c9d0 con 0x7fdfd8101fd0 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd810c850 con 0x7fdfd8102990 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdf96d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd810cb50 con 0x7fdfd8100e80 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdcf08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8101fd0 0x7fdfd819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdcf08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8101fd0 0x7fdfd819a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55220/0 (socket says 172.21.15.66:55220) 2024-04-21T09:08:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdcf08640 1 -- 172.21.15.66:0/115795428 learned_addr learned my addr 172.21.15.66:0/115795428 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfddf0a640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd8102990 0x7fdfd81a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdcf08640 1 -- 172.21.15.66:0/115795428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd8100e80 msgr2=0x7fdfd819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdcf08640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd8100e80 0x7fdfd819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdcf08640 1 -- 172.21.15.66:0/115795428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd8102990 msgr2=0x7fdfd81a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.049+0000 7fdfdcf08640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd8102990 0x7fdfd81a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfdcf08640 1 -- 172.21.15.66:0/115795428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd81a1ff0 con 0x7fdfd8101fd0 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfddf0a640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd8102990 0x7fdfd81a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfdcf08640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8101fd0 0x7fdfd819a910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e9c0 tx=0x7fdfc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfce7fc640 1 -- 172.21.15.66:0/115795428 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc800cd80 con 0x7fdfd8101fd0 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfce7fc640 1 -- 172.21.15.66:0/115795428 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc80035f0 con 0x7fdfd8101fd0 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd81a22e0 con 0x7fdfd8101fd0 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfce7fc640 1 -- 172.21.15.66:0/115795428 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8010690 con 0x7fdfd8101fd0 2024-04-21T09:08:01.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.050+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd81a2830 con 0x7fdfd8101fd0 2024-04-21T09:08:01.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.051+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd81a2af0 con 0x7fdfd8101fd0 2024-04-21T09:08:01.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.052+0000 7fdfce7fc640 1 -- 172.21.15.66:0/115795428 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdfc8004000 con 0x7fdfd8101fd0 2024-04-21T09:08:01.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.053+0000 7fdfce7fc640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.053+0000 7fdfdd709640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.053+0000 7fdfce7fc640 1 -- 172.21.15.66:0/115795428 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fdfc80d0d40 con 0x7fdfd8101fd0 2024-04-21T09:08:01.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.054+0000 7fdfdd709640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0006010 tx=0x7fdfd0032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:01.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.058+0000 7fdfce7fc640 1 -- 172.21.15.66:0/115795428 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfc809d390 con 0x7fdfd8101fd0 2024-04-21T09:08:01.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.243+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfd81a2cc0 con 0x7fdfd8101fd0 2024-04-21T09:08:01.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.245+0000 7fdfce7fc640 1 -- 172.21.15.66:0/115795428 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24710 (secure 0 0 0) 0x7fdfc809cc90 con 0x7fdfd8101fd0 2024-04-21T09:08:01.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:01.246 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:07:40.521019+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdfc40a3c30 msgr2=0x7fdfc40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0006010 tx=0x7fdfd0032340 comp rx=0 tx=0).stop 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8101fd0 msgr2=0x7fdfd819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8101fd0 0x7fdfd819a910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e9c0 tx=0x7fdfc800ee80 comp rx=0 tx=0).stop 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 shutdown_connections 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd8102990 0x7fdfd81a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd8101fd0 0x7fdfd819a910 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 --2- 172.21.15.66:0/115795428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd8100e80 0x7fdfd819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 >> 172.21.15.66:0/115795428 conn(0x7fdfd80fc410 msgr2=0x7fdfd80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:01.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 shutdown_connections 2024-04-21T09:08:01.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.247+0000 7fdfdf96d640 1 -- 172.21.15.66:0/115795428 wait complete. 2024-04-21T09:08:01.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:08:01.259 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:08:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.364+0000 7f036e4ba640 1 -- 172.21.15.66:0/597960420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03681029b0 msgr2=0x7f03681092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.364+0000 7f036e4ba640 1 --2- 172.21.15.66:0/597960420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03681029b0 0x7f03681092d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0360009ca0 tx=0x7f036002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- 172.21.15.66:0/597960420 shutdown_connections 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 --2- 172.21.15.66:0/597960420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03681029b0 0x7f03681092d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 --2- 172.21.15.66:0/597960420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0368101ff0 0x7f0368102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 --2- 172.21.15.66:0/597960420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0368100ea0 0x7f03681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- 172.21.15.66:0/597960420 >> 172.21.15.66:0/597960420 conn(0x7f03680fc410 msgr2=0x7f03680fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- 172.21.15.66:0/597960420 shutdown_connections 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- 172.21.15.66:0/597960420 wait complete. 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 Processor -- start 2024-04-21T09:08:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- start start 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0368100ea0 0x7f0368198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0368101ff0 0x7f0368198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03681029b0 0x7f036819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f036810ca50 con 0x7f0368101ff0 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f036810c8d0 con 0x7f0368100ea0 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.365+0000 7f036e4ba640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f036810cbd0 con 0x7f03681029b0 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0368101ff0 0x7f0368198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0368101ff0 0x7f0368198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55240/0 (socket says 172.21.15.66:55240) 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 -- 172.21.15.66:0/2701371587 learned_addr learned my addr 172.21.15.66:0/2701371587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f0367fff640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0368100ea0 0x7f0368198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f036ca57640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03681029b0 0x7f036819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 -- 172.21.15.66:0/2701371587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03681029b0 msgr2=0x7f036819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03681029b0 0x7f036819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 -- 172.21.15.66:0/2701371587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0368100ea0 msgr2=0x7f0368198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0368100ea0 0x7f0368198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 -- 172.21.15.66:0/2701371587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036819fe70 con 0x7f0368101ff0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03677fe640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0368101ff0 0x7f0368198790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f035800d770 tx=0x7f035800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03657fa640 1 -- 172.21.15.66:0/2701371587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358017070 con 0x7f0368101ff0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03657fa640 1 -- 172.21.15.66:0/2701371587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0358009cd0 con 0x7f0368101ff0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03681a0160 con 0x7f0368101ff0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f03657fa640 1 -- 172.21.15.66:0/2701371587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358003c60 con 0x7f0368101ff0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.366+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03681a06b0 con 0x7f0368101ff0 2024-04-21T09:08:01.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.367+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03681a0910 con 0x7f0368101ff0 2024-04-21T09:08:01.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.368+0000 7f03657fa640 1 -- 172.21.15.66:0/2701371587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0358004340 con 0x7f0368101ff0 2024-04-21T09:08:01.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.369+0000 7f03657fa640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f03540a3bb0 0x7f03540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.369+0000 7f0367fff640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f03540a3bb0 0x7f03540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.369+0000 7f03657fa640 1 -- 172.21.15.66:0/2701371587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f03580d0100 con 0x7f0368101ff0 2024-04-21T09:08:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.370+0000 7f0367fff640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f03540a3bb0 0x7f03540a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0350009990 tx=0x7f0350009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:01.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.374+0000 7f03657fa640 1 -- 172.21.15.66:0/2701371587 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f035809c750 con 0x7f0368101ff0 2024-04-21T09:08:01.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.560+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03681a0ae0 con 0x7f0368101ff0 2024-04-21T09:08:01.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.560+0000 7f03657fa640 1 -- 172.21.15.66:0/2701371587 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f035809c050 con 0x7f0368101ff0 2024-04-21T09:08:01.561 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:08:01.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f03540a3bb0 msgr2=0x7f03540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f03540a3bb0 0x7f03540a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0350009990 tx=0x7f0350009110 comp rx=0 tx=0).stop 2024-04-21T09:08:01.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0368101ff0 msgr2=0x7f0368198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0368101ff0 0x7f0368198790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f035800d770 tx=0x7f035800db70 comp rx=0 tx=0).stop 2024-04-21T09:08:01.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 shutdown_connections 2024-04-21T09:08:01.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f03540a3bb0 0x7f03540a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03681029b0 0x7f036819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0368101ff0 0x7f0368198790 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 --2- 172.21.15.66:0/2701371587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0368100ea0 0x7f0368198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 >> 172.21.15.66:0/2701371587 conn(0x7f03680fc410 msgr2=0x7f036806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:01.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 shutdown_connections 2024-04-21T09:08:01.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.562+0000 7f036e4ba640 1 -- 172.21.15.66:0/2701371587 wait complete. 2024-04-21T09:08:01.574 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:08:01.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 -- 172.21.15.66:0/2269595482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230105360 msgr2=0x7fa230107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 --2- 172.21.15.66:0/2269595482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230105360 0x7fa230107750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa224009900 tx=0x7fa22402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 -- 172.21.15.66:0/2269595482 shutdown_connections 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 --2- 172.21.15.66:0/2269595482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa230107c90 0x7fa23010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 --2- 172.21.15.66:0/2269595482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230105360 0x7fa230107750 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 --2- 172.21.15.66:0/2269595482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa230102a30 0x7fa230104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 -- 172.21.15.66:0/2269595482 >> 172.21.15.66:0/2269595482 conn(0x7fa2300fc440 msgr2=0x7fa2300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 -- 172.21.15.66:0/2269595482 shutdown_connections 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.679+0000 7fa235e63640 1 -- 172.21.15.66:0/2269595482 wait complete. 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 Processor -- start 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 -- start start 2024-04-21T09:08:01.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa230102a30 0x7fa230198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa230105360 0x7fa230198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230107c90 0x7fa23019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010daf0 con 0x7fa230107c90 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010d970 con 0x7fa230102a30 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa235e63640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010dc70 con 0x7fa230105360 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230107c90 0x7fa23019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230107c90 0x7fa23019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55262/0 (socket says 172.21.15.66:55262) 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa22ffff640 1 -- 172.21.15.66:0/1970941365 learned_addr learned my addr 172.21.15.66:0/1970941365 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa22f7fe640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa230102a30 0x7fa230198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.680+0000 7fa22effd640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa230105360 0x7fa230198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22ffff640 1 -- 172.21.15.66:0/1970941365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa230105360 msgr2=0x7fa230198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22ffff640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa230105360 0x7fa230198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22ffff640 1 -- 172.21.15.66:0/1970941365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa230102a30 msgr2=0x7fa230198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22ffff640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa230102a30 0x7fa230198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22ffff640 1 -- 172.21.15.66:0/1970941365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23019fe10 con 0x7fa230107c90 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22effd640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa230105360 0x7fa230198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22ffff640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230107c90 0x7fa23019f7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa22800cce0 tx=0x7fa22800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22cff9640 1 -- 172.21.15.66:0/1970941365 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22800f450 con 0x7fa230107c90 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2301a0100 con 0x7fa230107c90 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22cff9640 1 -- 172.21.15.66:0/1970941365 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa22800faa0 con 0x7fa230107c90 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa22cff9640 1 -- 172.21.15.66:0/1970941365 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228018b70 con 0x7fa230107c90 2024-04-21T09:08:01.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.681+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a05d0 con 0x7fa230107c90 2024-04-21T09:08:01.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.682+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2300fdf30 con 0x7fa230107c90 2024-04-21T09:08:01.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.683+0000 7fa22cff9640 1 -- 172.21.15.66:0/1970941365 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa228018d10 con 0x7fa230107c90 2024-04-21T09:08:01.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.684+0000 7fa22cff9640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2180a3bb0 0x7fa2180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.684+0000 7fa22f7fe640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2180a3bb0 0x7fa2180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.684+0000 7fa22cff9640 1 -- 172.21.15.66:0/1970941365 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa2280d0c10 con 0x7fa230107c90 2024-04-21T09:08:01.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.684+0000 7fa22f7fe640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2180a3bb0 0x7fa2180a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa220005fd0 tx=0x7fa22000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:01.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.688+0000 7fa22cff9640 1 -- 172.21.15.66:0/1970941365 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa228014030 con 0x7fa230107c90 2024-04-21T09:08:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:01 smithi186 ceph-mon[31505]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:08:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:01 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3609583142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:01 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/529725995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:01 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/115795428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:01 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2701371587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:01.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:01 smithi083 ceph-mon[31413]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:08:01.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:01 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3609583142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:01.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:01 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/529725995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:01.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:01 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/115795428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:01.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:01 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2701371587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:01.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:01 smithi066 ceph-mon[27953]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-21T09:08:01.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:01 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3609583142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:01.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:01 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/529725995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:01.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:01 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/115795428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:01.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:01 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2701371587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:01.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.873+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa230062b50 con 0x7fa230107c90 2024-04-21T09:08:01.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.874+0000 7fa22cff9640 1 -- 172.21.15.66:0/1970941365 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fa22809d260 con 0x7fa230107c90 2024-04-21T09:08:01.874 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:01.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2180a3bb0 msgr2=0x7fa2180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2180a3bb0 0x7fa2180a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa220005fd0 tx=0x7fa22000b390 comp rx=0 tx=0).stop 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230107c90 msgr2=0x7fa23019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230107c90 0x7fa23019f7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa22800cce0 tx=0x7fa22800a620 comp rx=0 tx=0).stop 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 shutdown_connections 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fa2180a3bb0 0x7fa2180a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230107c90 0x7fa23019f7a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa230105360 0x7fa230198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 --2- 172.21.15.66:0/1970941365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa230102a30 0x7fa230198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 >> 172.21.15.66:0/1970941365 conn(0x7fa2300fc440 msgr2=0x7fa230103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 shutdown_connections 2024-04-21T09:08:01.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.876+0000 7fa235e63640 1 -- 172.21.15.66:0/1970941365 wait complete. 2024-04-21T09:08:01.888 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:01.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 -- 172.21.15.66:0/3345609132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff028101fe0 msgr2=0x7ff028102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3345609132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff028101fe0 0x7ff028102460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009900 tx=0x7ff01c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 -- 172.21.15.66:0/3345609132 shutdown_connections 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3345609132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0281029a0 0x7ff0281092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3345609132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff028101fe0 0x7ff028102460 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3345609132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff028100e90 0x7ff0281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 -- 172.21.15.66:0/3345609132 >> 172.21.15.66:0/3345609132 conn(0x7ff0280fc400 msgr2=0x7ff0280fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 -- 172.21.15.66:0/3345609132 shutdown_connections 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 -- 172.21.15.66:0/3345609132 wait complete. 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 Processor -- start 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.994+0000 7ff02cddb640 1 -- start start 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02cddb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff028100e90 0x7ff028104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02cddb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff028101fe0 0x7ff028103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02cddb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0281029a0 0x7ff028103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02cddb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810c9d0 con 0x7ff0281029a0 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02cddb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810c850 con 0x7ff028100e90 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02cddb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810cb50 con 0x7ff028101fe0 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0281029a0 0x7ff028103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0281029a0 0x7ff028103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55290/0 (socket says 172.21.15.66:55290) 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 -- 172.21.15.66:0/3462376732 learned_addr learned my addr 172.21.15.66:0/3462376732 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff025d9b640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff028101fe0 0x7ff028103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02659c640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff028100e90 0x7ff028104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 -- 172.21.15.66:0/3462376732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff028101fe0 msgr2=0x7ff028103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff028101fe0 0x7ff028103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 -- 172.21.15.66:0/3462376732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff028100e90 msgr2=0x7ff028104dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff028100e90 0x7ff028104dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 -- 172.21.15.66:0/3462376732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff028104250 con 0x7ff0281029a0 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02659c640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff028100e90 0x7ff028104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:01.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff026d9d640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0281029a0 0x7ff028103990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff02000eaa0 tx=0x7ff02000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:01.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff0177fe640 1 -- 172.21.15.66:0/3462376732 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02000cdb0 con 0x7ff0281029a0 2024-04-21T09:08:01.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff0177fe640 1 -- 172.21.15.66:0/3462376732 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff020005020 con 0x7ff0281029a0 2024-04-21T09:08:01.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.995+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0281a2310 con 0x7ff0281029a0 2024-04-21T09:08:01.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.996+0000 7ff0177fe640 1 -- 172.21.15.66:0/3462376732 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200116d0 con 0x7ff0281029a0 2024-04-21T09:08:01.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.996+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281a27e0 con 0x7ff0281029a0 2024-04-21T09:08:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.996+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0281a2a70 con 0x7ff0281029a0 2024-04-21T09:08:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.998+0000 7ff0177fe640 1 -- 172.21.15.66:0/3462376732 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff0200051d0 con 0x7ff0281029a0 2024-04-21T09:08:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.998+0000 7ff0177fe640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff0100a3bb0 0x7ff0100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.998+0000 7ff02659c640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff0100a3bb0 0x7ff0100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.999+0000 7ff0177fe640 1 -- 172.21.15.66:0/3462376732 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7ff0200d0dc0 con 0x7ff0281029a0 2024-04-21T09:08:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:01.999+0000 7ff02659c640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff0100a3bb0 0x7ff0100a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff01800baa0 tx=0x7ff018005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:02.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.002+0000 7ff0177fe640 1 -- 172.21.15.66:0/3462376732 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff020016030 con 0x7ff0281029a0 2024-04-21T09:08:02.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.189+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff0281a2d60 con 0x7ff0281029a0 2024-04-21T09:08:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.189+0000 7ff0177fe640 1 -- 172.21.15.66:0/3462376732 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7ff02009d410 con 0x7ff0281029a0 2024-04-21T09:08:02.190 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff0100a3bb0 msgr2=0x7ff0100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff0100a3bb0 0x7ff0100a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff01800baa0 tx=0x7ff018005d90 comp rx=0 tx=0).stop 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0281029a0 msgr2=0x7ff028103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0281029a0 0x7ff028103990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff02000eaa0 tx=0x7ff02000ef60 comp rx=0 tx=0).stop 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 shutdown_connections 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7ff0100a3bb0 0x7ff0100a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0281029a0 0x7ff028103990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff028101fe0 0x7ff028103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 --2- 172.21.15.66:0/3462376732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff028100e90 0x7ff028104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 >> 172.21.15.66:0/3462376732 conn(0x7ff0280fc400 msgr2=0x7ff0280fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:02.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 shutdown_connections 2024-04-21T09:08:02.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.191+0000 7ff02cddb640 1 -- 172.21.15.66:0/3462376732 wait complete. 2024-04-21T09:08:02.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 -- 172.21.15.66:0/174096710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4100880 msgr2=0x7f8ac4100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/174096710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4100880 0x7f8ac4100ca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009900 tx=0x7f8abc02bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 -- 172.21.15.66:0/174096710 shutdown_connections 2024-04-21T09:08:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/174096710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac4101b80 0x7f8ac4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/174096710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac41011e0 0x7f8ac4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/174096710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4100880 0x7f8ac4100ca0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 -- 172.21.15.66:0/174096710 >> 172.21.15.66:0/174096710 conn(0x7f8ac40fc400 msgr2=0x7f8ac40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 -- 172.21.15.66:0/174096710 shutdown_connections 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.307+0000 7f8ac9a25640 1 -- 172.21.15.66:0/174096710 wait complete. 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 Processor -- start 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 -- start start 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac4100880 0x7f8ac419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac41011e0 0x7f8ac419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4101b80 0x7f8ac41a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac410fcd0 con 0x7f8ac4101b80 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac410fb50 con 0x7f8ac41011e0 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac9a25640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac410fe50 con 0x7f8ac4100880 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4101b80 0x7f8ac41a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4101b80 0x7f8ac41a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55302/0 (socket says 172.21.15.66:55302) 2024-04-21T09:08:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.308+0000 7f8ac37fe640 1 -- 172.21.15.66:0/2306764140 learned_addr learned my addr 172.21.15.66:0/2306764140 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac27fc640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac41011e0 0x7f8ac419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac2ffd640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac4100880 0x7f8ac419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac37fe640 1 -- 172.21.15.66:0/2306764140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac4100880 msgr2=0x7f8ac419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac37fe640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac4100880 0x7f8ac419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac37fe640 1 -- 172.21.15.66:0/2306764140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac41011e0 msgr2=0x7f8ac419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac37fe640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac41011e0 0x7f8ac419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac37fe640 1 -- 172.21.15.66:0/2306764140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac41a2050 con 0x7f8ac4101b80 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac27fc640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac41011e0 0x7f8ac419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac37fe640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4101b80 0x7f8ac41a19e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009fd0 tx=0x7f8ab400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac8a23640 1 -- 172.21.15.66:0/2306764140 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4017070 con 0x7f8ac4101b80 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac8a23640 1 -- 172.21.15.66:0/2306764140 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab4003e20 con 0x7f8ac4101b80 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac41a2340 con 0x7f8ac4101b80 2024-04-21T09:08:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac8a23640 1 -- 172.21.15.66:0/2306764140 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4010460 con 0x7f8ac4101b80 2024-04-21T09:08:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.309+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac41a2890 con 0x7f8ac4101b80 2024-04-21T09:08:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.310+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac410b8a0 con 0x7f8ac4101b80 2024-04-21T09:08:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.311+0000 7f8ac8a23640 1 -- 172.21.15.66:0/2306764140 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ab4003970 con 0x7f8ac4101b80 2024-04-21T09:08:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.312+0000 7f8ac8a23640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ab00a3c30 0x7f8ab00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.312+0000 7f8ac2ffd640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ab00a3c30 0x7f8ab00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.312+0000 7f8ac8a23640 1 -- 172.21.15.66:0/2306764140 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f8ab40d0580 con 0x7f8ac4101b80 2024-04-21T09:08:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.313+0000 7f8ac2ffd640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ab00a3c30 0x7f8ab00a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0096c0 tx=0x7f8abc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.317+0000 7f8ac8a23640 1 -- 172.21.15.66:0/2306764140 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ab409cbd0 con 0x7f8ac4101b80 2024-04-21T09:08:02.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.503+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8ac4062b50 con 0x7f8ac4101b80 2024-04-21T09:08:02.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.503+0000 7f8ac8a23640 1 -- 172.21.15.66:0/2306764140 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f8ab409c4d0 con 0x7f8ac4101b80 2024-04-21T09:08:02.504 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ab00a3c30 msgr2=0x7f8ab00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ab00a3c30 0x7f8ab00a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0096c0 tx=0x7f8abc000bc0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4101b80 msgr2=0x7f8ac41a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4101b80 0x7f8ac41a19e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009fd0 tx=0x7f8ab400b260 comp rx=0 tx=0).stop 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 shutdown_connections 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f8ab00a3c30 0x7f8ab00a60f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac4101b80 0x7f8ac41a19e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac41011e0 0x7f8ac419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 --2- 172.21.15.66:0/2306764140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac4100880 0x7f8ac419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 >> 172.21.15.66:0/2306764140 conn(0x7f8ac40fc400 msgr2=0x7f8ac40fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 shutdown_connections 2024-04-21T09:08:02.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.505+0000 7f8ac9a25640 1 -- 172.21.15.66:0/2306764140 wait complete. 2024-04-21T09:08:02.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:02.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 -- 172.21.15.66:0/651693529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 msgr2=0x7f25bc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 --2- 172.21.15.66:0/651693529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc101290 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009900 tx=0x7f25b002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 -- 172.21.15.66:0/651693529 shutdown_connections 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 --2- 172.21.15.66:0/651693529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25bc102980 0x7f25bc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 --2- 172.21.15.66:0/651693529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25bc101fc0 0x7f25bc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 --2- 172.21.15.66:0/651693529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc101290 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 -- 172.21.15.66:0/651693529 >> 172.21.15.66:0/651693529 conn(0x7f25bc0fc3e0 msgr2=0x7f25bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 -- 172.21.15.66:0/651693529 shutdown_connections 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 -- 172.21.15.66:0/651693529 wait complete. 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.619+0000 7f25c2da4640 1 Processor -- start 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 -- start start 2024-04-21T09:08:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25bc101fc0 0x7f25bc103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25bc102980 0x7f25bc103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc10c9a0 con 0x7f25bc100e70 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc10c820 con 0x7f25bc101fc0 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc10cb20 con 0x7f25bc102980 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc104df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55320/0 (socket says 172.21.15.66:55320) 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 -- 172.21.15.66:0/3523601243 learned_addr learned my addr 172.21.15.66:0/3523601243 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c1341640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25bc102980 0x7f25bc103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 -- 172.21.15.66:0/3523601243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25bc102980 msgr2=0x7f25bc103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25bc102980 0x7f25bc103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25bbfff640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25bc101fc0 0x7f25bc103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 -- 172.21.15.66:0/3523601243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25bc101fc0 msgr2=0x7f25bc103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25bc101fc0 0x7f25bc103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 -- 172.21.15.66:0/3523601243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc104200 con 0x7f25bc100e70 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25bbfff640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25bc101fc0 0x7f25bc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c0b40640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc104df0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25b0006010 tx=0x7f25b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25b9ffb640 1 -- 172.21.15.66:0/3523601243 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0043020 con 0x7f25bc100e70 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc1a22a0 con 0x7f25bc100e70 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25b9ffb640 1 -- 172.21.15.66:0/3523601243 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b0004aa0 con 0x7f25bc100e70 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.620+0000 7f25b9ffb640 1 -- 172.21.15.66:0/3523601243 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0003c70 con 0x7f25bc100e70 2024-04-21T09:08:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.621+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc1a2840 con 0x7f25bc100e70 2024-04-21T09:08:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.622+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25bc1a2b60 con 0x7f25bc100e70 2024-04-21T09:08:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.623+0000 7f25b9ffb640 1 -- 172.21.15.66:0/3523601243 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f25b0046020 con 0x7f25bc100e70 2024-04-21T09:08:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.623+0000 7f25b9ffb640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25ac0a3bb0 0x7f25ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.623+0000 7f25bbfff640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25ac0a3bb0 0x7f25ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.623+0000 7f25b9ffb640 1 -- 172.21.15.66:0/3523601243 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f25b00f2c10 con 0x7f25bc100e70 2024-04-21T09:08:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.624+0000 7f25bbfff640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25ac0a3bb0 0x7f25ac0a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f25bc104b60 tx=0x7f25a8005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.628+0000 7f25b9ffb640 1 -- 172.21.15.66:0/3523601243 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25b004d020 con 0x7f25bc100e70 2024-04-21T09:08:02.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1970941365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3462376732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:02 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2306764140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.855+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25bc1a2d30 con 0x7f25bc100e70 2024-04-21T09:08:02.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.856+0000 7f25b9ffb640 1 -- 172.21.15.66:0/3523601243 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11867 (secure 0 0 0) 0x7f25b00bf260 con 0x7f25bc100e70 2024-04-21T09:08:02.856 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:02.857 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:06:13.097844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24476,24490]},"id":1}]} 2024-04-21T09:08:02.857 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-04-21T09:08:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25ac0a3bb0 msgr2=0x7f25ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25ac0a3bb0 0x7f25ac0a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f25bc104b60 tx=0x7f25a8005e70 comp rx=0 tx=0).stop 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 msgr2=0x7f25bc104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc104df0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25b0006010 tx=0x7f25b00042f0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 shutdown_connections 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25ac0a3bb0 0x7f25ac0a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25bc102980 0x7f25bc103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25bc101fc0 0x7f25bc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 --2- 172.21.15.66:0/3523601243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25bc100e70 0x7f25bc104df0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 >> 172.21.15.66:0/3523601243 conn(0x7f25bc0fc3e0 msgr2=0x7f25bc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 shutdown_connections 2024-04-21T09:08:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.858+0000 7f25c2da4640 1 -- 172.21.15.66:0/3523601243 wait complete. 2024-04-21T09:08:02.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1970941365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3462376732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:02 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2306764140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.868 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1970941365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3462376732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:02 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2306764140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:02.870 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 -- 172.21.15.66:0/3765203218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808101fc0 msgr2=0x7f0808102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 --2- 172.21.15.66:0/3765203218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808101fc0 0x7f0808102440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009860 tx=0x7f07f802bb70 comp rx=0 tx=0).stop 2024-04-21T09:08:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 -- 172.21.15.66:0/3765203218 shutdown_connections 2024-04-21T09:08:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 --2- 172.21.15.66:0/3765203218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0808102980 0x7f08081092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 --2- 172.21.15.66:0/3765203218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808101fc0 0x7f0808102440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 --2- 172.21.15.66:0/3765203218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 0x7f0808101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 -- 172.21.15.66:0/3765203218 >> 172.21.15.66:0/3765203218 conn(0x7f08080fc3e0 msgr2=0x7f08080fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:02.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.972+0000 7f080f643640 1 -- 172.21.15.66:0/3765203218 shutdown_connections 2024-04-21T09:08:02.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 -- 172.21.15.66:0/3765203218 wait complete. 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 Processor -- start 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 -- start start 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 0x7f0808104da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0808101fc0 0x7f0808103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808102980 0x7f0808103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f080810c980 con 0x7f0808100e70 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f080810c800 con 0x7f0808102980 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080f643640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f080810cb00 con 0x7f0808101fc0 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080d3df640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 0x7f0808104da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080d3df640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 0x7f0808104da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55334/0 (socket says 172.21.15.66:55334) 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.973+0000 7f080d3df640 1 -- 172.21.15.66:0/1287206141 learned_addr learned my addr 172.21.15.66:0/1287206141 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:02.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080cbde640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0808101fc0 0x7f0808103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080dbe0640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808102980 0x7f0808103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080d3df640 1 -- 172.21.15.66:0/1287206141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0808101fc0 msgr2=0x7f0808103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080d3df640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0808101fc0 0x7f0808103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080d3df640 1 -- 172.21.15.66:0/1287206141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808102980 msgr2=0x7f0808103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080d3df640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808102980 0x7f0808103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080d3df640 1 -- 172.21.15.66:0/1287206141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0808104230 con 0x7f0808100e70 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080d3df640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 0x7f0808104da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f080400e9c0 tx=0x7f080400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f07fe7fc640 1 -- 172.21.15.66:0/1287206141 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080400ccc0 con 0x7f0808100e70 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08081a2330 con 0x7f0808100e70 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f07fe7fc640 1 -- 172.21.15.66:0/1287206141 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08040054d0 con 0x7f0808100e70 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f07fe7fc640 1 -- 172.21.15.66:0/1287206141 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804010430 con 0x7f0808100e70 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.974+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08081a2850 con 0x7f0808100e70 2024-04-21T09:08:02.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.975+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08081a2ae0 con 0x7f0808100e70 2024-04-21T09:08:02.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.977+0000 7f07fe7fc640 1 -- 172.21.15.66:0/1287206141 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0804004f80 con 0x7f0808100e70 2024-04-21T09:08:02.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.978+0000 7f07fe7fc640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07f40a3bb0 0x7f07f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:02.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.978+0000 7f080cbde640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07f40a3bb0 0x7f07f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:02.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.978+0000 7f07fe7fc640 1 -- 172.21.15.66:0/1287206141 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f08040d0bb0 con 0x7f0808100e70 2024-04-21T09:08:02.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.978+0000 7f080cbde640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07f40a3bb0 0x7f07f40a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009830 tx=0x7f07f80349b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:02.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:02.983+0000 7f07fe7fc640 1 -- 172.21.15.66:0/1287206141 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f080409d1a0 con 0x7f0808100e70 2024-04-21T09:08:03.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.171+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08081a31f0 con 0x7f0808100e70 2024-04-21T09:08:03.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.172+0000 7f07fe7fc640 1 -- 172.21.15.66:0/1287206141 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24710 (secure 0 0 0) 0x7f080409caa0 con 0x7f0808100e70 2024-04-21T09:08:03.173 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:03.173 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:07:40.521019+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07f40a3bb0 msgr2=0x7f07f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07f40a3bb0 0x7f07f40a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009830 tx=0x7f07f80349b0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 msgr2=0x7f0808104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 0x7f0808104da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f080400e9c0 tx=0x7f080400ee80 comp rx=0 tx=0).stop 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 shutdown_connections 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f07f40a3bb0 0x7f07f40a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0808102980 0x7f0808103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0808101fc0 0x7f0808103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 --2- 172.21.15.66:0/1287206141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0808100e70 0x7f0808104da0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 >> 172.21.15.66:0/1287206141 conn(0x7f08080fc3e0 msgr2=0x7f08080690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 shutdown_connections 2024-04-21T09:08:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.175+0000 7f080f643640 1 -- 172.21.15.66:0/1287206141 wait complete. 2024-04-21T09:08:03.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T09:05:46.330734+0000', 'modified': '2024-04-21T09:06:13.097844+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24490, 'mds_2': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/2851815427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 2851815427}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 2851815427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/1405629184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 1405629184}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 1405629184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/1780100152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1780100152}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1780100152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.83:6837/1568837633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1568837633}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1568837633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.186:6839/599772329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 599772329}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 599772329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/4140026829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 4140026829}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 4140026829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24512, 24476, 24490]} 2024-04-21T09:08:03.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-21T09:08:03.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T09:08:03.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3732999615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc102a10 msgr2=0x7fe4dc104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3732999615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc102a10 0x7fe4dc104e00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009900 tx=0x7fe4d402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3732999615 shutdown_connections 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3732999615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc107c70 0x7fe4dc10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3732999615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc105340 0x7fe4dc107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3732999615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc102a10 0x7fe4dc104e00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3732999615 >> 172.21.15.66:0/3732999615 conn(0x7fe4dc0fc400 msgr2=0x7fe4dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3732999615 shutdown_connections 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3732999615 wait complete. 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 Processor -- start 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- start start 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a10 0x7fe4dc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc105340 0x7fe4dc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc107c70 0x7fe4dc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10dac0 con 0x7fe4dc105340 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10d940 con 0x7fe4dc102a10 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.290+0000 7fe4e32a0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10dc40 con 0x7fe4dc107c70 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc105340 0x7fe4dc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc105340 0x7fe4dc198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55362/0 (socket says 172.21.15.66:55362) 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 -- 172.21.15.66:0/3819828232 learned_addr learned my addr 172.21.15.66:0/3819828232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a10 0x7fe4dc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e183d640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc107c70 0x7fe4dc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 -- 172.21.15.66:0/3819828232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc107c70 msgr2=0x7fe4dc19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc107c70 0x7fe4dc19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 -- 172.21.15.66:0/3819828232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a10 msgr2=0x7fe4dc1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a10 0x7fe4dc1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 -- 172.21.15.66:0/3819828232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc19fde0 con 0x7fe4dc105340 2024-04-21T09:08:03.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e083b640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc105340 0x7fe4dc198700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000ccb0 tx=0x7fe4d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4da7fc640 1 -- 172.21.15.66:0/3819828232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000ce60 con 0x7fe4dc105340 2024-04-21T09:08:03.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4da7fc640 1 -- 172.21.15.66:0/3819828232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d000fb00 con 0x7fe4dc105340 2024-04-21T09:08:03.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4da7fc640 1 -- 172.21.15.66:0/3819828232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0018bb0 con 0x7fe4dc105340 2024-04-21T09:08:03.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc1a00d0 con 0x7fe4dc105340 2024-04-21T09:08:03.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.291+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a0620 con 0x7fe4dc105340 2024-04-21T09:08:03.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4da7fc640 1 -- 172.21.15.66:0/3819828232 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe4d000f650 con 0x7fe4dc105340 2024-04-21T09:08:03.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 Processor -- start 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 -- start start 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd706670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706c40 0x5621bd7070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd7076e0 0x5621bd70e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5621bd666b10 con 0x5621bd7076e0 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5621bd604240 con 0x5621bd706250 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5621bd662ad0 con 0x5621bd706c40 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd7076e0 0x5621bd70e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.294+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd7076e0 0x5621bd70e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55364/0 (socket says 172.21.15.66:55364) 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e083b640 1 -- 172.21.15.66:0/3165398986 learned_addr learned my addr 172.21.15.66:0/3165398986 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e183d640 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd706670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706c40 0x5621bd7070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e183d640 1 -- 172.21.15.66:0/3165398986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706c40 msgr2=0x5621bd7070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e183d640 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706c40 0x5621bd7070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e183d640 1 -- 172.21.15.66:0/3165398986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd7076e0 msgr2=0x5621bd70e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e183d640 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd7076e0 0x5621bd70e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e183d640 1 -- 172.21.15.66:0/3165398986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621bd3792d0 con 0x5621bd706250 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706c40 0x5621bd7070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4da7fc640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c40a3bb0 0x7fe4c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c40a3bb0 0x7fe4c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4e183d640 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd706670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ca90 tx=0x7fe4cc00cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f3c9459678727d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4da7fc640 1 -- 172.21.15.66:0/3819828232 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fe4d00d0cc0 con 0x7fe4dc105340 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4c37fe640 1 -- 172.21.15.66:0/3165398986 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004020 con 0x5621bd706250 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4c37fe640 1 -- 172.21.15.66:0/3165398986 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc0041c0 con 0x5621bd706250 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.295+0000 7fe4c37fe640 1 -- 172.21.15.66:0/3165398986 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc010690 con 0x5621bd706250 2024-04-21T09:08:03.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3165398986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 msgr2=0x5621bd706670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c40a3bb0 0x7fe4c40a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd706670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ca90 tx=0x7fe4cc00cf50 comp rx=0 tx=0).stop 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3165398986 shutdown_connections 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd7076e0 0x5621bd70e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706c40 0x5621bd7070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3165398986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd706670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3165398986 >> 172.21.15.66:0/3165398986 conn(0x5621bd702350 msgr2=0x5621bd7008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3165398986 shutdown_connections 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3165398986 wait complete. 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.296+0000 7fe4e5f68740 1 -- start start 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 1 Processor -- start 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706250 0x5621bd7101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd7076e0 0x5621bd717730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5621bd70e8c0 con 0x5621bd706c40 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5621bd5ef190 con 0x5621bd7076e0 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5621bd70ea40 con 0x5621bd706250 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e5f68740 10 client.0 _ll_register_callbacks cb 0x5621bd5ea8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55378/0 (socket says 172.21.15.66:55378) 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 -- 172.21.15.66:0/3178705310 learned_addr learned my addr 172.21.15.66:0/3178705310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e183d640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd7076e0 0x5621bd717730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706250 0x5621bd7101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 -- 172.21.15.66:0/3178705310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706250 msgr2=0x5621bd7101b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706250 0x5621bd7101b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 -- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd7076e0 msgr2=0x5621bd717730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd7076e0 0x5621bd717730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e083b640 1 -- 172.21.15.66:0/3178705310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621bd4436b0 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.297+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706250 0x5621bd7101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e083b640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7106f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe4d009c910 tx=0x7fe4d0018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f477e3fa07b38d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/3178705310 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00d2070 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4cc0041c0 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/3178705310 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fe4d0018d50 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/3178705310 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe4d009c750 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 10 client.15390 fetch_fsmap learned FSMap version 24 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5621bd71abc0 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/3178705310 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc0041c0 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/3178705310 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe4d00bd0b0 con 0x5621bd706c40 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 10 client.15390 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 10 client.15390 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T09:08:03.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 20 client.15390 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd71e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 4 client.15390 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.298+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621bd674530 con 0x5621bd71d130 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.299+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd71e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.299+0000 7fe4e103c640 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd71e5a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93031c0f96446d52 server_cookie=67be06f53a7f147a in_seq=0 out_seq=0 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.299+0000 7fe4c0ff9640 10 client.15390 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/3178705310 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621bd674530 con 0x5621bd71d130 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4c0ff9640 10 client.15390 handle_command_reply: tid=0 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 client.15390 shutdown 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 msgr2=0x5621bd7106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7106f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe4d009c910 tx=0x7fe4d0018400 comp rx=0 tx=0).stop 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 shutdown_connections 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd71e5a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd7076e0 0x5621bd717730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7106f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4c0ff9640 0 client.15390 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/3178705310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd706250 0x5621bd7101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 >> 172.21.15.66:0/3178705310 conn(0x5621bd702350 msgr2=0x5621bd6fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 shutdown_connections 2024-04-21T09:08:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 1 -- 172.21.15.66:0/3178705310 wait complete. 2024-04-21T09:08:03.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.300+0000 7fe4e5f68740 20 client.15390 trim_cache size 0 max 16384 2024-04-21T09:08:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.318+0000 7fe4e5f68740 1 Processor -- start 2024-04-21T09:08:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.318+0000 7fe4e5f68740 1 -- start start 2024-04-21T09:08:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.318+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd70e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.318+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 0x5621bd70e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5621bd71c8e0 con 0x5621bd706c40 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5621bd71cfb0 con 0x5621bd706250 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5621bd71d680 con 0x5621bd7076e0 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55388/0 (socket says 172.21.15.66:55388) 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 -- 172.21.15.66:0/793623528 learned_addr learned my addr 172.21.15.66:0/793623528 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e103c640 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd70e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 -- 172.21.15.66:0/793623528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 msgr2=0x5621bd70e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 0x5621bd70e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 -- 172.21.15.66:0/793623528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 msgr2=0x5621bd70e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd70e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 -- 172.21.15.66:0/793623528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621bd6f7e70 con 0x5621bd706c40 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e103c640 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd70e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4e083b640 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7176f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0099590 tx=0x7fe4d0018470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc862e57c89beab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4c2ffd640 1 -- 172.21.15.66:0/793623528 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d009c570 con 0x5621bd706c40 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4c2ffd640 1 -- 172.21.15.66:0/793623528 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d000f940 con 0x5621bd706c40 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.319+0000 7fe4c2ffd640 1 -- 172.21.15.66:0/793623528 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0061be0 con 0x5621bd706c40 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 -- 172.21.15.66:0/793623528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 msgr2=0x5621bd7176f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7176f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0099590 tx=0x7fe4d0018470 comp rx=0 tx=0).stop 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 -- 172.21.15.66:0/793623528 shutdown_connections 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 0x5621bd70e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706c40 0x5621bd7176f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/793623528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706250 0x5621bd70e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 -- 172.21.15.66:0/793623528 >> 172.21.15.66:0/793623528 conn(0x5621bd702350 msgr2=0x5621bd70ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 -- 172.21.15.66:0/793623528 shutdown_connections 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 -- 172.21.15.66:0/793623528 wait complete. 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 -- start start 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.320+0000 7fe4e5f68740 1 Processor -- start 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706250 0x5621bd713880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706c40 0x5621bd713dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 0x5621bd7168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5621bd71e420 con 0x5621bd706250 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5621bd71dd50 con 0x5621bd706c40 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5621bd71eaf0 con 0x5621bd7076e0 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 10 client.0 _ll_register_callbacks cb 0x5621bd5ea8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706250 0x5621bd713880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706250 0x5621bd713880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55394/0 (socket says 172.21.15.66:55394) 2024-04-21T09:08:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 -- 172.21.15.66:0/139131892 learned_addr learned my addr 172.21.15.66:0/139131892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e183d640 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 0x5621bd7168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 -- 172.21.15.66:0/139131892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 msgr2=0x5621bd7168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 0x5621bd7168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e083b640 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706c40 0x5621bd713dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 -- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706c40 msgr2=0x5621bd713dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706c40 0x5621bd713dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 -- 172.21.15.66:0/139131892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621bd716ea0 con 0x5621bd706250 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e103c640 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706250 0x5621bd713880 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005ec0 tx=0x7fe4d403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf8aa639f14d977 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/139131892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4056030 con 0x5621bd706250 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/139131892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fe4d4052b50 con 0x5621bd706250 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.321+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621bd66e3a0 con 0x5621bd706250 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/139131892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe4d4063050 con 0x5621bd706250 2024-04-21T09:08:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5621bd7835e0 con 0x5621bd706250 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/139131892 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5621bd66e3a0 con 0x5621bd706250 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/139131892 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe4d4004850 con 0x5621bd706250 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 10 client.15396 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 20 client.15396 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd786fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 4 client.15396 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "status"}) -- 0x5621bd5ea960 con 0x5621bd71d130 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.322+0000 7fe4e083b640 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd786fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e083b640 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd786fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e5a1a38e1772043 server_cookie=761214c761ddd986 in_seq=0 out_seq=0 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4c0ff9640 10 client.15396 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4c0ff9640 1 -- 172.21.15.66:0/139131892 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5621bd5ea960 con 0x5621bd71d130 2024-04-21T09:08:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4c0ff9640 10 client.15396 handle_command_reply: tid=0 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 client.15396 shutdown 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706250 msgr2=0x5621bd713880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706250 0x5621bd713880 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005ec0 tx=0x7fe4d403f000 comp rx=0 tx=0).stop 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 shutdown_connections 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5621bd71d130 0x5621bd786fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5621bd7076e0 0x5621bd7168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4c0ff9640 0 client.15396 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621bd706c40 0x5621bd713dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 --2- 172.21.15.66:0/139131892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5621bd706250 0x5621bd713880 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 >> 172.21.15.66:0/139131892 conn(0x5621bd702350 msgr2=0x5621bd70e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 shutdown_connections 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 1 -- 172.21.15.66:0/139131892 wait complete. 2024-04-21T09:08:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.323+0000 7fe4e5f68740 20 client.15396 trim_cache size 0 max 16384 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 2, 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24476, 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 111.23373964699999, 2024-04-21T09:08:03.331 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 24, 2024-04-21T09:08:03.332 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 95, 2024-04-21T09:08:03.332 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 91, 2024-04-21T09:08:03.332 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 153.73207531400001 2024-04-21T09:08:03.332 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:03.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c40a3bb0 msgr2=0x7fe4c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c40a3bb0 0x7fe4c40a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4005d50 comp rx=0 tx=0).stop 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc105340 msgr2=0x7fe4dc198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc105340 0x7fe4dc198700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000ccb0 tx=0x7fe4d000a620 comp rx=0 tx=0).stop 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 shutdown_connections 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fe4c40a3bb0 0x7fe4c40a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc107c70 0x7fe4dc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4dc105340 0x7fe4dc198700 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 --2- 172.21.15.66:0/3819828232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a10 0x7fe4dc1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 >> 172.21.15.66:0/3819828232 conn(0x7fe4dc0fc400 msgr2=0x7fe4dc1032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 shutdown_connections 2024-04-21T09:08:03.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.324+0000 7fe4e32a0640 1 -- 172.21.15.66:0/3819828232 wait complete. 2024-04-21T09:08:03.335 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T09:08:03.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 -- 172.21.15.66:0/3365867363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 msgr2=0x7f4898101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 --2- 172.21.15.66:0/3365867363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898101280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f488c009900 tx=0x7f488c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:03.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 -- 172.21.15.66:0/3365867363 shutdown_connections 2024-04-21T09:08:03.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 --2- 172.21.15.66:0/3365867363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4898102a00 0x7f4898109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 --2- 172.21.15.66:0/3365867363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4898102040 0x7f48981024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 --2- 172.21.15.66:0/3365867363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898101280 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 -- 172.21.15.66:0/3365867363 >> 172.21.15.66:0/3365867363 conn(0x7f48980fc440 msgr2=0x7f48980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 -- 172.21.15.66:0/3365867363 shutdown_connections 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.437+0000 7f489e7a4640 1 -- 172.21.15.66:0/3365867363 wait complete. 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 Processor -- start 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 -- start start 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4898102040 0x7f4898198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4898102a00 0x7f489819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f489810c9d0 con 0x7f4898100e60 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f489810c850 con 0x7f4898102040 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489e7a4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f489810cb50 con 0x7f4898102a00 2024-04-21T09:08:03.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f4897fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f4897fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55412/0 (socket says 172.21.15.66:55412) 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f4897fff640 1 -- 172.21.15.66:0/2091011963 learned_addr learned my addr 172.21.15.66:0/2091011963 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f489cd41640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4898102a00 0x7f489819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.438+0000 7f48977fe640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4898102040 0x7f4898198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f4897fff640 1 -- 172.21.15.66:0/2091011963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4898102a00 msgr2=0x7f489819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f4897fff640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4898102a00 0x7f489819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f4897fff640 1 -- 172.21.15.66:0/2091011963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4898102040 msgr2=0x7f4898198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f4897fff640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4898102040 0x7f4898198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f4897fff640 1 -- 172.21.15.66:0/2091011963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489819fe70 con 0x7f4898100e60 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f48977fe640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4898102040 0x7f4898198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f4897fff640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898198250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f488c000c00 tx=0x7f488c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f48957fa640 1 -- 172.21.15.66:0/2091011963 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c043020 con 0x7f4898100e60 2024-04-21T09:08:03.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f48957fa640 1 -- 172.21.15.66:0/2091011963 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c004aa0 con 0x7f4898100e60 2024-04-21T09:08:03.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48981a0100 con 0x7f4898100e60 2024-04-21T09:08:03.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f48957fa640 1 -- 172.21.15.66:0/2091011963 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c003d00 con 0x7f4898100e60 2024-04-21T09:08:03.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.439+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48981a0650 con 0x7f4898100e60 2024-04-21T09:08:03.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.441+0000 7f48957fa640 1 -- 172.21.15.66:0/2091011963 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f488c046020 con 0x7f4898100e60 2024-04-21T09:08:03.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.441+0000 7f48a1390740 1 Processor -- start 2024-04-21T09:08:03.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.441+0000 7f48a1390740 1 -- start start 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48a1390740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124cf3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48a1390740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124cf3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48a1390740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124cfaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48a1390740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562124c57b10 con 0x562124cf42d0 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48a1390740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562124bf5240 con 0x562124cf3830 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48a1390740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562124c53ad0 con 0x562124cf2e40 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48957fa640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f48800a3bb0 0x7f48800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124cfaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124cfaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55420/0 (socket says 172.21.15.66:55420) 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f4897fff640 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124cf3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 -- 172.21.15.66:0/518136253 learned_addr learned my addr 172.21.15.66:0/518136253 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f489cd41640 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124cf3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f489cd41640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f48800a3bb0 0x7f48800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48957fa640 1 -- 172.21.15.66:0/2091011963 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f488c0f2d30 con 0x7f4898100e60 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 -- 172.21.15.66:0/518136253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 msgr2=0x562124cf3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124cf3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 -- 172.21.15.66:0/518136253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 msgr2=0x562124cf3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124cf3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 -- 172.21.15.66:0/518136253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56212496a2d0 con 0x562124cf42d0 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f48977fe640 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124cfaeb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f488800cc50 tx=0x7f488800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5696f57f474494c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f489cd41640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f48800a3bb0 0x7f48800a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x562124cee200 tx=0x7f4890009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.442+0000 7f487e7fc640 1 -- 172.21.15.66:0/518136253 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888014070 con 0x562124cf42d0 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f487e7fc640 1 -- 172.21.15.66:0/518136253 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4888019b10 con 0x562124cf42d0 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f487e7fc640 1 -- 172.21.15.66:0/518136253 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888010b70 con 0x562124cf42d0 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 -- 172.21.15.66:0/518136253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 msgr2=0x562124cfaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124cfaeb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f488800cc50 tx=0x7f488800e720 comp rx=0 tx=0).stop 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 -- 172.21.15.66:0/518136253 shutdown_connections 2024-04-21T09:08:03.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124cfaeb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124cf3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 --2- 172.21.15.66:0/518136253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124cf3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 -- 172.21.15.66:0/518136253 >> 172.21.15.66:0/518136253 conn(0x562124ceef40 msgr2=0x562124ced490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 -- 172.21.15.66:0/518136253 shutdown_connections 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.443+0000 7f48a1390740 1 -- 172.21.15.66:0/518136253 wait complete. 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 -- start start 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 Processor -- start 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf2e40 0x562124cfce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124cfd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf42d0 0x562124d04400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562124cfb4b0 con 0x562124cf2e40 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562124be0190 con 0x562124cf42d0 2024-04-21T09:08:03.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562124cfb630 con 0x562124cf3830 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48a1390740 10 client.0 _ll_register_callbacks cb 0x562124bdb8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf2e40 0x562124cfce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf2e40 0x562124cfce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55426/0 (socket says 172.21.15.66:55426) 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 -- 172.21.15.66:0/3243929239 learned_addr learned my addr 172.21.15.66:0/3243929239 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f489cd41640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf42d0 0x562124d04400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 -- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 msgr2=0x562124cfd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f48977fe640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124cfd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124cfd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 -- 172.21.15.66:0/3243929239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf42d0 msgr2=0x562124d04400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf42d0 0x562124d04400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f4897fff640 1 -- 172.21.15.66:0/3243929239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562124a346b0 con 0x562124cf2e40 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.444+0000 7f489cd41640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf42d0 0x562124d04400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f4897fff640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf2e40 0x562124cfce80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f488c0f0700 tx=0x7f488c090710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38236ba05a90d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48777fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48777fe640 1 -- 172.21.15.66:0/3243929239 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0bea50 con 0x562124cf2e40 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48777fe640 1 -- 172.21.15.66:0/3243929239 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f488c0bf0e0 con 0x562124cf2e40 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4888019b10 con 0x562124cf2e40 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48777fe640 1 -- 172.21.15.66:0/3243929239 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f488c0be5e0 con 0x562124cf2e40 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 10 client.15408 fetch_fsmap learned FSMap version 24 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562124d07770 con 0x562124cf2e40 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48777fe640 1 -- 172.21.15.66:0/3243929239 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4888019b10 con 0x562124cf2e40 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48777fe640 1 -- 172.21.15.66:0/3243929239 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f488c0c7460 con 0x562124cf2e40 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 10 client.15408 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 10 client.15408 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 20 client.15408 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d09e60 0x562124d0b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 4 client.15408 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.445+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562124c65530 con 0x562124d09e60 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.446+0000 7f48977fe640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d09e60 0x562124d0b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.446+0000 7f48977fe640 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d09e60 0x562124d0b2d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63e915a4a7751dc7 server_cookie=5bfd0b4f93f09a15 in_seq=0 out_seq=0 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.446+0000 7f48777fe640 10 client.15408 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48777fe640 1 -- 172.21.15.66:0/3243929239 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562124c65530 con 0x562124d09e60 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48777fe640 10 client.15408 handle_command_reply: tid=0 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 client.15408 shutdown 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf2e40 msgr2=0x562124cfce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf2e40 0x562124cfce80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f488c0f0700 tx=0x7f488c090710 comp rx=0 tx=0).stop 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 shutdown_connections 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf42d0 0x562124d04400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124cfd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d09e60 0x562124d0b2d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 --2- 172.21.15.66:0/3243929239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf2e40 0x562124cfce80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 >> 172.21.15.66:0/3243929239 conn(0x562124ceef40 msgr2=0x562124ceca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48777fe640 0 client.15408 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 shutdown_connections 2024-04-21T09:08:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 1 -- 172.21.15.66:0/3243929239 wait complete. 2024-04-21T09:08:03.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.447+0000 7f48a1390740 20 client.15408 trim_cache size 0 max 16384 2024-04-21T09:08:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 Processor -- start 2024-04-21T09:08:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 -- start start 2024-04-21T09:08:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf2e40 0x562124d04e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124ced400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124ced940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562124d09610 con 0x562124cf42d0 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562124d09ce0 con 0x562124cf2e40 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.465+0000 7f48a1390740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562124d0a3b0 con 0x562124cf3830 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124ced940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124ced940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55436/0 (socket says 172.21.15.66:55436) 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 -- 172.21.15.66:0/1552198590 learned_addr learned my addr 172.21.15.66:0/1552198590 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f4897fff640 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf2e40 0x562124d04e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f48977fe640 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124ced400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 -- 172.21.15.66:0/1552198590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 msgr2=0x562124ced400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124ced400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 -- 172.21.15.66:0/1552198590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf2e40 msgr2=0x562124d04e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf2e40 0x562124d04e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 -- 172.21.15.66:0/1552198590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562124cfaa70 con 0x562124cf42d0 2024-04-21T09:08:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f489cd41640 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124ced940 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4890012a20 tx=0x7f4890009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ff04e7913b3f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f487dffb640 1 -- 172.21.15.66:0/1552198590 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890012e20 con 0x562124cf42d0 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f487dffb640 1 -- 172.21.15.66:0/1552198590 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4890025070 con 0x562124cf42d0 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.466+0000 7f487dffb640 1 -- 172.21.15.66:0/1552198590 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890005350 con 0x562124cf42d0 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 -- 172.21.15.66:0/1552198590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 msgr2=0x562124ced940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124ced940 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4890012a20 tx=0x7f4890009a00 comp rx=0 tx=0).stop 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 -- 172.21.15.66:0/1552198590 shutdown_connections 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124ced940 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf3830 0x562124ced400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 --2- 172.21.15.66:0/1552198590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf2e40 0x562124d04e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 -- 172.21.15.66:0/1552198590 >> 172.21.15.66:0/1552198590 conn(0x562124ceef40 msgr2=0x562124d08ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 -- 172.21.15.66:0/1552198590 shutdown_connections 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 -- 172.21.15.66:0/1552198590 wait complete. 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 -- start start 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.467+0000 7f48a1390740 1 Processor -- start 2024-04-21T09:08:03.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124d6de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124d70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124d729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562124d0b150 con 0x562124cf42d0 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562124d0aa80 con 0x562124cf3830 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562124d0b820 con 0x562124cf2e40 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 10 client.0 _ll_register_callbacks cb 0x7f488c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124d729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124d729e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55446/0 (socket says 172.21.15.66:55446) 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 -- 172.21.15.66:0/373650017 learned_addr learned my addr 172.21.15.66:0/373650017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48977fe640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124d70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f4897fff640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124d6de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 -- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 msgr2=0x562124d6de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124d6de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 -- 172.21.15.66:0/373650017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 msgr2=0x562124d70350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124d70350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 -- 172.21.15.66:0/373650017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562124d73210 con 0x562124cf42d0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f4897fff640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124d6de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f489cd41640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124d729e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4890012bb0 tx=0x7f4890009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8003b5d4e98f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48777fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48777fe640 1 -- 172.21.15.66:0/373650017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890004a40 con 0x562124cf42d0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48777fe640 1 -- 172.21.15.66:0/373650017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f4890004be0 con 0x562124cf42d0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.468+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4890025070 con 0x562124cf42d0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48777fe640 1 -- 172.21.15.66:0/373650017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4890004e20 con 0x562124cf42d0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 10 client.15414 fetch_fsmap learned FSMap version 24 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562124d74510 con 0x562124cf42d0 2024-04-21T09:08:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48777fe640 1 -- 172.21.15.66:0/373650017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4890025070 con 0x562124cf42d0 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48777fe640 1 -- 172.21.15.66:0/373650017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4890030070 con 0x562124cf42d0 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 10 client.15414 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 20 client.15414 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d78eb0 0x562124d759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 4 client.15414 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "status"}) -- 0x562124c5f3a0 con 0x562124d78eb0 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.469+0000 7f4897fff640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d78eb0 0x562124d759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f4897fff640 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d78eb0 0x562124d759c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99e2bcc0022338e4 server_cookie=5a3978fb7d870e58 in_seq=0 out_seq=0 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48777fe640 10 client.15414 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48777fe640 1 -- 172.21.15.66:0/373650017 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562124c5f3a0 con 0x562124d78eb0 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48777fe640 10 client.15414 handle_command_reply: tid=0 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 client.15414 shutdown 2024-04-21T09:08:03.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 msgr2=0x562124d729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124d729e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4890012bb0 tx=0x7f4890009e20 comp rx=0 tx=0).stop 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 shutdown_connections 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x562124d78eb0 0x562124d759c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562124cf42d0 0x562124d729e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562124cf3830 0x562124d70350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48777fe640 0 client.15414 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 --2- 172.21.15.66:0/373650017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562124cf2e40 0x562124d6de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 >> 172.21.15.66:0/373650017 conn(0x562124ceef40 msgr2=0x562124ced400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 shutdown_connections 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 1 -- 172.21.15.66:0/373650017 wait complete. 2024-04-21T09:08:03.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.470+0000 7f48a1390740 20 client.15414 trim_cache size 0 max 16384 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 1, 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24490, 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 112.389828896, 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 24, 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 95, 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 91, 2024-04-21T09:08:03.478 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 151.165140581 2024-04-21T09:08:03.479 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:03.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f48800a3bb0 msgr2=0x7f48800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f48800a3bb0 0x7f48800a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x562124cee200 tx=0x7f4890009600 comp rx=0 tx=0).stop 2024-04-21T09:08:03.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 msgr2=0x7f4898198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898198250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f488c000c00 tx=0x7f488c004270 comp rx=0 tx=0).stop 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 shutdown_connections 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f48800a3bb0 0x7f48800a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4898102a00 0x7f489819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4898102040 0x7f4898198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 --2- 172.21.15.66:0/2091011963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4898100e60 0x7f4898198250 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 >> 172.21.15.66:0/2091011963 conn(0x7f48980fc440 msgr2=0x7f489806a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 shutdown_connections 2024-04-21T09:08:03.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.471+0000 7f489e7a4640 1 -- 172.21.15.66:0/2091011963 wait complete. 2024-04-21T09:08:03.482 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 -- 172.21.15.66:0/513143418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 msgr2=0x7f596810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 --2- 172.21.15.66:0/513143418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596810a0c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5960009ca0 tx=0x7f596002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 -- 172.21.15.66:0/513143418 shutdown_connections 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 --2- 172.21.15.66:0/513143418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596810a0c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 --2- 172.21.15.66:0/513143418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59681053a0 0x7f5968107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 --2- 172.21.15.66:0/513143418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5968102a70 0x7f5968104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 -- 172.21.15.66:0/513143418 >> 172.21.15.66:0/513143418 conn(0x7f59680fc400 msgr2=0x7f59680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 -- 172.21.15.66:0/513143418 shutdown_connections 2024-04-21T09:08:03.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.585+0000 7f596dfa4640 1 -- 172.21.15.66:0/513143418 wait complete. 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 Processor -- start 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 -- start start 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5968102a70 0x7f59681981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59681053a0 0x7f5968198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f596810dae0 con 0x7f5968107cd0 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f596810d960 con 0x7f59681053a0 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f596dfa4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f596810dc60 con 0x7f5968102a70 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f5967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f5967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55470/0 (socket says 172.21.15.66:55470) 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.586+0000 7f5967fff640 1 -- 172.21.15.66:0/147554464 learned_addr learned my addr 172.21.15.66:0/147554464 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5966ffd640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59681053a0 0x7f5968198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f59677fe640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5968102a70 0x7f59681981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5967fff640 1 -- 172.21.15.66:0/147554464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5968102a70 msgr2=0x7f59681981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5967fff640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5968102a70 0x7f59681981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5967fff640 1 -- 172.21.15.66:0/147554464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59681053a0 msgr2=0x7f5968198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5967fff640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59681053a0 0x7f5968198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5967fff640 1 -- 172.21.15.66:0/147554464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596819fe00 con 0x7f5968107cd0 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f59677fe640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5968102a70 0x7f59681981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5967fff640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596819f790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f596002c700 tx=0x7f59600346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5964ff9640 1 -- 172.21.15.66:0/147554464 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960041020 con 0x7f5968107cd0 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5964ff9640 1 -- 172.21.15.66:0/147554464 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5960034ca0 con 0x7f5968107cd0 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59681a0090 con 0x7f5968107cd0 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f5964ff9640 1 -- 172.21.15.66:0/147554464 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960039040 con 0x7f5968107cd0 2024-04-21T09:08:03.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.587+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59681a0580 con 0x7f5968107cd0 2024-04-21T09:08:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5964ff9640 1 -- 172.21.15.66:0/147554464 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5960044020 con 0x7f5968107cd0 2024-04-21T09:08:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 Processor -- start 2024-04-21T09:08:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 -- start start 2024-04-21T09:08:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa4bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abbf9afb10 con 0x55abbfa4c2d0 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55abbf94d240 con 0x55abbfa4ae40 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5970b94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55abbf9abad0 con 0x55abbfa4b830 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa52eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55484/0 (socket says 172.21.15.66:55484) 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.590+0000 7f5966ffd640 1 -- 172.21.15.66:0/4081945273 learned_addr learned my addr 172.21.15.66:0/4081945273 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5964ff9640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f59500a3bb0 0x7f59500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5966ffd640 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f59677fe640 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa4bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5967fff640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f59500a3bb0 0x7f59500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5966ffd640 1 -- 172.21.15.66:0/4081945273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 msgr2=0x55abbfa4bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5966ffd640 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa4bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5966ffd640 1 -- 172.21.15.66:0/4081945273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 msgr2=0x55abbfa4b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5966ffd640 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa4b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5966ffd640 1 -- 172.21.15.66:0/4081945273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abbf6c22d0 con 0x55abbfa4c2d0 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5964ff9640 1 -- 172.21.15.66:0/147554464 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f59600f25a0 con 0x7f5968107cd0 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f59677fe640 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa4bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5966ffd640 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa52eb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f595c009fd0 tx=0x7f595c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e238393629af0e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f5967fff640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f59500a3bb0 0x7f59500a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5960009dc0 tx=0x7f596002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f594dffb640 1 -- 172.21.15.66:0/4081945273 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c004030 con 0x55abbfa4c2d0 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f594dffb640 1 -- 172.21.15.66:0/4081945273 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c00f790 con 0x55abbfa4c2d0 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.591+0000 7f594dffb640 1 -- 172.21.15.66:0/4081945273 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c003850 con 0x55abbfa4c2d0 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 -- 172.21.15.66:0/4081945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 msgr2=0x55abbfa52eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa52eb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f595c009fd0 tx=0x7f595c005850 comp rx=0 tx=0).stop 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 -- 172.21.15.66:0/4081945273 shutdown_connections 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa52eb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa4bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 --2- 172.21.15.66:0/4081945273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 -- 172.21.15.66:0/4081945273 >> 172.21.15.66:0/4081945273 conn(0x55abbfa46f40 msgr2=0x55abbfa45490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 -- 172.21.15.66:0/4081945273 shutdown_connections 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 -- 172.21.15.66:0/4081945273 wait complete. 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.592+0000 7f5970b94740 1 -- start start 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 1 Processor -- start 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4ae40 0x55abbfa54ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4b830 0x55abbfa55420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa534b0 con 0x55abbfa4b830 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55abbf938190 con 0x55abbfa4c2d0 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa53630 con 0x55abbfa4ae40 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5970b94740 10 client.0 _ll_register_callbacks cb 0x55abbf9338b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4b830 0x55abbfa55420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4b830 0x55abbfa55420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55496/0 (socket says 172.21.15.66:55496) 2024-04-21T09:08:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 -- 172.21.15.66:0/3108500797 learned_addr learned my addr 172.21.15.66:0/3108500797 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4ae40 0x55abbfa54ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 -- 172.21.15.66:0/3108500797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4ae40 msgr2=0x55abbfa54ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4ae40 0x55abbfa54ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 -- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 msgr2=0x55abbfa5c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 -- 172.21.15.66:0/3108500797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abbf78c6b0 con 0x55abbfa4b830 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f59677fe640 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4b830 0x55abbfa55420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5958013790 tx=0x7f5958013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f0690f8542afd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.593+0000 7f5946ffd640 1 -- 172.21.15.66:0/3108500797 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958020060 con 0x55abbfa4b830 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c00f790 con 0x55abbfa4b830 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5946ffd640 1 -- 172.21.15.66:0/3108500797 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f5958011cf0 con 0x55abbfa4b830 2024-04-21T09:08:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5946ffd640 1 -- 172.21.15.66:0/3108500797 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5958027050 con 0x55abbfa4b830 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 10 client.15426 fetch_fsmap learned FSMap version 24 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55abbfa5f8f0 con 0x55abbfa4b830 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5946ffd640 1 -- 172.21.15.66:0/3108500797 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c00f790 con 0x55abbfa4b830 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5946ffd640 1 -- 172.21.15.66:0/3108500797 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f595801a030 con 0x55abbfa4b830 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 10 client.15426 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 20 client.15426 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfa61f00 0x55abbfa63370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 4 client.15426 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abbf9bd530 con 0x55abbfa61f00 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.594+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfa61f00 0x55abbfa63370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.595+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfa61f00 0x55abbfa63370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363577bd87bb502e server_cookie=41bc797672a3c83 in_seq=0 out_seq=0 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.595+0000 7f5946ffd640 10 client.15426 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.595+0000 7f5946ffd640 1 -- 172.21.15.66:0/3108500797 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abbf9bd530 con 0x55abbfa61f00 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.595+0000 7f5946ffd640 10 client.15426 handle_command_reply: tid=0 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.595+0000 7f5970b94740 1 client.15426 shutdown 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4b830 msgr2=0x55abbfa55420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4b830 0x55abbfa55420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5958013790 tx=0x7f5958013c50 comp rx=0 tx=0).stop 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 shutdown_connections 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfa61f00 0x55abbfa63370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4b830 0x55abbfa55420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5946ffd640 0 client.15426 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 --2- 172.21.15.66:0/3108500797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4ae40 0x55abbfa54ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 >> 172.21.15.66:0/3108500797 conn(0x55abbfa46f40 msgr2=0x55abbfa44a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 shutdown_connections 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 1 -- 172.21.15.66:0/3108500797 wait complete. 2024-04-21T09:08:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.596+0000 7f5970b94740 20 client.15426 trim_cache size 0 max 16384 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 Processor -- start 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 -- start start 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa44170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa616b0 con 0x55abbfa4c2d0 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa61d80 con 0x55abbfa4ae40 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5970b94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa62450 con 0x55abbfa4b830 2024-04-21T09:08:03.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55506/0 (socket says 172.21.15.66:55506) 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5966ffd640 1 -- 172.21.15.66:0/1609521673 learned_addr learned my addr 172.21.15.66:0/1609521673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f5966ffd640 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa44170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.614+0000 7f59677fe640 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5966ffd640 1 -- 172.21.15.66:0/1609521673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 msgr2=0x55abbfa5c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5966ffd640 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5966ffd640 1 -- 172.21.15.66:0/1609521673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 msgr2=0x55abbfa44170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5966ffd640 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa44170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5966ffd640 1 -- 172.21.15.66:0/1609521673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abbfa5f710 con 0x55abbfa4c2d0 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f59677fe640 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5966ffd640 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa446b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f595c0058c0 tx=0x7f595c00f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea0ba4e79cf6aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f594d7fa640 1 -- 172.21.15.66:0/1609521673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c00f790 con 0x55abbfa4c2d0 2024-04-21T09:08:03.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f594d7fa640 1 -- 172.21.15.66:0/1609521673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c02f880 con 0x55abbfa4c2d0 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f594d7fa640 1 -- 172.21.15.66:0/1609521673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0035f0 con 0x55abbfa4c2d0 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5970b94740 1 -- 172.21.15.66:0/1609521673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 msgr2=0x55abbfa446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5970b94740 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa446b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f595c0058c0 tx=0x7f595c00f9e0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5970b94740 1 -- 172.21.15.66:0/1609521673 shutdown_connections 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5970b94740 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4c2d0 0x55abbfa446b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5970b94740 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5970b94740 1 --2- 172.21.15.66:0/1609521673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4ae40 0x55abbfa44170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.615+0000 7f5970b94740 1 -- 172.21.15.66:0/1609521673 >> 172.21.15.66:0/1609521673 conn(0x55abbfa46f40 msgr2=0x55abbfa4f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 -- 172.21.15.66:0/1609521673 shutdown_connections 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 -- 172.21.15.66:0/1609521673 wait complete. 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 -- start start 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 Processor -- start 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4ae40 0x55abbfa58650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa631f0 con 0x55abbfa4ae40 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa62b20 con 0x55abbfa4c2d0 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55abbfa638c0 con 0x55abbfa4b830 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5970b94740 10 client.0 _ll_register_callbacks cb 0x7f59580096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4ae40 0x55abbfa58650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4ae40 0x55abbfa58650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55520/0 (socket says 172.21.15.66:55520) 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f5966ffd640 1 -- 172.21.15.66:0/3808394702 learned_addr learned my addr 172.21.15.66:0/3808394702 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.616+0000 7f59677fe640 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5966ffd640 1 -- 172.21.15.66:0/3808394702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 msgr2=0x55abbfa5aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5966ffd640 1 -- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 msgr2=0x55abbfa5b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5966ffd640 1 -- 172.21.15.66:0/3808394702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abbfa5b8f0 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5966ffd640 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4ae40 0x55abbfa58650 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f595c006010 tx=0x7f595c009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce526f1fd2e7a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5946ffd640 1 -- 172.21.15.66:0/3808394702 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0050b0 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5946ffd640 1 -- 172.21.15.66:0/3808394702 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f595c005250 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c02f880 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5946ffd640 1 -- 172.21.15.66:0/3808394702 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f595c02f3d0 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5970b94740 10 client.15432 fetch_fsmap learned FSMap version 24 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55abbfac80c0 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.617+0000 7f5946ffd640 1 -- 172.21.15.66:0/3808394702 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c02f880 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5946ffd640 1 -- 172.21.15.66:0/3808394702 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f595c0039c0 con 0x55abbfa4ae40 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5970b94740 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T09:08:03.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5970b94740 10 client.15432 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5970b94740 20 client.15432 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5970b94740 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfacc830 0x55abbfacb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5970b94740 4 client.15432 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "status"}) -- 0x55abbf9b73a0 con 0x55abbfacc830 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f59677fe640 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfacc830 0x55abbfacb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f59677fe640 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfacc830 0x55abbfacb540 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14baa0fdfe54da56 server_cookie=e5f881f6a45a72c8 in_seq=0 out_seq=0 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5946ffd640 10 client.15432 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5946ffd640 1 -- 172.21.15.66:0/3808394702 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55abbf9b73a0 con 0x55abbfacc830 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5946ffd640 10 client.15432 handle_command_reply: tid=0 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.618+0000 7f5970b94740 1 client.15432 shutdown 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4ae40 msgr2=0x55abbfa58650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4ae40 0x55abbfa58650 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f595c006010 tx=0x7f595c009d50 comp rx=0 tx=0).stop 2024-04-21T09:08:03.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 shutdown_connections 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abbfacc830 0x55abbfacb540 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abbfa4c2d0 0x55abbfa5b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abbfa4b830 0x55abbfa5aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5946ffd640 0 client.15432 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 --2- 172.21.15.66:0/3808394702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abbfa4ae40 0x55abbfa58650 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 >> 172.21.15.66:0/3808394702 conn(0x55abbfa46f40 msgr2=0x55abbfa44170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 shutdown_connections 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 1 -- 172.21.15.66:0/3808394702 wait complete. 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f5970b94740 20 client.15432 trim_cache size 0 max 16384 2024-04-21T09:08:03.626 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24512, 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 136.65470761200001, 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 24, 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 95, 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 91, 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 142.75457285799999 2024-04-21T09:08:03.627 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:03.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f59500a3bb0 msgr2=0x7f59500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f59500a3bb0 0x7f59500a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5960009dc0 tx=0x7f596002e040 comp rx=0 tx=0).stop 2024-04-21T09:08:03.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 msgr2=0x7f596819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596819f790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f596002c700 tx=0x7f59600346d0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 shutdown_connections 2024-04-21T09:08:03.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f59500a3bb0 0x7f59500a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5968107cd0 0x7f596819f790 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59681053a0 0x7f5968198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 --2- 172.21.15.66:0/147554464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5968102a70 0x7f59681981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.619+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 >> 172.21.15.66:0/147554464 conn(0x7f59680fc400 msgr2=0x7f59681033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.620+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 shutdown_connections 2024-04-21T09:08:03.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.620+0000 7f596dfa4640 1 -- 172.21.15.66:0/147554464 wait complete. 2024-04-21T09:08:03.631 INFO:teuthology.run_tasks:Running task sequential... 2024-04-21T09:08:03.640 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-21T09:08:03.640 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-21T09:08:03.641 INFO:teuthology.task.print:Enabling mgr modules 2024-04-21T09:08:03.641 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-21T09:08:03.641 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T09:08:03.641 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:08:03.642 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 -- 172.21.15.66:0/3290650972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b90103b90 msgr2=0x7f0b9010a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 --2- 172.21.15.66:0/3290650972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b90103b90 0x7f0b9010a420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009ca0 tx=0x7f0b8802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 -- 172.21.15.66:0/3290650972 shutdown_connections 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 --2- 172.21.15.66:0/3290650972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b90103b90 0x7f0b9010a420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 --2- 172.21.15.66:0/3290650972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b901031d0 0x7f0b90103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 --2- 172.21.15.66:0/3290650972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b90101fe0 0x7f0b90102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 -- 172.21.15.66:0/3290650972 >> 172.21.15.66:0/3290650972 conn(0x7f0b900fd570 msgr2=0x7f0b900ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 -- 172.21.15.66:0/3290650972 shutdown_connections 2024-04-21T09:08:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.742+0000 7f0b970fd640 1 -- 172.21.15.66:0/3290650972 wait complete. 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 Processor -- start 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 -- start start 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b90101fe0 0x7f0b901980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b901031d0 0x7f0b90198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b90103b90 0x7f0b9019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9010db20 con 0x7f0b901031d0 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9010d9a0 con 0x7f0b90101fe0 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b970fd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9010dca0 con 0x7f0b90103b90 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b901031d0 0x7f0b90198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b901031d0 0x7f0b90198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55530/0 (socket says 172.21.15.66:55530) 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b8ffff640 1 -- 172.21.15.66:0/753734020 learned_addr learned my addr 172.21.15.66:0/753734020 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b9569a640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b90103b90 0x7f0b9019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.743+0000 7f0b94e99640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b90101fe0 0x7f0b901980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8ffff640 1 -- 172.21.15.66:0/753734020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b90103b90 msgr2=0x7f0b9019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8ffff640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b90103b90 0x7f0b9019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8ffff640 1 -- 172.21.15.66:0/753734020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b90101fe0 msgr2=0x7f0b901980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8ffff640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b90101fe0 0x7f0b901980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8ffff640 1 -- 172.21.15.66:0/753734020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9019fd10 con 0x7f0b901031d0 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b94e99640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b90101fe0 0x7f0b901980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8ffff640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b901031d0 0x7f0b90198630 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d770 tx=0x7f0b7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8dffb640 1 -- 172.21.15.66:0/753734020 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c017070 con 0x7f0b901031d0 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8dffb640 1 -- 172.21.15.66:0/753734020 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c0054d0 con 0x7f0b901031d0 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b8dffb640 1 -- 172.21.15.66:0/753734020 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004c30 con 0x7f0b901031d0 2024-04-21T09:08:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9019ffa0 con 0x7f0b901031d0 2024-04-21T09:08:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.744+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b901a04c0 con 0x7f0b901031d0 2024-04-21T09:08:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.745+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b90109300 con 0x7f0b901031d0 2024-04-21T09:08:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.746+0000 7f0b8dffb640 1 -- 172.21.15.66:0/753734020 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0b7c004e10 con 0x7f0b901031d0 2024-04-21T09:08:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.747+0000 7f0b8dffb640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b800a3cd0 0x7f0b800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.747+0000 7f0b94e99640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b800a3cd0 0x7f0b800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.747+0000 7f0b8dffb640 1 -- 172.21.15.66:0/753734020 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f0b7c0d01a0 con 0x7f0b901031d0 2024-04-21T09:08:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.748+0000 7f0b94e99640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b800a3cd0 0x7f0b800a6190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b840058c0 tx=0x7f0b84005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.752+0000 7f0b8dffb640 1 -- 172.21.15.66:0/753734020 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b7c09c7f0 con 0x7f0b901031d0 2024-04-21T09:08:03.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:03 smithi186 ceph-mon[31505]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:08:03.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:03 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3523601243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:03.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:03 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1287206141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:03.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:03 smithi083 ceph-mon[31413]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:08:03.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:03 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3523601243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:03.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:03 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1287206141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:03.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:03 smithi066 ceph-mon[27953]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T09:08:03.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:03 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3523601243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:03.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:03 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1287206141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:03.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:03.988+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f0b90062bf0 con 0x7f0b901031d0 2024-04-21T09:08:04.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.590+0000 7f0b8dffb640 1 -- 172.21.15.66:0/753734020 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f0b7c09c0f0 con 0x7f0b901031d0 2024-04-21T09:08:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b800a3cd0 msgr2=0x7f0b800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b800a3cd0 0x7f0b800a6190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b840058c0 tx=0x7f0b84005850 comp rx=0 tx=0).stop 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b901031d0 msgr2=0x7f0b90198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b901031d0 0x7f0b90198630 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d770 tx=0x7f0b7c00dc30 comp rx=0 tx=0).stop 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 shutdown_connections 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f0b800a3cd0 0x7f0b800a6190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b90103b90 0x7f0b9019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b901031d0 0x7f0b90198630 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 --2- 172.21.15.66:0/753734020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b90101fe0 0x7f0b901980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 >> 172.21.15.66:0/753734020 conn(0x7f0b900fd570 msgr2=0x7f0b901064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 shutdown_connections 2024-04-21T09:08:04.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.593+0000 7f0b970fd640 1 -- 172.21.15.66:0/753734020 wait complete. 2024-04-21T09:08:04.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-21T09:08:04.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 -- 172.21.15.66:0/2442004742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102a20 msgr2=0x7f765c109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:04.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 --2- 172.21.15.66:0/2442004742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102a20 0x7f765c109340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7654009ca0 tx=0x7f765402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 -- 172.21.15.66:0/2442004742 shutdown_connections 2024-04-21T09:08:04.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 --2- 172.21.15.66:0/2442004742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102a20 0x7f765c109340 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 --2- 172.21.15.66:0/2442004742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765c102060 0x7f765c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 --2- 172.21.15.66:0/2442004742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c100e80 0x7f765c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 -- 172.21.15.66:0/2442004742 >> 172.21.15.66:0/2442004742 conn(0x7f765c0fc460 msgr2=0x7f765c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:04.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 -- 172.21.15.66:0/2442004742 shutdown_connections 2024-04-21T09:08:04.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 -- 172.21.15.66:0/2442004742 wait complete. 2024-04-21T09:08:04.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 Processor -- start 2024-04-21T09:08:04.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.723+0000 7f7660b59640 1 -- start start 2024-04-21T09:08:04.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7660b59640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765c100e80 0x7f765c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:04.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7660b59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102060 0x7f765c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:04.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7660b59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c102a20 0x7f765c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:04.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7660b59640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f765c10ca30 con 0x7f765c102060 2024-04-21T09:08:04.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7660b59640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f765c10c8b0 con 0x7f765c102a20 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7660b59640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f765c10cbb0 con 0x7f765c100e80 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102060 0x7f765c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102060 0x7f765c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46202/0 (socket says 172.21.15.66:46202) 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 -- 172.21.15.66:0/3465808882 learned_addr learned my addr 172.21.15.66:0/3465808882 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f765a59c640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765c100e80 0x7f765c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f765ad9d640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c102a20 0x7f765c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 -- 172.21.15.66:0/3465808882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765c100e80 msgr2=0x7f765c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765c100e80 0x7f765c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 -- 172.21.15.66:0/3465808882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c102a20 msgr2=0x7f765c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c102a20 0x7f765c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 -- 172.21.15.66:0/3465808882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c19fd30 con 0x7f765c102060 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f7659d9b640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102060 0x7f765c198650 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f764c00d770 tx=0x7f764c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.724+0000 7f764b7fe640 1 -- 172.21.15.66:0/3465808882 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c017070 con 0x7f765c102060 2024-04-21T09:08:04.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.725+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765c19ffc0 con 0x7f765c102060 2024-04-21T09:08:04.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.725+0000 7f764b7fe640 1 -- 172.21.15.66:0/3465808882 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c0054d0 con 0x7f765c102060 2024-04-21T09:08:04.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.725+0000 7f764b7fe640 1 -- 172.21.15.66:0/3465808882 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c004c30 con 0x7f765c102060 2024-04-21T09:08:04.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.725+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765c1a04e0 con 0x7f765c102060 2024-04-21T09:08:04.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.726+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f765c1a0710 con 0x7f765c102060 2024-04-21T09:08:04.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.727+0000 7f764b7fe640 1 -- 172.21.15.66:0/3465808882 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f764c004dd0 con 0x7f765c102060 2024-04-21T09:08:04.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.727+0000 7f764b7fe640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 0x7f76440a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:04.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.728+0000 7f765a59c640 1 -- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 msgr2=0x7f76440a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:04.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.728+0000 7f764b7fe640 1 -- 172.21.15.66:0/3465808882 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f764c0d0550 con 0x7f765c102060 2024-04-21T09:08:04.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.728+0000 7f765a59c640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 0x7f76440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:04.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.736+0000 7f764b7fe640 1 -- 172.21.15.66:0/3465808882 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f764c09c550 con 0x7f765c102060 2024-04-21T09:08:04.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:04 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/753734020' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T09:08:04.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: ignoring --setuser ceph since I am not root 2024-04-21T09:08:04.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: ignoring --setgroup ceph since I am not root 2024-04-21T09:08:04.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:04 smithi186 ceph-mgr[33460]: -- 172.21.15.186:0/1043376607 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a902e725a0 con 0x55a902e46c00 2024-04-21T09:08:04.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:04.774+0000 7f338e9ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T09:08:04.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:04 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/753734020' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T09:08:04.860 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:04 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: ignoring --setuser ceph since I am not root 2024-04-21T09:08:04.860 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:04 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: ignoring --setgroup ceph since I am not root 2024-04-21T09:08:04.860 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:04 smithi083 ceph-mgr[32843]: -- 172.21.15.83:0/1676364316 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557929c9a5a0 con 0x557929c6f000 2024-04-21T09:08:04.860 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:04 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:04.769+0000 7f1048d2e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T09:08:04.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:04 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/753734020' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T09:08:04.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: ignoring --setuser ceph since I am not root 2024-04-21T09:08:04.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: ignoring --setgroup ceph since I am not root 2024-04-21T09:08:04.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:04.788+0000 7f1d04a04200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T09:08:04.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.924+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f765c1a08e0 con 0x7f765c102060 2024-04-21T09:08:04.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.928+0000 7f764b7fe640 1 -- 172.21.15.66:0/3465808882 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f764c0a0850 con 0x7f765c102060 2024-04-21T09:08:04.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.929+0000 7f765a59c640 1 -- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 msgr2=0x7f76440a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:04.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.929+0000 7f765a59c640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 0x7f76440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T09:08:04.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.931+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 msgr2=0x7f76440a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:04.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.931+0000 7f7660b59640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 0x7f76440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.931+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102060 msgr2=0x7f765c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:04.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.931+0000 7f7660b59640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102060 0x7f765c198650 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f764c00d770 tx=0x7f764c00dc30 comp rx=0 tx=0).stop 2024-04-21T09:08:04.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 shutdown_connections 2024-04-21T09:08:04.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f76440a3ca0 0x7f76440a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c102a20 0x7f765c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f765c102060 0x7f765c198650 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 --2- 172.21.15.66:0/3465808882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765c100e80 0x7f765c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:04.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 >> 172.21.15.66:0/3465808882 conn(0x7f765c0fc460 msgr2=0x7f765c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:04.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 shutdown_connections 2024-04-21T09:08:04.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:04.932+0000 7f7660b59640 1 -- 172.21.15.66:0/3465808882 wait complete. 2024-04-21T09:08:04.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-21T09:08:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 -- 172.21.15.66:0/2134053836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 msgr2=0x7fd4bc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/2134053836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc101280 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009900 tx=0x7fd4b802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 -- 172.21.15.66:0/2134053836 shutdown_connections 2024-04-21T09:08:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/2134053836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc102a00 0x7fd4bc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/2134053836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4bc102040 0x7fd4bc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/2134053836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc101280 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 -- 172.21.15.66:0/2134053836 >> 172.21.15.66:0/2134053836 conn(0x7fd4bc0fc420 msgr2=0x7fd4bc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 -- 172.21.15.66:0/2134053836 shutdown_connections 2024-04-21T09:08:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.051+0000 7fd4c3c77640 1 -- 172.21.15.66:0/2134053836 wait complete. 2024-04-21T09:08:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 Processor -- start 2024-04-21T09:08:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 -- start start 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc102040 0x7fd4bc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4bc102a00 0x7fd4bc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc10c9f0 con 0x7fd4bc100e60 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc10c870 con 0x7fd4bc102a00 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c3c77640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc10cb70 con 0x7fd4bc102040 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c1a13640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c1a13640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46220/0 (socket says 172.21.15.66:46220) 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c1a13640 1 -- 172.21.15.66:0/3910816741 learned_addr learned my addr 172.21.15.66:0/3910816741 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c1212640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc102040 0x7fd4bc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.052+0000 7fd4c2214640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4bc102a00 0x7fd4bc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c1a13640 1 -- 172.21.15.66:0/3910816741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc102040 msgr2=0x7fd4bc198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c1a13640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc102040 0x7fd4bc198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c1a13640 1 -- 172.21.15.66:0/3910816741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4bc102a00 msgr2=0x7fd4bc19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c1a13640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4bc102a00 0x7fd4bc19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c1a13640 1 -- 172.21.15.66:0/3910816741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc19fd70 con 0x7fd4bc100e60 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c1a13640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc198150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8000c00 tx=0x7fd4b8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4b2ffd640 1 -- 172.21.15.66:0/3910816741 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8043020 con 0x7fd4bc100e60 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4b2ffd640 1 -- 172.21.15.66:0/3910816741 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b8004350 con 0x7fd4bc100e60 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc1a0000 con 0x7fd4bc100e60 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4b2ffd640 1 -- 172.21.15.66:0/3910816741 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8003b80 con 0x7fd4bc100e60 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.053+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc1a05a0 con 0x7fd4bc100e60 2024-04-21T09:08:05.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.054+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4bc1a07a0 con 0x7fd4bc100e60 2024-04-21T09:08:05.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.055+0000 7fd4b2ffd640 1 -- 172.21.15.66:0/3910816741 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fd4b8046020 con 0x7fd4bc100e60 2024-04-21T09:08:05.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.056+0000 7fd4b2ffd640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd4a80a3d20 0x7fd4a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:05.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.056+0000 7fd4c1212640 1 -- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd4a80a3d20 msgr2=0x7fd4a80a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:05.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.056+0000 7fd4b2ffd640 1 -- 172.21.15.66:0/3910816741 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fd4b80f2870 con 0x7fd4bc100e60 2024-04-21T09:08:05.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.056+0000 7fd4c1212640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd4a80a3d20 0x7fd4a80a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.060+0000 7fd4b2ffd640 1 -- 172.21.15.66:0/3910816741 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4b804d020 con 0x7fd4bc100e60 2024-04-21T09:08:05.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.248+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd4bc1a0e40 con 0x7fd4bc100e60 2024-04-21T09:08:05.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.253+0000 7fd4b2ffd640 1 -- 172.21.15.66:0/3910816741 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7fd4b80beec0 con 0x7fd4bc100e60 2024-04-21T09:08:05.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd4a80a3d20 msgr2=0x7fd4a80a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd4a80a3d20 0x7fd4a80a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 msgr2=0x7fd4bc198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc198150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8000c00 tx=0x7fd4b8005dc0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 shutdown_connections 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fd4a80a3d20 0x7fd4a80a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4bc102a00 0x7fd4bc19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4bc102040 0x7fd4bc198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 --2- 172.21.15.66:0/3910816741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4bc100e60 0x7fd4bc198150 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 >> 172.21.15.66:0/3910816741 conn(0x7fd4bc0fc420 msgr2=0x7fd4bc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:05.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 shutdown_connections 2024-04-21T09:08:05.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:05.256+0000 7fd4c3c77640 1 -- 172.21.15.66:0/3910816741 wait complete. 2024-04-21T09:08:05.266 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:04.989+0000 7f1d04a04200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T09:08:05.266 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:05.101+0000 7f1d04a04200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T09:08:05.267 INFO:teuthology.run_tasks:Running task kclient... 2024-04-21T09:08:05.280 INFO:tasks.kclient:Mounting kernel clients... 2024-04-21T09:08:05.281 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-21T09:08:05.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:05.281 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-21T09:08:05.296 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-21T09:08:05.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:05.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T09:08:05.337 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:04.959+0000 7f338e9ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T09:08:05.337 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:05.062+0000 7f338e9ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T09:08:05.359 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:04 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:04.953+0000 7f1048d2e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T09:08:05.359 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:05 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:05.055+0000 7f1048d2e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T09:08:05.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:05.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-04-21T09:08:05.461 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-21T09:08:05.462 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T09:08:05.462 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:05.462 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T09:08:05.527 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:05.432+0000 7f1d04a04200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T09:08:05.720 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:08:05.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:05.375+0000 7f338e9ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T09:08:05.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:05.460+0000 7f338e9ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T09:08:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:05 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/753734020' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T09:08:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:05 smithi186 ceph-mon[31505]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T09:08:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:05 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3465808882' entity='client.admin' 2024-04-21T09:08:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:05 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3910816741' entity='client.admin' 2024-04-21T09:08:05.842 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:05.526+0000 7f1d04a04200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T09:08:05.843 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:05 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/753734020' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T09:08:05.843 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:05 smithi066 ceph-mon[27953]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T09:08:05.843 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:05 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3465808882' entity='client.admin' 2024-04-21T09:08:05.843 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:05 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3910816741' entity='client.admin' 2024-04-21T09:08:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:05 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/753734020' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T09:08:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:05 smithi083 ceph-mon[31413]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T09:08:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:05 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3465808882' entity='client.admin' 2024-04-21T09:08:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:05 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3910816741' entity='client.admin' 2024-04-21T09:08:05.860 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:05 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:05.362+0000 7f1048d2e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T09:08:05.860 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:05 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:05.447+0000 7f1048d2e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T09:08:06.869 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:06 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:06.812+0000 7f1d04a04200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T09:08:06.958 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:06 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:06.620+0000 7f1048d2e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T09:08:06.958 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:06 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:06.790+0000 7f1048d2e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T09:08:06.958 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:06 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:06.958+0000 7f1048d2e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T09:08:06.996 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:06.655+0000 7f338e9ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T09:08:06.996 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:06.825+0000 7f338e9ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T09:08:06.996 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:06.996+0000 7f338e9ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T09:08:07.183 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:07.000+0000 7f1d04a04200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/3449588597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 msgr2=0x7f1f08101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/3449588597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f08101490 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009990 tx=0x7f1efc009ca0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/3449588597 shutdown_connections 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/3449588597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f08108ad0 0x7f1f08108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/3449588597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f081019d0 0x7f1f0810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/3449588597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f08101490 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/3449588597 >> 172.21.15.66:0/3449588597 conn(0x7f1f08077fb0 msgr2=0x7f1f080ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.424+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/3449588597 shutdown_connections 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.425+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/3449588597 wait complete. 2024-04-21T09:08:07.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.425+0000 7f1f0e6c1700 1 Processor -- start 2024-04-21T09:08:07.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.425+0000 7f1f0e6c1700 1 -- start start 2024-04-21T09:08:07.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f0e6c1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f081990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:07.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f0e6c1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f081019d0 0x7f1f08199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:07.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f0e6c1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f08108ad0 0x7f1f0819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:07.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f0e6c1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08112c00 con 0x7f1f08101030 2024-04-21T09:08:07.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f0e6c1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08112a80 con 0x7f1f081019d0 2024-04-21T09:08:07.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f0e6c1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08112d80 con 0x7f1f08108ad0 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f081990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f077fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f081019d0 0x7f1f08199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f081990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46224/0 (socket says 172.21.15.66:46224) 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f07fff700 1 -- 172.21.15.66:0/4267609174 learned_addr learned my addr 172.21.15.66:0/4267609174 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f0cc5e700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f08108ad0 0x7f1f0819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f07fff700 1 -- 172.21.15.66:0/4267609174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f08108ad0 msgr2=0x7f1f0819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.426+0000 7f1f07fff700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f08108ad0 0x7f1f0819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f07fff700 1 -- 172.21.15.66:0/4267609174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f081019d0 msgr2=0x7f1f08199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f07fff700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f081019d0 0x7f1f08199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f07fff700 1 -- 172.21.15.66:0/4267609174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0819e0b0 con 0x7f1f08101030 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f0cc5e700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f08108ad0 0x7f1f0819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f077fe700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f081019d0 0x7f1f08199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f07fff700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f081990f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800d980 tx=0x7f1ef800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:07.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f057fa700 1 -- 172.21.15.66:0/4267609174 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8016070 con 0x7f1f08101030 2024-04-21T09:08:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0819e3a0 con 0x7f1f08101030 2024-04-21T09:08:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f057fa700 1 -- 172.21.15.66:0/4267609174 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef8004830 con 0x7f1f08101030 2024-04-21T09:08:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f057fa700 1 -- 172.21.15.66:0/4267609174 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8005540 con 0x7f1f08101030 2024-04-21T09:08:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.427+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f081a5c20 con 0x7f1f08101030 2024-04-21T09:08:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.428+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f08104350 con 0x7f1f08101030 2024-04-21T09:08:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.430+0000 7f1f057fa700 1 -- 172.21.15.66:0/4267609174 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1ef8004220 con 0x7f1f08101030 2024-04-21T09:08:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.431+0000 7f1f057fa700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 0x7f1ef00a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.431+0000 7f1f077fe700 1 -- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 msgr2=0x7f1ef00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.431+0000 7f1f057fa700 1 -- 172.21.15.66:0/4267609174 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f1ef80d0a40 con 0x7f1f08101030 2024-04-21T09:08:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.431+0000 7f1f077fe700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 0x7f1ef00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:07.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.435+0000 7f1f057fa700 1 -- 172.21.15.66:0/4267609174 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ef801f020 con 0x7f1f08101030 2024-04-21T09:08:07.435 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:07.182+0000 7f1d04a04200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T09:08:07.542 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:07 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:07.246+0000 7f1048d2e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T09:08:07.542 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:07 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:07.339+0000 7f1048d2e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T09:08:07.542 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:07 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:07.542+0000 7f1048d2e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T09:08:07.586 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:07.288+0000 7f338e9ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T09:08:07.586 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:07.381+0000 7f338e9ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T09:08:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.632+0000 7f1f077fe700 1 -- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 msgr2=0x7f1ef00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.632+0000 7f1f077fe700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 0x7f1ef00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T09:08:07.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.691+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f0819a400 con 0x7f1f08101030 2024-04-21T09:08:07.691 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:07.496+0000 7f1d04a04200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T09:08:07.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:07.594+0000 7f1d04a04200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T09:08:07.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.691+0000 7f1f057fa700 1 -- 172.21.15.66:0/4267609174 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f1ef809d090 con 0x7f1f08101030 2024-04-21T09:08:07.692 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T09:08:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 msgr2=0x7f1ef00a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 0x7f1ef00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 msgr2=0x7f1f081990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f081990f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800d980 tx=0x7f1ef800dc90 comp rx=0 tx=0).stop 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 shutdown_connections 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f1ef00a3c90 0x7f1ef00a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f08108ad0 0x7f1f0819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f08101030 0x7f1f081990f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 --2- 172.21.15.66:0/4267609174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f081019d0 0x7f1f08199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 >> 172.21.15.66:0/4267609174 conn(0x7f1f08077fb0 msgr2=0x7f1f080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 shutdown_connections 2024-04-21T09:08:07.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:07.695+0000 7f1f0e6c1700 1 -- 172.21.15.66:0/4267609174 wait complete. 2024-04-21T09:08:07.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:07 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4267609174' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T09:08:07.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:07.586+0000 7f338e9ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T09:08:07.838 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:07.785+0000 7f338e9ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T09:08:07.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:07 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4267609174' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T09:08:07.859 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:07 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:07.735+0000 7f1048d2e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T09:08:08.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:07 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4267609174' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T09:08:08.021 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:07.817+0000 7f1d04a04200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T09:08:08.324 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:08.020+0000 7f1d04a04200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T09:08:08.354 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-21T09:08:08.354 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T09:08:08.354 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-04-21T09:08:08.354 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-21T09:08:08.354 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:08.355 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-21T09:08:08.355 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T09:08:08.355 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T09:08:08.355 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-21T09:08:08.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:08.355 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-04-21T09:08:08.371 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42048sec preferred_lft 42048sec 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::37c:f2c2:71f0:1287/64 scope link noprefixroute 2024-04-21T09:08:08.372 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-21T09:08:08.373 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T09:08:08.373 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:08:08.373 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:08:08.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-04-21T09:08:08.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-04-21T09:08:08.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-04-21T09:08:08.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T09:08:08.373 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:08:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:08:08.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:08.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:08.515 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T09:08:08.540 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-04-21T09:08:08.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:08.541 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-04-21T09:08:08.599 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-04-21T09:08:08.599 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T09:08:08.599 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-21T09:08:08.599 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-21T09:08:08.599 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T09:08:08.599 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:08:08.599 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:08:08.600 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-21T09:08:08.600 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-21T09:08:08.600 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-21T09:08:08.600 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:08:08.609 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:08 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:08.331+0000 7f1048d2e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T09:08:08.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:08:08.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:08.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:08.729 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-21T09:08:08.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:08.744 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-04-21T09:08:08.799 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:08:08.799 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:08:08.799 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T09:08:08.800 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-21T09:08:08.800 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:08:08.821 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:08.395+0000 7f338e9ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T09:08:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:08:08.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:08.918 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-21T09:08:08.918 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:08:08.918 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:08:08.918 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-21T09:08:08.918 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T09:08:08.919 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-21T09:08:08.919 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-21T09:08:08.919 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-21T09:08:08.919 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:08:08.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:08:08.976 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:08.664+0000 7f1d04a04200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T09:08:09.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:09.034 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T09:08:09.034 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-21T09:08:09.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-04-21T09:08:09.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T09:08:09.034 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-21T09:08:09.048 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:08 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:08.753+0000 7f1048d2e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T09:08:09.048 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:08 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:08.848+0000 7f1048d2e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T09:08:09.048 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:09.048+0000 7f1048d2e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T09:08:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:08:09.087 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:08.822+0000 7f338e9ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T09:08:09.087 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:08.919+0000 7f338e9ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T09:08:09.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:09.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T09:08:09.112 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:09.126 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-21T09:08:09.126 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T09:08:09.127 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:09.182 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-21T09:08:09.182 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-21T09:08:09.182 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/1215737e-19f8-4b71-ae4b-bbe3123bfec4 2024-04-21T09:08:09.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:09.182 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/1215737e-19f8-4b71-ae4b-bbe3123bfec4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-21T09:08:09.236 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:09.133+0000 7f1d04a04200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T09:08:09.266 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-21T09:08:09.266 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-21T09:08:09.266 INFO:teuthology.orchestra.run.smithi066.stdout:invalid new device string format 2024-04-21T09:08:09.393 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-21T09:08:09.393 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-21T09:08:09.393 INFO:teuthology.orchestra.run.smithi066.stdout:invalid new device string format 2024-04-21T09:08:09.393 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.83:3300,172.21.15.186:3300" 2024-04-21T09:08:09.393 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.66:3300,172.21.15.83:3300,172.21.15.186:3300:/volumes/qa/sv_1/1215737e-19f8-4b71-ae4b-bbe3123bfec4 2024-04-21T09:08:09.393 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=0cca84f6-ffbd-11ee-bc93-c7b262605968" will pass to kernel 2024-04-21T09:08:09.394 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-21T09:08:09.394 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-04-21T09:08:09.395 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-21T09:08:09.395 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-04-21T09:08:09.395 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-21T09:08:09.396 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-21T09:08:09.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:09.397 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:09.467 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T09:08:09.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T09:08:09.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:09.568 INFO:teuthology.orchestra.run.smithi066.stdout:client25063 2024-04-21T09:08:09.568 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:08:09.568 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T09:08:09.586 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T09:08:09.587 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:09.125+0000 7f338e9ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T09:08:09.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:09.236+0000 7f1d04a04200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T09:08:09.619 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:09.456+0000 7f1d04a04200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T09:08:09.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T09:08:09.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:09.679 INFO:teuthology.orchestra.run.smithi066.stdout:client25063 2024-04-21T09:08:09.679 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T09:08:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T09:08:09.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:09.738 INFO:teuthology.orchestra.run.smithi066.stdout:client25063 2024-04-21T09:08:09.738 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968.client25063/status') 2024-04-21T09:08:09.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T09:08:09.797 INFO:teuthology.orchestra.run.smithi066.stdout:instance: client.25063 (3)172.21.15.66:0/3367175102 2024-04-21T09:08:09.798 INFO:teuthology.orchestra.run.smithi066.stdout:blocklisted: false 2024-04-21T09:08:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records in 2024-04-21T09:08:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records out 2024-04-21T09:08:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:71 bytes copied, 2.4264e-05 s, 2.9 MB/s 2024-04-21T09:08:09.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T09:08:09.805 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T09:08:09.814 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T09:08:09.814 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:08:09.814 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-21T09:08:09.859 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:09.573+0000 7f1048d2e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T09:08:09.859 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:09.665+0000 7f1048d2e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 -- 172.21.15.66:0/3296930190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac102a00 msgr2=0x7f23ac109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/3296930190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac102a00 0x7f23ac109320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009ca0 tx=0x7f23a402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 -- 172.21.15.66:0/3296930190 shutdown_connections 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/3296930190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac102a00 0x7f23ac109320 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/3296930190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac102040 0x7f23ac1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/3296930190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ac100e60 0x7f23ac101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 -- 172.21.15.66:0/3296930190 >> 172.21.15.66:0/3296930190 conn(0x7f23ac0fc420 msgr2=0x7f23ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 -- 172.21.15.66:0/3296930190 shutdown_connections 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.918+0000 7f23b3c9a640 1 -- 172.21.15.66:0/3296930190 wait complete. 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 Processor -- start 2024-04-21T09:08:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 -- start start 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac100e60 0x7f23ac104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac102040 0x7f23ac1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ac102a00 0x7f23ac1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10ca10 con 0x7f23ac100e60 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10c890 con 0x7f23ac102040 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b3c9a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10cb90 con 0x7f23ac102a00 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b1a36640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac100e60 0x7f23ac104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b1a36640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac100e60 0x7f23ac104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46250/0 (socket says 172.21.15.66:46250) 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b1a36640 1 -- 172.21.15.66:0/292077596 learned_addr learned my addr 172.21.15.66:0/292077596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b1235640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac102040 0x7f23ac1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.919+0000 7f23b2237640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ac102a00 0x7f23ac1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b1a36640 1 -- 172.21.15.66:0/292077596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ac102a00 msgr2=0x7f23ac1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b1a36640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ac102a00 0x7f23ac1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b1a36640 1 -- 172.21.15.66:0/292077596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac102040 msgr2=0x7f23ac1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b1a36640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac102040 0x7f23ac1034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b1a36640 1 -- 172.21.15.66:0/292077596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac104280 con 0x7f23ac100e60 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b1235640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac102040 0x7f23ac1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b1a36640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac100e60 0x7f23ac104e70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f23a800e9c0 tx=0x7f23a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23a2ffd640 1 -- 172.21.15.66:0/292077596 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a800cd80 con 0x7f23ac100e60 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23a2ffd640 1 -- 172.21.15.66:0/292077596 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a80035f0 con 0x7f23ac100e60 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23a2ffd640 1 -- 172.21.15.66:0/292077596 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8010690 con 0x7f23ac100e60 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac1a2210 con 0x7f23ac100e60 2024-04-21T09:08:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.920+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac1a2710 con 0x7f23ac100e60 2024-04-21T09:08:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.921+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac1a2a30 con 0x7f23ac100e60 2024-04-21T09:08:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.922+0000 7f23a2ffd640 1 -- 172.21.15.66:0/292077596 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f23a8004000 con 0x7f23ac100e60 2024-04-21T09:08:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.923+0000 7f23a2ffd640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 0x7f23980a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.923+0000 7f23b1235640 1 -- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 msgr2=0x7f23980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.923+0000 7f23a2ffd640 1 -- 172.21.15.66:0/292077596 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f23a80d0ea0 con 0x7f23ac100e60 2024-04-21T09:08:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.923+0000 7f23b1235640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 0x7f23980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:09.926+0000 7f23a2ffd640 1 -- 172.21.15.66:0/292077596 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23a809d4f0 con 0x7f23ac100e60 2024-04-21T09:08:09.995 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:09.657+0000 7f338e9ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T09:08:09.995 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:09.749+0000 7f338e9ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T09:08:10.119 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:10.027+0000 7f1d04a04200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T09:08:10.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.124+0000 7f23b1235640 1 -- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 msgr2=0x7f23980a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:10.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.124+0000 7f23b1235640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 0x7f23980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T09:08:10.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.158+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f23ac1a2c00 con 0x7f23ac100e60 2024-04-21T09:08:10.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.210+0000 7f23a2ffd640 1 -- 172.21.15.66:0/292077596 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f23a809cdf0 con 0x7f23ac100e60 2024-04-21T09:08:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 msgr2=0x7f23980a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 0x7f23980a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac100e60 msgr2=0x7f23ac104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac100e60 0x7f23ac104e70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f23a800e9c0 tx=0x7f23a800ee80 comp rx=0 tx=0).stop 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 shutdown_connections 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f23980a3ca0 0x7f23980a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23ac102a00 0x7f23ac1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac102040 0x7f23ac1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 --2- 172.21.15.66:0/292077596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23ac100e60 0x7f23ac104e70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 >> 172.21.15.66:0/292077596 conn(0x7f23ac0fc420 msgr2=0x7f23ac0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 shutdown_connections 2024-04-21T09:08:10.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.213+0000 7f23b3c9a640 1 -- 172.21.15.66:0/292077596 wait complete. 2024-04-21T09:08:10.217 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:09.907+0000 7f1048d2e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T09:08:10.217 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:10.039+0000 7f1048d2e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T09:08:10.217 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:10.132+0000 7f1048d2e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T09:08:10.224 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-21T09:08:10.235 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-21T09:08:10.278 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:10.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/292077596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T09:08:10.338 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:09.995+0000 7f338e9ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T09:08:10.338 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:10.130+0000 7f338e9ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T09:08:10.338 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:10.226+0000 7f338e9ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.386+0000 7f17cbf06640 1 -- 172.21.15.66:0/3886610712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c41007e0 msgr2=0x7f17c4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.386+0000 7f17cbf06640 1 --2- 172.21.15.66:0/3886610712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c41007e0 0x7f17c4100c00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009900 tx=0x7f17b802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.386+0000 7f17cbf06640 1 -- 172.21.15.66:0/3886610712 shutdown_connections 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.386+0000 7f17cbf06640 1 --2- 172.21.15.66:0/3886610712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17c4101ae0 0x7f17c4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.386+0000 7f17cbf06640 1 --2- 172.21.15.66:0/3886610712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c4101140 0x7f17c41015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.386+0000 7f17cbf06640 1 --2- 172.21.15.66:0/3886610712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c41007e0 0x7f17c4100c00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.386+0000 7f17cbf06640 1 -- 172.21.15.66:0/3886610712 >> 172.21.15.66:0/3886610712 conn(0x7f17c40fc3e0 msgr2=0x7f17c40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 -- 172.21.15.66:0/3886610712 shutdown_connections 2024-04-21T09:08:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 -- 172.21.15.66:0/3886610712 wait complete. 2024-04-21T09:08:10.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 Processor -- start 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 -- start start 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c41007e0 0x7f17c419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17c4101140 0x7f17c419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c4101ae0 0x7f17c41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410fcc0 con 0x7f17c4101ae0 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410fb40 con 0x7f17c4101140 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17cbf06640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410fe40 con 0x7f17c41007e0 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17ca4a3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c4101ae0 0x7f17c41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17ca4a3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c4101ae0 0x7f17c41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46272/0 (socket says 172.21.15.66:46272) 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.387+0000 7f17ca4a3640 1 -- 172.21.15.66:0/44262923 learned_addr learned my addr 172.21.15.66:0/44262923 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17c9ca2640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c41007e0 0x7f17c419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:10.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17c94a1640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17c4101140 0x7f17c419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17ca4a3640 1 -- 172.21.15.66:0/44262923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c41007e0 msgr2=0x7f17c419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17ca4a3640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c41007e0 0x7f17c419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17ca4a3640 1 -- 172.21.15.66:0/44262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17c4101140 msgr2=0x7f17c419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17ca4a3640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17c4101140 0x7f17c419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17ca4a3640 1 -- 172.21.15.66:0/44262923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c41a1fa0 con 0x7f17c4101ae0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17ca4a3640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c4101ae0 0x7f17c41a1930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00eb10 tx=0x7f17bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17b6ffd640 1 -- 172.21.15.66:0/44262923 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004b20 con 0x7f17c4101ae0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17b6ffd640 1 -- 172.21.15.66:0/44262923 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17bc004cc0 con 0x7f17c4101ae0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c41a2290 con 0x7f17c4101ae0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.388+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a2790 con 0x7f17c4101ae0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.389+0000 7f17b6ffd640 1 -- 172.21.15.66:0/44262923 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc021640 con 0x7f17c4101ae0 2024-04-21T09:08:10.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.389+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c410b740 con 0x7f17c4101ae0 2024-04-21T09:08:10.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.391+0000 7f17b6ffd640 1 -- 172.21.15.66:0/44262923 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f17bc005020 con 0x7f17c4101ae0 2024-04-21T09:08:10.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.391+0000 7f17b6ffd640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 0x7f17b00a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.391+0000 7f17c9ca2640 1 -- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 msgr2=0x7f17b00a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:10.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.392+0000 7f17b6ffd640 1 -- 172.21.15.66:0/44262923 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f17bc0d11d0 con 0x7f17c4101ae0 2024-04-21T09:08:10.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.392+0000 7f17c9ca2640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 0x7f17b00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:10.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.399+0000 7f17b6ffd640 1 -- 172.21.15.66:0/44262923 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17bc09d820 con 0x7f17c4101ae0 2024-04-21T09:08:10.405 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/292077596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T09:08:10.405 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:10.126+0000 7f1d04a04200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T09:08:10.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/292077596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T09:08:10.547 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:10.248+0000 7f1048d2e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T09:08:10.547 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:10.547+0000 7f1048d2e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T09:08:10.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.590+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17c4062b50 con 0x7f17c4101ae0 2024-04-21T09:08:10.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.591+0000 7f17b6ffd640 1 -- 172.21.15.66:0/44262923 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24710 (secure 0 0 0) 0x7f17bc09d120 con 0x7f17c4101ae0 2024-04-21T09:08:10.592 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:10.592 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:07:40.521019+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:10.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.593+0000 7f17c9ca2640 1 -- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 msgr2=0x7f17b00a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:10.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.593+0000 7f17c9ca2640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 0x7f17b00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 msgr2=0x7f17b00a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 0x7f17b00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c4101ae0 msgr2=0x7f17c41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c4101ae0 0x7f17c41a1930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00eb10 tx=0x7f17bc00c2d0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 shutdown_connections 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f17b00a3c00 0x7f17b00a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c4101ae0 0x7f17c41a1930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17c4101140 0x7f17c419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 --2- 172.21.15.66:0/44262923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c41007e0 0x7f17c419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 >> 172.21.15.66:0/44262923 conn(0x7f17c40fc3e0 msgr2=0x7f17c40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:10.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 shutdown_connections 2024-04-21T09:08:10.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.594+0000 7f17cbf06640 1 -- 172.21.15.66:0/44262923 wait complete. 2024-04-21T09:08:10.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:08:10.607 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:08:10.656 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:10.405+0000 7f1d04a04200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T09:08:10.656 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:10.549+0000 7f1d04a04200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T09:08:10.660 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:10.344+0000 7f338e9ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T09:08:10.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 -- 172.21.15.66:0/1896357242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 msgr2=0x7fede410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/1896357242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede410a0c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feddc009ca0 tx=0x7feddc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 -- 172.21.15.66:0/1896357242 shutdown_connections 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/1896357242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede410a0c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/1896357242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede4105340 0x7fede4107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/1896357242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede4102a10 0x7fede4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 -- 172.21.15.66:0/1896357242 >> 172.21.15.66:0/1896357242 conn(0x7fede40fc400 msgr2=0x7fede40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 -- 172.21.15.66:0/1896357242 shutdown_connections 2024-04-21T09:08:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 -- 172.21.15.66:0/1896357242 wait complete. 2024-04-21T09:08:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 Processor -- start 2024-04-21T09:08:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.713+0000 7fedeb6d2640 1 -- start start 2024-04-21T09:08:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fedeb6d2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede4102a10 0x7fede41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fedeb6d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede4105340 0x7fede4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fedeb6d2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fedeb6d2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fede410dac0 con 0x7fede4107c70 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fedeb6d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fede410d940 con 0x7fede4105340 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fedeb6d2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fede410dc40 con 0x7fede4102a10 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede9c6f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede946e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede4102a10 0x7fede41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede9c6f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46300/0 (socket says 172.21.15.66:46300) 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede9c6f640 1 -- 172.21.15.66:0/654258097 learned_addr learned my addr 172.21.15.66:0/654258097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede8c6d640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede4105340 0x7fede4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede946e640 1 -- 172.21.15.66:0/654258097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede4105340 msgr2=0x7fede4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede946e640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede4105340 0x7fede4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede946e640 1 -- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 msgr2=0x7fede419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede946e640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede946e640 1 -- 172.21.15.66:0/654258097 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede419fde0 con 0x7fede4102a10 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede9c6f640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fede946e640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede4102a10 0x7fede41981c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedd400e9f0 tx=0x7fedd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:10.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.714+0000 7fedda7fc640 1 -- 172.21.15.66:0/654258097 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd400cd80 con 0x7fede4102a10 2024-04-21T09:08:10.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.715+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede41a00d0 con 0x7fede4102a10 2024-04-21T09:08:10.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.715+0000 7fedda7fc640 1 -- 172.21.15.66:0/654258097 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd40035f0 con 0x7fede4102a10 2024-04-21T09:08:10.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.715+0000 7fedda7fc640 1 -- 172.21.15.66:0/654258097 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4010690 con 0x7fede4102a10 2024-04-21T09:08:10.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.715+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede41a0670 con 0x7fede4102a10 2024-04-21T09:08:10.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.715+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede40fdef0 con 0x7fede4102a10 2024-04-21T09:08:10.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.717+0000 7fedda7fc640 1 -- 172.21.15.66:0/654258097 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fedd40037a0 con 0x7fede4102a10 2024-04-21T09:08:10.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.717+0000 7fedda7fc640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedd00a3c00 0x7fedd00a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:10.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.717+0000 7fede8c6d640 1 -- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedd00a3c00 msgr2=0x7fedd00a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:10.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.717+0000 7fedda7fc640 1 -- 172.21.15.66:0/654258097 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fedd40d0c40 con 0x7fede4102a10 2024-04-21T09:08:10.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.717+0000 7fede8c6d640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedd00a3c00 0x7fedd00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:10.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.721+0000 7fedda7fc640 1 -- 172.21.15.66:0/654258097 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedd4043e20 con 0x7fede4102a10 2024-04-21T09:08:10.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.914+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fede402d850 con 0x7fede4102a10 2024-04-21T09:08:10.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.914+0000 7fedda7fc640 1 -- 172.21.15.66:0/654258097 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fedd409d290 con 0x7fede4102a10 2024-04-21T09:08:10.915 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:08:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedd00a3c00 msgr2=0x7fedd00a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedd00a3c00 0x7fedd00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede4102a10 msgr2=0x7fede41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede4102a10 0x7fede41981c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedd400e9f0 tx=0x7fedd400eeb0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 shutdown_connections 2024-04-21T09:08:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7fedd00a3c00 0x7fedd00a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fede4107c70 0x7fede419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede4105340 0x7fede4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 --2- 172.21.15.66:0/654258097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede4102a10 0x7fede41981c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 >> 172.21.15.66:0/654258097 conn(0x7fede40fc400 msgr2=0x7fede41033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 shutdown_connections 2024-04-21T09:08:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:10.917+0000 7fedeb6d2640 1 -- 172.21.15.66:0/654258097 wait complete. 2024-04-21T09:08:10.927 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:10.655+0000 7f1d04a04200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T09:08:10.927 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:10.783+0000 7f1d04a04200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T09:08:10.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:08:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 -- 172.21.15.66:0/2978702679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc107d30 msgr2=0x7f25fc10a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 --2- 172.21.15.66:0/2978702679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc107d30 0x7f25fc10a120 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009ca0 tx=0x7f25f402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 -- 172.21.15.66:0/2978702679 shutdown_connections 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 --2- 172.21.15.66:0/2978702679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc107d30 0x7f25fc10a120 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 --2- 172.21.15.66:0/2978702679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc105400 0x7f25fc1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 --2- 172.21.15.66:0/2978702679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25fc102ad0 0x7f25fc104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 -- 172.21.15.66:0/2978702679 >> 172.21.15.66:0/2978702679 conn(0x7f25fc0fc440 msgr2=0x7f25fc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.040+0000 7f26027a4640 1 -- 172.21.15.66:0/2978702679 shutdown_connections 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.041+0000 7f26027a4640 1 -- 172.21.15.66:0/2978702679 wait complete. 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.041+0000 7f26027a4640 1 Processor -- start 2024-04-21T09:08:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.041+0000 7f26027a4640 1 -- start start 2024-04-21T09:08:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f26027a4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc102ad0 0x7f25fc102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f26027a4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25fc105400 0x7f25fc100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f26027a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc107d30 0x7f25fc100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f26027a4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc10db10 con 0x7f25fc102ad0 2024-04-21T09:08:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f26027a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc10d990 con 0x7f25fc107d30 2024-04-21T09:08:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f26027a4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc10dc90 con 0x7f25fc105400 2024-04-21T09:08:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc102ad0 0x7f25fc102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc102ad0 0x7f25fc102440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46314/0 (socket says 172.21.15.66:46314) 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 -- 172.21.15.66:0/797895247 learned_addr learned my addr 172.21.15.66:0/797895247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f2600d41640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc107d30 0x7f25fc100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fb7fe640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25fc105400 0x7f25fc100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 -- 172.21.15.66:0/797895247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25fc105400 msgr2=0x7f25fc100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25fc105400 0x7f25fc100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 -- 172.21.15.66:0/797895247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc107d30 msgr2=0x7f25fc100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc107d30 0x7f25fc100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 -- 172.21.15.66:0/797895247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc101850 con 0x7f25fc102ad0 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fb7fe640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25fc105400 0x7f25fc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.042+0000 7f25fbfff640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc102ad0 0x7f25fc102440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e9f0 tx=0x7f25e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.043+0000 7f25f97fa640 1 -- 172.21.15.66:0/797895247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e400cd80 con 0x7f25fc102ad0 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.043+0000 7f25f97fa640 1 -- 172.21.15.66:0/797895247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e40054d0 con 0x7f25fc102ad0 2024-04-21T09:08:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.043+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc1a2320 con 0x7f25fc102ad0 2024-04-21T09:08:11.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.043+0000 7f25f97fa640 1 -- 172.21.15.66:0/797895247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4010690 con 0x7f25fc102ad0 2024-04-21T09:08:11.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.043+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc1a2870 con 0x7f25fc102ad0 2024-04-21T09:08:11.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.044+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0fdf30 con 0x7f25fc102ad0 2024-04-21T09:08:11.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.045+0000 7f25f97fa640 1 -- 172.21.15.66:0/797895247 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f25e4004e90 con 0x7f25fc102ad0 2024-04-21T09:08:11.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.046+0000 7f25f97fa640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 0x7f25e80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.046+0000 7f25fb7fe640 1 -- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 msgr2=0x7f25e80a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:11.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.046+0000 7f25f97fa640 1 -- 172.21.15.66:0/797895247 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f25e40d0bc0 con 0x7f25fc102ad0 2024-04-21T09:08:11.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.046+0000 7f25fb7fe640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 0x7f25e80a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.050+0000 7f25f97fa640 1 -- 172.21.15.66:0/797895247 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25e409d210 con 0x7f25fc102ad0 2024-04-21T09:08:11.087 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:10.661+0000 7f338e9ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T09:08:11.216 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:11.114+0000 7f1d04a04200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T09:08:11.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.247+0000 7f25fb7fe640 1 -- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 msgr2=0x7f25e80a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:11.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.247+0000 7f25fb7fe640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 0x7f25e80a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T09:08:11.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.257+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f25fc062b50 con 0x7f25fc102ad0 2024-04-21T09:08:11.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.258+0000 7f25f97fa640 1 -- 172.21.15.66:0/797895247 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f25e409cb10 con 0x7f25fc102ad0 2024-04-21T09:08:11.258 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:11.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 msgr2=0x7f25e80a6140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 0x7f25e80a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc102ad0 msgr2=0x7f25fc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc102ad0 0x7f25fc102440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e9f0 tx=0x7f25e400edf0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 shutdown_connections 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f25e80a3c80 0x7f25e80a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc107d30 0x7f25fc100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25fc105400 0x7f25fc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 --2- 172.21.15.66:0/797895247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25fc102ad0 0x7f25fc102440 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 >> 172.21.15.66:0/797895247 conn(0x7f25fc0fc440 msgr2=0x7f25fc1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 shutdown_connections 2024-04-21T09:08:11.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.260+0000 7f26027a4640 1 -- 172.21.15.66:0/797895247 wait complete. 2024-04-21T09:08:11.271 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 -- 172.21.15.66:0/1529875174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc1029a0 msgr2=0x7f69dc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/1529875174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc1029a0 0x7f69dc1092c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009ca0 tx=0x7f69d402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 -- 172.21.15.66:0/1529875174 shutdown_connections 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/1529875174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc1029a0 0x7f69dc1092c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/1529875174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc101fe0 0x7f69dc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/1529875174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69dc100e90 0x7f69dc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 -- 172.21.15.66:0/1529875174 >> 172.21.15.66:0/1529875174 conn(0x7f69dc0fc400 msgr2=0x7f69dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 -- 172.21.15.66:0/1529875174 shutdown_connections 2024-04-21T09:08:11.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.377+0000 7f69e0bf4640 1 -- 172.21.15.66:0/1529875174 wait complete. 2024-04-21T09:08:11.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 Processor -- start 2024-04-21T09:08:11.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 -- start start 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc100e90 0x7f69dc104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69dc101fe0 0x7f69dc103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc1029a0 0x7f69dc103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc10c9d0 con 0x7f69dc100e90 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc10c850 con 0x7f69dc1029a0 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69e0bf4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc10cb50 con 0x7f69dc101fe0 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69da59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc100e90 0x7f69dc104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69dad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc1029a0 0x7f69dc103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69da59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc100e90 0x7f69dc104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46338/0 (socket says 172.21.15.66:46338) 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69da59c640 1 -- 172.21.15.66:0/552422731 learned_addr learned my addr 172.21.15.66:0/552422731 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.378+0000 7f69d9d9b640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69dc101fe0 0x7f69dc103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69d9d9b640 1 -- 172.21.15.66:0/552422731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc1029a0 msgr2=0x7f69dc103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69d9d9b640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc1029a0 0x7f69dc103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69d9d9b640 1 -- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc100e90 msgr2=0x7f69dc104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69d9d9b640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc100e90 0x7f69dc104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69d9d9b640 1 -- 172.21.15.66:0/552422731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc104220 con 0x7f69dc101fe0 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69dad9d640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc1029a0 0x7f69dc103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69d9d9b640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69dc101fe0 0x7f69dc103450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00e9c0 tx=0x7f69cc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69cb7fe640 1 -- 172.21.15.66:0/552422731 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc00ccc0 con 0x7f69dc101fe0 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc1a22f0 con 0x7f69dc101fe0 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69cb7fe640 1 -- 172.21.15.66:0/552422731 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69cc0035f0 con 0x7f69dc101fe0 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69cb7fe640 1 -- 172.21.15.66:0/552422731 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc00ce60 con 0x7f69dc101fe0 2024-04-21T09:08:11.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.379+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc1a2890 con 0x7f69dc101fe0 2024-04-21T09:08:11.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.380+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc1a2bb0 con 0x7f69dc101fe0 2024-04-21T09:08:11.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.381+0000 7f69cb7fe640 1 -- 172.21.15.66:0/552422731 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f69cc004000 con 0x7f69dc101fe0 2024-04-21T09:08:11.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.382+0000 7f69cb7fe640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69c40a3c00 0x7f69c40a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.382+0000 7f69da59c640 1 -- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69c40a3c00 msgr2=0x7f69c40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:11.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.382+0000 7f69cb7fe640 1 -- 172.21.15.66:0/552422731 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f69cc0d0c40 con 0x7f69dc101fe0 2024-04-21T09:08:11.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.382+0000 7f69da59c640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69c40a3c00 0x7f69c40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:11.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.386+0000 7f69cb7fe640 1 -- 172.21.15.66:0/552422731 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69cc09d290 con 0x7f69dc101fe0 2024-04-21T09:08:11.492 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:11 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:11.275+0000 7f338e9ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T09:08:11.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/292077596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T09:08:11.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:11 smithi186 ceph-mon[31505]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T09:08:11.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/44262923' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:11.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/654258097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.573+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f69dc1a2d80 con 0x7f69dc101fe0 2024-04-21T09:08:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.573+0000 7f69cb7fe640 1 -- 172.21.15.66:0/552422731 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f69cc09cb90 con 0x7f69dc101fe0 2024-04-21T09:08:11.574 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69c40a3c00 msgr2=0x7f69c40a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69c40a3c00 0x7f69c40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69dc101fe0 msgr2=0x7f69dc103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69dc101fe0 0x7f69dc103450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00e9c0 tx=0x7f69cc00ee80 comp rx=0 tx=0).stop 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 shutdown_connections 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f69c40a3c00 0x7f69c40a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc1029a0 0x7f69dc103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69dc101fe0 0x7f69dc103450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 --2- 172.21.15.66:0/552422731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69dc100e90 0x7f69dc104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 >> 172.21.15.66:0/552422731 conn(0x7f69dc0fc400 msgr2=0x7f69dc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:11.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 shutdown_connections 2024-04-21T09:08:11.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.576+0000 7f69e0bf4640 1 -- 172.21.15.66:0/552422731 wait complete. 2024-04-21T09:08:11.590 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/292077596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T09:08:11.590 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:11 smithi066 ceph-mon[27953]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T09:08:11.590 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/44262923' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:11.591 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/654258097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:11.591 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:11.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/292077596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T09:08:11.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:11 smithi083 ceph-mon[31413]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T09:08:11.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/44262923' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:11.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/654258097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:11.610 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:11 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:11.151+0000 7f1048d2e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T09:08:11.610 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:08:11 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y[32839]: 2024-04-21T09:08:11.366+0000 7f1048d2e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 -- 172.21.15.66:0/1896921001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4105360 msgr2=0x7f82b4107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 --2- 172.21.15.66:0/1896921001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4105360 0x7f82b4107750 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009900 tx=0x7f82a402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 -- 172.21.15.66:0/1896921001 shutdown_connections 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 --2- 172.21.15.66:0/1896921001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b4107c90 0x7f82b410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 --2- 172.21.15.66:0/1896921001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4105360 0x7f82b4107750 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 --2- 172.21.15.66:0/1896921001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82b4102a30 0x7f82b4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 -- 172.21.15.66:0/1896921001 >> 172.21.15.66:0/1896921001 conn(0x7f82b40fc420 msgr2=0x7f82b40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 -- 172.21.15.66:0/1896921001 shutdown_connections 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.697+0000 7f82b962e640 1 -- 172.21.15.66:0/1896921001 wait complete. 2024-04-21T09:08:11.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 Processor -- start 2024-04-21T09:08:11.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 -- start start 2024-04-21T09:08:11.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4102a30 0x7f82b4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b4105360 0x7f82b41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82b4107c90 0x7f82b419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82b410daf0 con 0x7f82b4102a30 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82b410d970 con 0x7f82b4105360 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b962e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f82b410dc70 con 0x7f82b4107c90 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4102a30 0x7f82b4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4102a30 0x7f82b4198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46350/0 (socket says 172.21.15.66:46350) 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b2ffd640 1 -- 172.21.15.66:0/1728831968 learned_addr learned my addr 172.21.15.66:0/1728831968 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b27fc640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b4105360 0x7f82b41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.698+0000 7f82b37fe640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82b4107c90 0x7f82b419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b2ffd640 1 -- 172.21.15.66:0/1728831968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82b4107c90 msgr2=0x7f82b419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b2ffd640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82b4107c90 0x7f82b419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b2ffd640 1 -- 172.21.15.66:0/1728831968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b4105360 msgr2=0x7f82b41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b2ffd640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b4105360 0x7f82b41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b2ffd640 1 -- 172.21.15.66:0/1728831968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b419fe90 con 0x7f82b4102a30 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b2ffd640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4102a30 0x7f82b4198270 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e9f0 tx=0x7f82a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f829bfff640 1 -- 172.21.15.66:0/1728831968 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a800cd80 con 0x7f82b4102a30 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b41a0180 con 0x7f82b4102a30 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f829bfff640 1 -- 172.21.15.66:0/1728831968 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a80054d0 con 0x7f82b4102a30 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f829bfff640 1 -- 172.21.15.66:0/1728831968 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8010690 con 0x7f82b4102a30 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.699+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b41a06a0 con 0x7f82b4102a30 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.700+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b40fdf10 con 0x7f82b4102a30 2024-04-21T09:08:11.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.702+0000 7f829bfff640 1 -- 172.21.15.66:0/1728831968 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f82a8004e90 con 0x7f82b4102a30 2024-04-21T09:08:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.703+0000 7f829bfff640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f829c0a3c00 0x7f829c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.703+0000 7f82b27fc640 1 -- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f829c0a3c00 msgr2=0x7f829c0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.703+0000 7f829bfff640 1 -- 172.21.15.66:0/1728831968 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f82a80d0be0 con 0x7f82b4102a30 2024-04-21T09:08:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.703+0000 7f82b27fc640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f829c0a3c00 0x7f829c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:11.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.712+0000 7f829bfff640 1 -- 172.21.15.66:0/1728831968 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82a809d230 con 0x7f82b4102a30 2024-04-21T09:08:11.837 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:08:11 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z[33421]: 2024-04-21T09:08:11.493+0000 7f338e9ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T09:08:11.869 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:11.786+0000 7f1d04a04200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T09:08:11.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.899+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82b402d5c0 con 0x7f82b4102a30 2024-04-21T09:08:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.900+0000 7f829bfff640 1 -- 172.21.15.66:0/1728831968 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f82b402d5c0 con 0x7f82b4102a30 2024-04-21T09:08:11.900 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f829c0a3c00 msgr2=0x7f829c0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f829c0a3c00 0x7f829c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4102a30 msgr2=0x7f82b4198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4102a30 0x7f82b4198270 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e9f0 tx=0x7f82a800edf0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 shutdown_connections 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f829c0a3c00 0x7f829c0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82b4107c90 0x7f82b419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b4105360 0x7f82b41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 --2- 172.21.15.66:0/1728831968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b4102a30 0x7f82b4198270 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 >> 172.21.15.66:0/1728831968 conn(0x7f82b40fc420 msgr2=0x7f82b41034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 shutdown_connections 2024-04-21T09:08:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:11.902+0000 7f82b962e640 1 -- 172.21.15.66:0/1728831968 wait complete. 2024-04-21T09:08:11.914 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 -- 172.21.15.66:0/1261302714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c101fc0 msgr2=0x7f879c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 --2- 172.21.15.66:0/1261302714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c101fc0 0x7f879c102440 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 -- 172.21.15.66:0/1261302714 shutdown_connections 2024-04-21T09:08:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 --2- 172.21.15.66:0/1261302714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f879c102980 0x7f879c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 --2- 172.21.15.66:0/1261302714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c101fc0 0x7f879c102440 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 --2- 172.21.15.66:0/1261302714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 0x7f879c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 -- 172.21.15.66:0/1261302714 >> 172.21.15.66:0/1261302714 conn(0x7f879c0fc3e0 msgr2=0x7f879c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 -- 172.21.15.66:0/1261302714 shutdown_connections 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 -- 172.21.15.66:0/1261302714 wait complete. 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 Processor -- start 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.017+0000 7f87a3759640 1 -- start start 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 0x7f879c069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f879c101fc0 0x7f879c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c102980 0x7f879c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f879c10c9c0 con 0x7f879c102980 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f879c10c840 con 0x7f879c101fc0 2024-04-21T09:08:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f879c10cb40 con 0x7f879c100e70 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c102980 0x7f879c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a14f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 0x7f879c069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c102980 0x7f879c19f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46360/0 (socket says 172.21.15.66:46360) 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a14f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 0x7f879c069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:37932/0 (socket says 172.21.15.66:37932) 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 -- 172.21.15.66:0/88877113 learned_addr learned my addr 172.21.15.66:0/88877113 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a0cf4640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f879c101fc0 0x7f879c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 -- 172.21.15.66:0/88877113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 msgr2=0x7f879c069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 0x7f879c069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 -- 172.21.15.66:0/88877113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f879c101fc0 msgr2=0x7f879c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f879c101fc0 0x7f879c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 -- 172.21.15.66:0/88877113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c19fde0 con 0x7f879c102980 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a14f5640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 0x7f879c069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a1cf6640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c102980 0x7f879c19f520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f879400eb10 tx=0x7f879400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794004b20 con 0x7f879c102980 2024-04-21T09:08:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c1a00d0 con 0x7f879c102980 2024-04-21T09:08:12.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8794004cc0 con 0x7f879c102980 2024-04-21T09:08:12.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794010670 con 0x7f879c102980 2024-04-21T09:08:12.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.018+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c1a05f0 con 0x7f879c102980 2024-04-21T09:08:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.019+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c108200 con 0x7f879c102980 2024-04-21T09:08:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.021+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f8794005020 con 0x7f879c102980 2024-04-21T09:08:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.022+0000 7f87927fc640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87880a3c00 0x7f87880a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.022+0000 7f87a14f5640 1 -- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87880a3c00 msgr2=0x7f87880a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3032531159 2024-04-21T09:08:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.022+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f87940d0da0 con 0x7f879c102980 2024-04-21T09:08:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.022+0000 7f87a14f5640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87880a3c00 0x7f87880a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T09:08:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.027+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f879409d3f0 con 0x7f879c102980 2024-04-21T09:08:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.039+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7f8794099360 con 0x7f879c102980 2024-04-21T09:08:12.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.040+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87880a3c00 msgr2=0x7f87880a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T09:08:12.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.040+0000 7f87927fc640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87880a3c00 0x7f87880a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.212 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.025+0000 7f1d04a04200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T09:08:12.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.276+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f879c062b50 con 0x7f879c102980 2024-04-21T09:08:12.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.278+0000 7f87927fc640 1 -- 172.21.15.66:0/88877113 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+25086 (secure 0 0 0) 0x7f87940574c0 con 0x7f879c102980 2024-04-21T09:08:12.278 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:12.279 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:08:12.028415+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:12.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c102980 msgr2=0x7f879c19f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c102980 0x7f879c19f520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f879400eb10 tx=0x7f879400c2d0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 shutdown_connections 2024-04-21T09:08:12.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:6800/3032531159,v1:172.21.15.66:6801/3032531159] conn(0x7f87880a3c00 0x7f87880a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f879c102980 0x7f879c19f520 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f879c101fc0 0x7f879c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 --2- 172.21.15.66:0/88877113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f879c100e70 0x7f879c069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 >> 172.21.15.66:0/88877113 conn(0x7f879c0fc3e0 msgr2=0x7f879c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:12.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 shutdown_connections 2024-04-21T09:08:12.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.281+0000 7f87a3759640 1 -- 172.21.15.66:0/88877113 wait complete. 2024-04-21T09:08:12.296 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:08:12.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 -- 172.21.15.66:0/2039906151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc107c70 msgr2=0x7f3bdc10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 --2- 172.21.15.66:0/2039906151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc107c70 0x7f3bdc10a0c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009ca0 tx=0x7f3bd402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 -- 172.21.15.66:0/2039906151 shutdown_connections 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 --2- 172.21.15.66:0/2039906151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc107c70 0x7f3bdc10a0c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 --2- 172.21.15.66:0/2039906151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc105340 0x7f3bdc107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 --2- 172.21.15.66:0/2039906151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc102a10 0x7f3bdc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 -- 172.21.15.66:0/2039906151 >> 172.21.15.66:0/2039906151 conn(0x7f3bdc0fc400 msgr2=0x7f3bdc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 -- 172.21.15.66:0/2039906151 shutdown_connections 2024-04-21T09:08:12.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.408+0000 7f3be214b640 1 -- 172.21.15.66:0/2039906151 wait complete. 2024-04-21T09:08:12.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 Processor -- start 2024-04-21T09:08:12.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 -- start start 2024-04-21T09:08:12.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc102a10 0x7f3bdc1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc105340 0x7f3bdc198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc107c70 0x7f3bdc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10dad0 con 0x7f3bdc102a10 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10d950 con 0x7f3bdc105340 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3be214b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10dc50 con 0x7f3bdc107c70 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3bdb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc102a10 0x7f3bdc1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3bdb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc102a10 0x7f3bdc1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46520/0 (socket says 172.21.15.66:46520) 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.409+0000 7f3bdb7fe640 1 -- 172.21.15.66:0/4118727351 learned_addr learned my addr 172.21.15.66:0/4118727351 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdaffd640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc105340 0x7f3bdc198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdbfff640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc107c70 0x7f3bdc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdb7fe640 1 -- 172.21.15.66:0/4118727351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc107c70 msgr2=0x7f3bdc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdb7fe640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc107c70 0x7f3bdc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdb7fe640 1 -- 172.21.15.66:0/4118727351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc105340 msgr2=0x7f3bdc198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdb7fe640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc105340 0x7f3bdc198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdb7fe640 1 -- 172.21.15.66:0/4118727351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc19fdf0 con 0x7f3bdc102a10 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bdb7fe640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc102a10 0x7f3bdc1981d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00ea60 tx=0x7f3bcc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bd8ff9640 1 -- 172.21.15.66:0/4118727351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00cde0 con 0x7f3bdc102a10 2024-04-21T09:08:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.410+0000 7f3bd8ff9640 1 -- 172.21.15.66:0/4118727351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bcc0035f0 con 0x7f3bdc102a10 2024-04-21T09:08:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.411+0000 7f3bd8ff9640 1 -- 172.21.15.66:0/4118727351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc010690 con 0x7f3bdc102a10 2024-04-21T09:08:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.411+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bdc1a00e0 con 0x7f3bdc102a10 2024-04-21T09:08:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.411+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bdc1a0630 con 0x7f3bdc102a10 2024-04-21T09:08:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.412+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bdc0fdef0 con 0x7f3bdc102a10 2024-04-21T09:08:12.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.414+0000 7f3bd8ff9640 1 -- 172.21.15.66:0/4118727351 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7f3bcc0037a0 con 0x7f3bdc102a10 2024-04-21T09:08:12.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.415+0000 7f3bd8ff9640 1 -- 172.21.15.66:0/4118727351 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f3bcc0d06a0 con 0x7f3bdc102a10 2024-04-21T09:08:12.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.424+0000 7f3bd8ff9640 1 -- 172.21.15.66:0/4118727351 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bcc09cff0 con 0x7f3bdc102a10 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.307+0000 7f1c66afd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.307+0000 7f1c66afd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.307+0000 7f1c66afd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.307+0000 7f1c66afd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.307+0000 7f1c66afd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.309+0000 7f1c63af7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.309+0000 7f1c63af7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.467 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.309+0000 7f1c63af7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.309+0000 7f1c63af7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:08:12.309+0000 7f1c63af7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/797895247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: Standby manager daemon y restarted 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: Standby manager daemon y started 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T09:08:12.468 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: Standby manager daemon z restarted 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: Standby manager daemon z started 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/552422731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1728831968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: Active manager daemon x restarted 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: Activating manager daemon x 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: mgrmap e20: x(active, starting, since 0.0133317s), standbys: z, y 2024-04-21T09:08:12.469 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T09:08:12.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:08:12.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: Manager daemon x is now available 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T09:08:12.472 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:12 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T09:08:12.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/797895247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: Standby manager daemon y restarted 2024-04-21T09:08:12.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: Standby manager daemon y started 2024-04-21T09:08:12.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T09:08:12.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T09:08:12.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: Standby manager daemon z restarted 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: Standby manager daemon z started 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/552422731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1728831968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: Active manager daemon x restarted 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: Activating manager daemon x 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: mgrmap e20: x(active, starting, since 0.0133317s), standbys: z, y 2024-04-21T09:08:12.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:08:12.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:08:12.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: Manager daemon x is now available 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T09:08:12.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:12 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/797895247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: Standby manager daemon y restarted 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: Standby manager daemon y started 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.83:0/1708920141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T09:08:12.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: Standby manager daemon z restarted 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: Standby manager daemon z started 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.? 172.21.15.186:0/1700345166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/552422731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1728831968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: Active manager daemon x restarted 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: Activating manager daemon x 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: mgrmap e20: x(active, starting, since 0.0133317s), standbys: z, y 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T09:08:12.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T09:08:12.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T09:08:12.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T09:08:12.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T09:08:12.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: Manager daemon x is now available 2024-04-21T09:08:12.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:08:12.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T09:08:12.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T09:08:12.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T09:08:12.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:12 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T09:08:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.631+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bdc02d820 con 0x7f3bdc102a10 2024-04-21T09:08:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.632+0000 7f3bd8ff9640 1 -- 172.21.15.66:0/4118727351 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f3bcc09c8f0 con 0x7f3bdc102a10 2024-04-21T09:08:12.632 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc102a10 msgr2=0x7f3bdc1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc102a10 0x7f3bdc1981d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00ea60 tx=0x7f3bcc00ef20 comp rx=0 tx=0).stop 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 shutdown_connections 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc107c70 0x7f3bdc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc105340 0x7f3bdc198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 --2- 172.21.15.66:0/4118727351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bdc102a10 0x7f3bdc1981d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 >> 172.21.15.66:0/4118727351 conn(0x7f3bdc0fc400 msgr2=0x7f3bdc1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 shutdown_connections 2024-04-21T09:08:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.634+0000 7f3be214b640 1 -- 172.21.15.66:0/4118727351 wait complete. 2024-04-21T09:08:12.646 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:08:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 -- 172.21.15.66:0/2752838804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc101ff0 msgr2=0x7feedc102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 --2- 172.21.15.66:0/2752838804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc101ff0 0x7feedc102470 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feed8009900 tx=0x7feed802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 -- 172.21.15.66:0/2752838804 shutdown_connections 2024-04-21T09:08:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 --2- 172.21.15.66:0/2752838804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc1029b0 0x7feedc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 --2- 172.21.15.66:0/2752838804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc101ff0 0x7feedc102470 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 --2- 172.21.15.66:0/2752838804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feedc100ea0 0x7feedc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 -- 172.21.15.66:0/2752838804 >> 172.21.15.66:0/2752838804 conn(0x7feedc0fc410 msgr2=0x7feedc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 -- 172.21.15.66:0/2752838804 shutdown_connections 2024-04-21T09:08:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 -- 172.21.15.66:0/2752838804 wait complete. 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.753+0000 7feee3911640 1 Processor -- start 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee3911640 1 -- start start 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee3911640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc100ea0 0x7feedc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee3911640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101ff0 0x7feedc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee3911640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feedc1029b0 0x7feedc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee3911640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10ca00 con 0x7feedc100ea0 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee3911640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10c880 con 0x7feedc101ff0 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee3911640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10cb80 con 0x7feedc1029b0 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc100ea0 0x7feedc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc100ea0 0x7feedc198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46548/0 (socket says 172.21.15.66:46548) 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 -- 172.21.15.66:0/4109806943 learned_addr learned my addr 172.21.15.66:0/4109806943 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee1eae640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feedc1029b0 0x7feedc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee0eac640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101ff0 0x7feedc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 -- 172.21.15.66:0/4109806943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feedc1029b0 msgr2=0x7feedc19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feedc1029b0 0x7feedc19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 -- 172.21.15.66:0/4109806943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101ff0 msgr2=0x7feedc1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101ff0 0x7feedc1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 -- 172.21.15.66:0/4109806943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc19fe80 con 0x7feedc100ea0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feee16ad640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc100ea0 0x7feedc198260 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9f0 tx=0x7feecc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.754+0000 7feed27fc640 1 -- 172.21.15.66:0/4109806943 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc00ccc0 con 0x7feedc100ea0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.755+0000 7feed27fc640 1 -- 172.21.15.66:0/4109806943 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc00ce60 con 0x7feedc100ea0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.755+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc1a0170 con 0x7feedc100ea0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.755+0000 7feed27fc640 1 -- 172.21.15.66:0/4109806943 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc010430 con 0x7feedc100ea0 2024-04-21T09:08:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.755+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a0640 con 0x7feedc100ea0 2024-04-21T09:08:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.756+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feedc1a0900 con 0x7feedc100ea0 2024-04-21T09:08:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.757+0000 7feed27fc640 1 -- 172.21.15.66:0/4109806943 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7feecc0046c0 con 0x7feedc100ea0 2024-04-21T09:08:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.758+0000 7feed27fc640 1 -- 172.21.15.66:0/4109806943 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7feecc0d06c0 con 0x7feedc100ea0 2024-04-21T09:08:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.762+0000 7feed27fc640 1 -- 172.21.15.66:0/4109806943 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feecc09d010 con 0x7feedc100ea0 2024-04-21T09:08:12.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.981+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feedc1a0ad0 con 0x7feedc100ea0 2024-04-21T09:08:12.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.982+0000 7feed27fc640 1 -- 172.21.15.66:0/4109806943 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7feecc09c910 con 0x7feedc100ea0 2024-04-21T09:08:12.982 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc100ea0 msgr2=0x7feedc198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc100ea0 0x7feedc198260 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9f0 tx=0x7feecc00edf0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 shutdown_connections 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feedc1029b0 0x7feedc19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101ff0 0x7feedc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 --2- 172.21.15.66:0/4109806943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feedc100ea0 0x7feedc198260 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 >> 172.21.15.66:0/4109806943 conn(0x7feedc0fc410 msgr2=0x7feedc06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 shutdown_connections 2024-04-21T09:08:12.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:12.984+0000 7feee3911640 1 -- 172.21.15.66:0/4109806943 wait complete. 2024-04-21T09:08:12.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:13.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.128+0000 7fdd854f2640 1 -- 172.21.15.66:0/1682333172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80103be0 msgr2=0x7fdd8010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.128+0000 7fdd854f2640 1 --2- 172.21.15.66:0/1682333172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80103be0 0x7fdd8010a470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009ca0 tx=0x7fdd7802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.128+0000 7fdd854f2640 1 -- 172.21.15.66:0/1682333172 shutdown_connections 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.128+0000 7fdd854f2640 1 --2- 172.21.15.66:0/1682333172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80103be0 0x7fdd8010a470 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.128+0000 7fdd854f2640 1 --2- 172.21.15.66:0/1682333172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103220 0x7fdd801036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.128+0000 7fdd854f2640 1 --2- 172.21.15.66:0/1682333172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd80100250 0x7fdd80100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.128+0000 7fdd854f2640 1 -- 172.21.15.66:0/1682333172 >> 172.21.15.66:0/1682333172 conn(0x7fdd800fbb80 msgr2=0x7fdd800fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 -- 172.21.15.66:0/1682333172 shutdown_connections 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 -- 172.21.15.66:0/1682333172 wait complete. 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 Processor -- start 2024-04-21T09:08:13.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 -- start start 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80100250 0x7fdd8019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd80103220 0x7fdd8019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103be0 0x7fdd801a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010daf0 con 0x7fdd80103be0 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010d970 con 0x7fdd80100250 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.129+0000 7fdd854f2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010dc70 con 0x7fdd80103220 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103be0 0x7fdd801a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103be0 0x7fdd801a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46564/0 (socket says 172.21.15.66:46564) 2024-04-21T09:08:13.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 -- 172.21.15.66:0/4203144360 learned_addr learned my addr 172.21.15.66:0/4203144360 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7effd640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80100250 0x7fdd8019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7e7fc640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd80103220 0x7fdd8019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 -- 172.21.15.66:0/4203144360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd80103220 msgr2=0x7fdd8019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd80103220 0x7fdd8019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 -- 172.21.15.66:0/4203144360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80100250 msgr2=0x7fdd8019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80100250 0x7fdd8019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 -- 172.21.15.66:0/4203144360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd801a1f90 con 0x7fdd80103be0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd7f7fe640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103be0 0x7fdd801a1920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd7802c700 tx=0x7fdd78032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.130+0000 7fdd67fff640 1 -- 172.21.15.66:0/4203144360 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78041020 con 0x7fdd80103be0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.131+0000 7fdd67fff640 1 -- 172.21.15.66:0/4203144360 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd780327b0 con 0x7fdd80103be0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.131+0000 7fdd67fff640 1 -- 172.21.15.66:0/4203144360 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78039040 con 0x7fdd80103be0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.131+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd801a2220 con 0x7fdd80103be0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.131+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801a2710 con 0x7fdd80103be0 2024-04-21T09:08:13.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.132+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd801a2970 con 0x7fdd80103be0 2024-04-21T09:08:13.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.134+0000 7fdd67fff640 1 -- 172.21.15.66:0/4203144360 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fdd78044020 con 0x7fdd80103be0 2024-04-21T09:08:13.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.134+0000 7fdd67fff640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdd680a39b0 0x7fdd680a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.135+0000 7fdd7effd640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdd680a39b0 0x7fdd680a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.135+0000 7fdd67fff640 1 -- 172.21.15.66:0/4203144360 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdd780f2780 con 0x7fdd80103be0 2024-04-21T09:08:13.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.135+0000 7fdd7effd640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdd680a39b0 0x7fdd680a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd80068ee0 tx=0x7fdd70005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:13.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.138+0000 7fdd67fff640 1 -- 172.21.15.66:0/4203144360 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd7804b020 con 0x7fdd80103be0 2024-04-21T09:08:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.333+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdd801a2d00 con 0x7fdd80103be0 2024-04-21T09:08:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.334+0000 7fdd67fff640 1 -- 172.21.15.66:0/4203144360 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7fdd780befc0 con 0x7fdd80103be0 2024-04-21T09:08:13.334 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdd680a39b0 msgr2=0x7fdd680a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdd680a39b0 0x7fdd680a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd80068ee0 tx=0x7fdd70005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103be0 msgr2=0x7fdd801a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103be0 0x7fdd801a1920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd7802c700 tx=0x7fdd78032e00 comp rx=0 tx=0).stop 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 shutdown_connections 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdd680a39b0 0x7fdd680a5e70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd80103be0 0x7fdd801a1920 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd80103220 0x7fdd8019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 --2- 172.21.15.66:0/4203144360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd80100250 0x7fdd8019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 >> 172.21.15.66:0/4203144360 conn(0x7fdd800fbb80 msgr2=0x7fdd80101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 shutdown_connections 2024-04-21T09:08:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.336+0000 7fdd854f2640 1 -- 172.21.15.66:0/4203144360 wait complete. 2024-04-21T09:08:13.346 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/88877113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:13.346 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:13 smithi066 ceph-mon[27953]: [21/Apr/2024:09:08:12] ENGINE Bus STARTING 2024-04-21T09:08:13.346 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4118727351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:13.346 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4109806943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:13.346 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:13 smithi066 ceph-mon[27953]: mgrmap e21: x(active, since 1.01945s), standbys: z, y 2024-04-21T09:08:13.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 -- 172.21.15.66:0/3801104856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 msgr2=0x7fb67410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 --2- 172.21.15.66:0/3801104856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb67410a3d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009ca0 tx=0x7fb66c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 -- 172.21.15.66:0/3801104856 shutdown_connections 2024-04-21T09:08:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 --2- 172.21.15.66:0/3801104856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb67410a3d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 --2- 172.21.15.66:0/3801104856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb674103180 0x7fb674103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 --2- 172.21.15.66:0/3801104856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674101f90 0x7fb6741023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 -- 172.21.15.66:0/3801104856 >> 172.21.15.66:0/3801104856 conn(0x7fb6740fd500 msgr2=0x7fb6740ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.453+0000 7fb67a281640 1 -- 172.21.15.66:0/3801104856 shutdown_connections 2024-04-21T09:08:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.454+0000 7fb67a281640 1 -- 172.21.15.66:0/3801104856 wait complete. 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.454+0000 7fb67a281640 1 Processor -- start 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.454+0000 7fb67a281640 1 -- start start 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67a281640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb674101f90 0x7fb67419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67a281640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674103180 0x7fb67419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67a281640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb6741a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67a281640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb67410dad0 con 0x7fb674103b40 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67a281640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb67410d950 con 0x7fb674103180 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67a281640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb67410dc50 con 0x7fb674101f90 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb6741a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb6741a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46584/0 (socket says 172.21.15.66:46584) 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 -- 172.21.15.66:0/3943877073 learned_addr learned my addr 172.21.15.66:0/3943877073 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb673fff640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb674101f90 0x7fb67419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb6737fe640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674103180 0x7fb67419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 -- 172.21.15.66:0/3943877073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb674101f90 msgr2=0x7fb67419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb674101f90 0x7fb67419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 -- 172.21.15.66:0/3943877073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674103180 msgr2=0x7fb67419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674103180 0x7fb67419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 -- 172.21.15.66:0/3943877073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6741a1fe0 con 0x7fb674103b40 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.455+0000 7fb67881e640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb6741a1970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02c700 tx=0x7fb66c0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.456+0000 7fb6717fa640 1 -- 172.21.15.66:0/3943877073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c041020 con 0x7fb674103b40 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.456+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6741a2270 con 0x7fb674103b40 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.456+0000 7fb6717fa640 1 -- 172.21.15.66:0/3943877073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c034ca0 con 0x7fb674103b40 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.456+0000 7fb6717fa640 1 -- 172.21.15.66:0/3943877073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c039040 con 0x7fb674103b40 2024-04-21T09:08:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.456+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6741a2760 con 0x7fb674103b40 2024-04-21T09:08:13.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.457+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6741a29c0 con 0x7fb674103b40 2024-04-21T09:08:13.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.458+0000 7fb6717fa640 1 -- 172.21.15.66:0/3943877073 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fb66c044020 con 0x7fb674103b40 2024-04-21T09:08:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.459+0000 7fb6717fa640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb65c0a39b0 0x7fb65c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.459+0000 7fb673fff640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb65c0a39b0 0x7fb65c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.459+0000 7fb6717fa640 1 -- 172.21.15.66:0/3943877073 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb66c0f2210 con 0x7fb674103b40 2024-04-21T09:08:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.459+0000 7fb673fff640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb65c0a39b0 0x7fb65c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb6640095f0 tx=0x7fb664005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.464+0000 7fb6717fa640 1 -- 172.21.15.66:0/3943877073 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb66c04b020 con 0x7fb674103b40 2024-04-21T09:08:13.554 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/88877113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:13.555 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:13 smithi186 ceph-mon[31505]: [21/Apr/2024:09:08:12] ENGINE Bus STARTING 2024-04-21T09:08:13.555 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4118727351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:13.555 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4109806943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:13.555 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:13 smithi186 ceph-mon[31505]: mgrmap e21: x(active, since 1.01945s), standbys: z, y 2024-04-21T09:08:13.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/88877113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:13.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:13 smithi083 ceph-mon[31413]: [21/Apr/2024:09:08:12] ENGINE Bus STARTING 2024-04-21T09:08:13.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4118727351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:13.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4109806943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:13.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:13 smithi083 ceph-mon[31413]: mgrmap e21: x(active, since 1.01945s), standbys: z, y 2024-04-21T09:08:13.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.654+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb6741a2b90 con 0x7fb674103b40 2024-04-21T09:08:13.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.654+0000 7fb6717fa640 1 -- 172.21.15.66:0/3943877073 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7fb66c0bea30 con 0x7fb674103b40 2024-04-21T09:08:13.655 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb65c0a39b0 msgr2=0x7fb65c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb65c0a39b0 0x7fb65c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb6640095f0 tx=0x7fb664005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 msgr2=0x7fb6741a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb6741a1970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02c700 tx=0x7fb66c0346d0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 shutdown_connections 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb65c0a39b0 0x7fb65c0a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674103b40 0x7fb6741a1970 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674103180 0x7fb67419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 --2- 172.21.15.66:0/3943877073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb674101f90 0x7fb67419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 >> 172.21.15.66:0/3943877073 conn(0x7fb6740fd500 msgr2=0x7fb674074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:13.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 shutdown_connections 2024-04-21T09:08:13.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.657+0000 7fb67a281640 1 -- 172.21.15.66:0/3943877073 wait complete. 2024-04-21T09:08:13.669 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-21T09:08:13.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 -- 172.21.15.66:0/833764350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410101fa0 msgr2=0x7f04101023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 --2- 172.21.15.66:0/833764350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410101fa0 0x7f04101023c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0404009900 tx=0x7f040402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 -- 172.21.15.66:0/833764350 shutdown_connections 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 --2- 172.21.15.66:0/833764350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410103b50 0x7f041010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 --2- 172.21.15.66:0/833764350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0410103190 0x7f0410103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 --2- 172.21.15.66:0/833764350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410101fa0 0x7f04101023c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 -- 172.21.15.66:0/833764350 >> 172.21.15.66:0/833764350 conn(0x7f04100fd530 msgr2=0x7f04100ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 -- 172.21.15.66:0/833764350 shutdown_connections 2024-04-21T09:08:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.772+0000 7f0416e25640 1 -- 172.21.15.66:0/833764350 wait complete. 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 Processor -- start 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 -- start start 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0410101fa0 0x7f0410075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410103190 0x7f0410074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410103b50 0x7f0410074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f041010dae0 con 0x7f0410103190 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f041010d960 con 0x7f0410103b50 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0416e25640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f041010dc60 con 0x7f0410101fa0 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f040ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410103190 0x7f0410074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f040ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410103190 0x7f0410074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46600/0 (socket says 172.21.15.66:46600) 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f040ffff640 1 -- 172.21.15.66:0/2205992275 learned_addr learned my addr 172.21.15.66:0/2205992275 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f04153c2640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410103b50 0x7f0410074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.773+0000 7f0414bc1640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0410101fa0 0x7f0410075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040ffff640 1 -- 172.21.15.66:0/2205992275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0410101fa0 msgr2=0x7f0410075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040ffff640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0410101fa0 0x7f0410075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040ffff640 1 -- 172.21.15.66:0/2205992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410103b50 msgr2=0x7f0410074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040ffff640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410103b50 0x7f0410074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040ffff640 1 -- 172.21.15.66:0/2205992275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0410075110 con 0x7f0410103190 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040ffff640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410103190 0x7f0410074340 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00d770 tx=0x7f03fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040dffb640 1 -- 172.21.15.66:0/2205992275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc017070 con 0x7f0410103190 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040dffb640 1 -- 172.21.15.66:0/2205992275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03fc0054d0 con 0x7f0410103190 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f040dffb640 1 -- 172.21.15.66:0/2205992275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004c30 con 0x7f0410103190 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04101a22c0 con 0x7f0410103190 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.774+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04101a27c0 con 0x7f0410103190 2024-04-21T09:08:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.775+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0410109340 con 0x7f0410103190 2024-04-21T09:08:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.776+0000 7f040dffb640 1 -- 172.21.15.66:0/2205992275 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f03fc004e50 con 0x7f0410103190 2024-04-21T09:08:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.777+0000 7f040dffb640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04000a3a30 0x7f04000a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.777+0000 7f0414bc1640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04000a3a30 0x7f04000a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.777+0000 7f040dffb640 1 -- 172.21.15.66:0/2205992275 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f03fc0cfe40 con 0x7f0410103190 2024-04-21T09:08:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.777+0000 7f0414bc1640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04000a3a30 0x7f04000a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0404005ab0 tx=0x7f0404005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:13.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:13.781+0000 7f040dffb640 1 -- 172.21.15.66:0/2205992275 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03fc09c030 con 0x7f0410103190 2024-04-21T09:08:14.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.023+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f041002d5c0 con 0x7f0410103190 2024-04-21T09:08:14.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.024+0000 7f040dffb640 1 -- 172.21.15.66:0/2205992275 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f03fc0a0330 con 0x7f0410103190 2024-04-21T09:08:14.025 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:14.025 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:08:14.025 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04000a3a30 msgr2=0x7f04000a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04000a3a30 0x7f04000a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0404005ab0 tx=0x7f0404005fd0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410103190 msgr2=0x7f0410074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410103190 0x7f0410074340 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00d770 tx=0x7f03fc00dc30 comp rx=0 tx=0).stop 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 shutdown_connections 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04000a3a30 0x7f04000a5ef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410103b50 0x7f0410074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0410103190 0x7f0410074340 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 --2- 172.21.15.66:0/2205992275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0410101fa0 0x7f0410075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.026+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 >> 172.21.15.66:0/2205992275 conn(0x7f04100fd530 msgr2=0x7f0410106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:14.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.027+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 shutdown_connections 2024-04-21T09:08:14.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.027+0000 7f0416e25640 1 -- 172.21.15.66:0/2205992275 wait complete. 2024-04-21T09:08:14.039 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-21T09:08:14.040 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-21T09:08:14.040 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:08:14.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 -- 172.21.15.66:0/2468944550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 msgr2=0x7fd8ec10a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/2468944550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec10a480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009cd0 tx=0x7fd8e402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 -- 172.21.15.66:0/2468944550 shutdown_connections 2024-04-21T09:08:14.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/2468944550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec10a480 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/2468944550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec103230 0x7fd8ec1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/2468944550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8ec100260 0x7fd8ec100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 -- 172.21.15.66:0/2468944550 >> 172.21.15.66:0/2468944550 conn(0x7fd8ec0fbb90 msgr2=0x7fd8ec0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 -- 172.21.15.66:0/2468944550 shutdown_connections 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.151+0000 7fd8f42ba640 1 -- 172.21.15.66:0/2468944550 wait complete. 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 Processor -- start 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 -- start start 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8ec100260 0x7fd8ec198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec103230 0x7fd8ec198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec10dac0 con 0x7fd8ec103bf0 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec10d940 con 0x7fd8ec100260 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f42ba640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec10dc40 con 0x7fd8ec103230 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f2056640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8ec100260 0x7fd8ec198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f2857640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f2056640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8ec100260 0x7fd8ec198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:46892/0 (socket says 172.21.15.66:46892) 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f2056640 1 -- 172.21.15.66:0/1435514026 learned_addr learned my addr 172.21.15.66:0/1435514026 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f2857640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46628/0 (socket says 172.21.15.66:46628) 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f1855640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec103230 0x7fd8ec198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f2857640 1 -- 172.21.15.66:0/1435514026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec103230 msgr2=0x7fd8ec198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.152+0000 7fd8f2857640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec103230 0x7fd8ec198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8f2857640 1 -- 172.21.15.66:0/1435514026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8ec100260 msgr2=0x7fd8ec198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8f2857640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8ec100260 0x7fd8ec198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8f2857640 1 -- 172.21.15.66:0/1435514026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec19fd20 con 0x7fd8ec103bf0 2024-04-21T09:08:14.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8f2857640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec19f6b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd8e402c630 tx=0x7fd8e40346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4041020 con 0x7fd8ec103bf0 2024-04-21T09:08:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec101020 con 0x7fd8ec103bf0 2024-04-21T09:08:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e40327c0 con 0x7fd8ec103bf0 2024-04-21T09:08:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4039040 con 0x7fd8ec103bf0 2024-04-21T09:08:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.153+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec101560 con 0x7fd8ec103bf0 2024-04-21T09:08:14.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.155+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec109360 con 0x7fd8ec103bf0 2024-04-21T09:08:14.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.157+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fd8e4044020 con 0x7fd8ec103bf0 2024-04-21T09:08:14.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.158+0000 7fd8e37fe640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd8d80a39b0 0x7fd8d80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.158+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd8e40f2570 con 0x7fd8ec103bf0 2024-04-21T09:08:14.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.158+0000 7fd8f2056640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd8d80a39b0 0x7fd8d80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.159+0000 7fd8f2056640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd8d80a39b0 0x7fd8d80a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80058c0 tx=0x7fd8e8005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:14.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.161+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8e404b020 con 0x7fd8ec103bf0 2024-04-21T09:08:14.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.235+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7fd8e4003970 con 0x7fd8ec103bf0 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: [21/Apr/2024:09:08:12] ENGINE Serving on http://172.21.15.66:8765 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: [21/Apr/2024:09:08:12] ENGINE Serving on https://172.21.15.66:7150 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: [21/Apr/2024:09:08:12] ENGINE Bus STARTED 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4203144360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3943877073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2205992275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:14 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.387+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8ec062390 con 0x7fd8d80a39b0 2024-04-21T09:08:14.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.406+0000 7fd8e37fe640 1 -- 172.21.15.66:0/1435514026 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339695 (secure 0 0 0) 0x7fd8ec062390 con 0x7fd8d80a39b0 2024-04-21T09:08:14.410 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:14.412 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:08:14.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd8d80a39b0 msgr2=0x7fd8d80a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd8d80a39b0 0x7fd8d80a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80058c0 tx=0x7fd8e8005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 msgr2=0x7fd8ec19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec19f6b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd8e402c630 tx=0x7fd8e40346d0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 shutdown_connections 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd8d80a39b0 0x7fd8d80a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec103bf0 0x7fd8ec19f6b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec103230 0x7fd8ec198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 --2- 172.21.15.66:0/1435514026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8ec100260 0x7fd8ec198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.409+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 >> 172.21.15.66:0/1435514026 conn(0x7fd8ec0fbb90 msgr2=0x7fd8ec106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.410+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 shutdown_connections 2024-04-21T09:08:14.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.410+0000 7fd8f42ba640 1 -- 172.21.15.66:0/1435514026 wait complete. 2024-04-21T09:08:14.421 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T09:08:14.047546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187546,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":967,"num_read_kb":644,"num_write":470,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357988,"kb_used_data":12944,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583836,"statfs":{"total":1151940427776,"available":1151573848064,"internally_reserved":0,"allocated":13254656,"data_stored":10021305,"data_compressed":571306,"data_compressed_allocated":778240,"data_compressed_original":4407296,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989630"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764225+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:08:12.764225+0000","last_peered":"2024-04-21T09:08:12.764225+0000","last_clean":"2024-04-21T09:08:12.764225+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:08:12.764225+0000","last_undegraded":"2024-04-21T09:08:12.764225+0000","last_fullsized":"2024-04-21T09:08:12.764225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970868+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:07:40.970868+0000","last_peered":"2024-04-21T09:07:40.970868+0000","last_clean":"2024-04-21T09:07:40.970868+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:07:40.970868+0000","last_undegraded":"2024-04-21T09:07:40.970868+0000","last_fullsized":"2024-04-21T09:07:40.970868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970770+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:07:40.970770+0000","last_peered":"2024-04-21T09:07:40.970770+0000","last_clean":"2024-04-21T09:07:40.970770+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:07:40.970770+0000","last_undegraded":"2024-04-21T09:07:40.970770+0000","last_fullsized":"2024-04-21T09:07:40.970770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281097+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:08:12.281097+0000","last_peered":"2024-04-21T09:08:12.281097+0000","last_clean":"2024-04-21T09:08:12.281097+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:08:12.281097+0000","last_undegraded":"2024-04-21T09:08:12.281097+0000","last_fullsized":"2024-04-21T09:08:12.281097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142191+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:08:12.142191+0000","last_peered":"2024-04-21T09:08:12.142191+0000","last_clean":"2024-04-21T09:08:12.142191+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:08:12.142191+0000","last_undegraded":"2024-04-21T09:08:12.142191+0000","last_fullsized":"2024-04-21T09:08:12.142191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034608+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:08:12.034608+0000","last_peered":"2024-04-21T09:08:12.034608+0000","last_clean":"2024-04-21T09:08:12.034608+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:08:12.034608+0000","last_undegraded":"2024-04-21T09:08:12.034608+0000","last_fullsized":"2024-04-21T09:08:12.034608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324820+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:08:12.324820+0000","last_peered":"2024-04-21T09:08:12.324820+0000","last_clean":"2024-04-21T09:08:12.324820+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:08:12.324820+0000","last_undegraded":"2024-04-21T09:08:12.324820+0000","last_fullsized":"2024-04-21T09:08:12.324820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142634+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:08:12.142634+0000","last_peered":"2024-04-21T09:08:12.142634+0000","last_clean":"2024-04-21T09:08:12.142634+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:08:12.142634+0000","last_undegraded":"2024-04-21T09:08:12.142634+0000","last_fullsized":"2024-04-21T09:08:12.142634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325536+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:08:12.325536+0000","last_peered":"2024-04-21T09:08:12.325536+0000","last_clean":"2024-04-21T09:08:12.325536+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:08:12.325536+0000","last_undegraded":"2024-04-21T09:08:12.325536+0000","last_fullsized":"2024-04-21T09:08:12.325536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035030+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:08:12.035030+0000","last_peered":"2024-04-21T09:08:12.035030+0000","last_clean":"2024-04-21T09:08:12.035030+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:08:12.035030+0000","last_undegraded":"2024-04-21T09:08:12.035030+0000","last_fullsized":"2024-04-21T09:08:12.035030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325727+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:08:12.325727+0000","last_peered":"2024-04-21T09:08:12.325727+0000","last_clean":"2024-04-21T09:08:12.325727+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:08:12.325727+0000","last_undegraded":"2024-04-21T09:08:12.325727+0000","last_fullsized":"2024-04-21T09:08:12.325727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281072+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:08:12.281072+0000","last_peered":"2024-04-21T09:08:12.281072+0000","last_clean":"2024-04-21T09:08:12.281072+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:08:12.281072+0000","last_undegraded":"2024-04-21T09:08:12.281072+0000","last_fullsized":"2024-04-21T09:08:12.281072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033565+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:08:12.033565+0000","last_peered":"2024-04-21T09:08:12.033565+0000","last_clean":"2024-04-21T09:08:12.033565+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:08:12.033565+0000","last_undegraded":"2024-04-21T09:08:12.033565+0000","last_fullsized":"2024-04-21T09:08:12.033565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970421+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:07:40.970421+0000","last_peered":"2024-04-21T09:07:40.970421+0000","last_clean":"2024-04-21T09:07:40.970421+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:07:40.970421+0000","last_undegraded":"2024-04-21T09:07:40.970421+0000","last_fullsized":"2024-04-21T09:07:40.970421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970476+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:07:40.970476+0000","last_peered":"2024-04-21T09:07:40.970476+0000","last_clean":"2024-04-21T09:07:40.970476+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:07:40.970476+0000","last_undegraded":"2024-04-21T09:07:40.970476+0000","last_fullsized":"2024-04-21T09:07:40.970476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323842+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:08:12.323842+0000","last_peered":"2024-04-21T09:08:12.323842+0000","last_clean":"2024-04-21T09:08:12.323842+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:08:12.323842+0000","last_undegraded":"2024-04-21T09:08:12.323842+0000","last_fullsized":"2024-04-21T09:08:12.323842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764134+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:08:12.764134+0000","last_peered":"2024-04-21T09:08:12.764134+0000","last_clean":"2024-04-21T09:08:12.764134+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:08:12.764134+0000","last_undegraded":"2024-04-21T09:08:12.764134+0000","last_fullsized":"2024-04-21T09:08:12.764134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140490+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:08:12.140490+0000","last_peered":"2024-04-21T09:08:12.140490+0000","last_clean":"2024-04-21T09:08:12.140490+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:08:12.140490+0000","last_undegraded":"2024-04-21T09:08:12.140490+0000","last_fullsized":"2024-04-21T09:08:12.140490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035166+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:08:12.035166+0000","last_peered":"2024-04-21T09:08:12.035166+0000","last_clean":"2024-04-21T09:08:12.035166+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:08:12.035166+0000","last_undegraded":"2024-04-21T09:08:12.035166+0000","last_fullsized":"2024-04-21T09:08:12.035166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034821+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:08:12.034821+0000","last_peered":"2024-04-21T09:08:12.034821+0000","last_clean":"2024-04-21T09:08:12.034821+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:08:12.034821+0000","last_undegraded":"2024-04-21T09:08:12.034821+0000","last_fullsized":"2024-04-21T09:08:12.034821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140286+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:08:12.140286+0000","last_peered":"2024-04-21T09:08:12.140286+0000","last_clean":"2024-04-21T09:08:12.140286+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:08:12.140286+0000","last_undegraded":"2024-04-21T09:08:12.140286+0000","last_fullsized":"2024-04-21T09:08:12.140286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143771+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:08:12.143771+0000","last_peered":"2024-04-21T09:08:12.143771+0000","last_clean":"2024-04-21T09:08:12.143771+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:08:12.143771+0000","last_undegraded":"2024-04-21T09:08:12.143771+0000","last_fullsized":"2024-04-21T09:08:12.143771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280701+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:08:12.280701+0000","last_peered":"2024-04-21T09:08:12.280701+0000","last_clean":"2024-04-21T09:08:12.280701+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:08:12.280701+0000","last_undegraded":"2024-04-21T09:08:12.280701+0000","last_fullsized":"2024-04-21T09:08:12.280701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.144097+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:08:12.144097+0000","last_peered":"2024-04-21T09:08:12.144097+0000","last_clean":"2024-04-21T09:08:12.144097+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:08:12.144097+0000","last_undegraded":"2024-04-21T09:08:12.144097+0000","last_fullsized":"2024-04-21T09:08:12.144097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970613+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:07:40.970613+0000","last_peered":"2024-04-21T09:07:40.970613+0000","last_clean":"2024-04-21T09:07:40.970613+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:07:40.970613+0000","last_undegraded":"2024-04-21T09:07:40.970613+0000","last_fullsized":"2024-04-21T09:07:40.970613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280878+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:08:12.280878+0000","last_peered":"2024-04-21T09:08:12.280878+0000","last_clean":"2024-04-21T09:08:12.280878+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:08:12.280878+0000","last_undegraded":"2024-04-21T09:08:12.280878+0000","last_fullsized":"2024-04-21T09:08:12.280878+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142469+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:08:12.142469+0000","last_peered":"2024-04-21T09:08:12.142469+0000","last_clean":"2024-04-21T09:08:12.142469+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:08:12.142469+0000","last_undegraded":"2024-04-21T09:08:12.142469+0000","last_fullsized":"2024-04-21T09:08:12.142469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325055+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:08:12.325055+0000","last_peered":"2024-04-21T09:08:12.325055+0000","last_clean":"2024-04-21T09:08:12.325055+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:08:12.325055+0000","last_undegraded":"2024-04-21T09:08:12.325055+0000","last_fullsized":"2024-04-21T09:08:12.325055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280726+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:08:12.280726+0000","last_peered":"2024-04-21T09:08:12.280726+0000","last_clean":"2024-04-21T09:08:12.280726+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:08:12.280726+0000","last_undegraded":"2024-04-21T09:08:12.280726+0000","last_fullsized":"2024-04-21T09:08:12.280726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142104+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:08:12.142104+0000","last_peered":"2024-04-21T09:08:12.142104+0000","last_clean":"2024-04-21T09:08:12.142104+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:08:12.142104+0000","last_undegraded":"2024-04-21T09:08:12.142104+0000","last_fullsized":"2024-04-21T09:08:12.142104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150895+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:08:12.150895+0000","last_peered":"2024-04-21T09:08:12.150895+0000","last_clean":"2024-04-21T09:08:12.150895+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:08:12.150895+0000","last_undegraded":"2024-04-21T09:08:12.150895+0000","last_fullsized":"2024-04-21T09:08:12.150895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034947+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:08:12.034947+0000","last_peered":"2024-04-21T09:08:12.034947+0000","last_clean":"2024-04-21T09:08:12.034947+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:08:12.034947+0000","last_undegraded":"2024-04-21T09:08:12.034947+0000","last_fullsized":"2024-04-21T09:08:12.034947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'6","reported_seq":175,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.314168+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:08:12.314168+0000","last_peered":"2024-04-21T09:08:12.314168+0000","last_clean":"2024-04-21T09:08:12.314168+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:08:12.314168+0000","last_undegraded":"2024-04-21T09:08:12.314168+0000","last_fullsized":"2024-04-21T09:08:12.314168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142179+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:08:12.142179+0000","last_peered":"2024-04-21T09:08:12.142179+0000","last_clean":"2024-04-21T09:08:12.142179+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:08:12.142179+0000","last_undegraded":"2024-04-21T09:08:12.142179+0000","last_fullsized":"2024-04-21T09:08:12.142179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035266+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:08:12.035266+0000","last_peered":"2024-04-21T09:08:12.035266+0000","last_clean":"2024-04-21T09:08:12.035266+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:08:12.035266+0000","last_undegraded":"2024-04-21T09:08:12.035266+0000","last_fullsized":"2024-04-21T09:08:12.035266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034693+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:08:12.034693+0000","last_peered":"2024-04-21T09:08:12.034693+0000","last_clean":"2024-04-21T09:08:12.034693+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:08:12.034693+0000","last_undegraded":"2024-04-21T09:08:12.034693+0000","last_fullsized":"2024-04-21T09:08:12.034693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150724+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:08:12.150724+0000","last_peered":"2024-04-21T09:08:12.150724+0000","last_clean":"2024-04-21T09:08:12.150724+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:08:12.150724+0000","last_undegraded":"2024-04-21T09:08:12.150724+0000","last_fullsized":"2024-04-21T09:08:12.150724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763967+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:08:12.763967+0000","last_peered":"2024-04-21T09:08:12.763967+0000","last_clean":"2024-04-21T09:08:12.763967+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:08:12.763967+0000","last_undegraded":"2024-04-21T09:08:12.763967+0000","last_fullsized":"2024-04-21T09:08:12.763967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033440+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:08:12.033440+0000","last_peered":"2024-04-21T09:08:12.033440+0000","last_clean":"2024-04-21T09:08:12.033440+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:08:12.033440+0000","last_undegraded":"2024-04-21T09:08:12.033440+0000","last_fullsized":"2024-04-21T09:08:12.033440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142773+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:08:12.142773+0000","last_peered":"2024-04-21T09:08:12.142773+0000","last_clean":"2024-04-21T09:08:12.142773+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:08:12.142773+0000","last_undegraded":"2024-04-21T09:08:12.142773+0000","last_fullsized":"2024-04-21T09:08:12.142773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970644+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:07:40.970644+0000","last_peered":"2024-04-21T09:07:40.970644+0000","last_clean":"2024-04-21T09:07:40.970644+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:07:40.970644+0000","last_undegraded":"2024-04-21T09:07:40.970644+0000","last_fullsized":"2024-04-21T09:07:40.970644+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150624+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:08:12.150624+0000","last_peered":"2024-04-21T09:08:12.150624+0000","last_clean":"2024-04-21T09:08:12.150624+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:08:12.150624+0000","last_undegraded":"2024-04-21T09:08:12.150624+0000","last_fullsized":"2024-04-21T09:08:12.150624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139585+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:08:12.139585+0000","last_peered":"2024-04-21T09:08:12.139585+0000","last_clean":"2024-04-21T09:08:12.139585+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:08:12.139585+0000","last_undegraded":"2024-04-21T09:08:12.139585+0000","last_fullsized":"2024-04-21T09:08:12.139585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142906+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:08:12.142906+0000","last_peered":"2024-04-21T09:08:12.142906+0000","last_clean":"2024-04-21T09:08:12.142906+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:08:12.142906+0000","last_undegraded":"2024-04-21T09:08:12.142906+0000","last_fullsized":"2024-04-21T09:08:12.142906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":160,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.328407+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:08:12.328407+0000","last_peered":"2024-04-21T09:08:12.328407+0000","last_clean":"2024-04-21T09:08:12.328407+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:08:12.328407+0000","last_undegraded":"2024-04-21T09:08:12.328407+0000","last_fullsized":"2024-04-21T09:08:12.328407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325164+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:08:12.325164+0000","last_peered":"2024-04-21T09:08:12.325164+0000","last_clean":"2024-04-21T09:08:12.325164+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:08:12.325164+0000","last_undegraded":"2024-04-21T09:08:12.325164+0000","last_fullsized":"2024-04-21T09:08:12.325164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139403+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:08:12.139403+0000","last_peered":"2024-04-21T09:08:12.139403+0000","last_clean":"2024-04-21T09:08:12.139403+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:08:12.139403+0000","last_undegraded":"2024-04-21T09:08:12.139403+0000","last_fullsized":"2024-04-21T09:08:12.139403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325630+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:08:12.325630+0000","last_peered":"2024-04-21T09:08:12.325630+0000","last_clean":"2024-04-21T09:08:12.325630+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:08:12.325630+0000","last_undegraded":"2024-04-21T09:08:12.325630+0000","last_fullsized":"2024-04-21T09:08:12.325630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970868+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:07:40.970868+0000","last_peered":"2024-04-21T09:07:40.970868+0000","last_clean":"2024-04-21T09:07:40.970868+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:07:40.970868+0000","last_undegraded":"2024-04-21T09:07:40.970868+0000","last_fullsized":"2024-04-21T09:07:40.970868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150830+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:08:12.150830+0000","last_peered":"2024-04-21T09:08:12.150830+0000","last_clean":"2024-04-21T09:08:12.150830+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:08:12.150830+0000","last_undegraded":"2024-04-21T09:08:12.150830+0000","last_fullsized":"2024-04-21T09:08:12.150830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325080+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:08:12.325080+0000","last_peered":"2024-04-21T09:08:12.325080+0000","last_clean":"2024-04-21T09:08:12.325080+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:08:12.325080+0000","last_undegraded":"2024-04-21T09:08:12.325080+0000","last_fullsized":"2024-04-21T09:08:12.325080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139822+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:08:12.139822+0000","last_peered":"2024-04-21T09:08:12.139822+0000","last_clean":"2024-04-21T09:08:12.139822+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:08:12.139822+0000","last_undegraded":"2024-04-21T09:08:12.139822+0000","last_fullsized":"2024-04-21T09:08:12.139822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":325,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.330114+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:08:12.330114+0000","last_peered":"2024-04-21T09:08:12.330114+0000","last_clean":"2024-04-21T09:08:12.330114+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:08:12.330114+0000","last_undegraded":"2024-04-21T09:08:12.330114+0000","last_fullsized":"2024-04-21T09:08:12.330114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":191,"num_write":40,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142319+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:08:12.142319+0000","last_peered":"2024-04-21T09:08:12.142319+0000","last_clean":"2024-04-21T09:08:12.142319+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:08:12.142319+0000","last_undegraded":"2024-04-21T09:08:12.142319+0000","last_fullsized":"2024-04-21T09:08:12.142319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.151015+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:08:12.151015+0000","last_peered":"2024-04-21T09:08:12.151015+0000","last_clean":"2024-04-21T09:08:12.151015+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:08:12.151015+0000","last_undegraded":"2024-04-21T09:08:12.151015+0000","last_fullsized":"2024-04-21T09:08:12.151015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035051+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:08:12.035051+0000","last_peered":"2024-04-21T09:08:12.035051+0000","last_clean":"2024-04-21T09:08:12.035051+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:08:12.035051+0000","last_undegraded":"2024-04-21T09:08:12.035051+0000","last_fullsized":"2024-04-21T09:08:12.035051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033721+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:08:12.033721+0000","last_peered":"2024-04-21T09:08:12.033721+0000","last_clean":"2024-04-21T09:08:12.033721+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:08:12.033721+0000","last_undegraded":"2024-04-21T09:08:12.033721+0000","last_fullsized":"2024-04-21T09:08:12.033721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150756+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:08:12.150756+0000","last_peered":"2024-04-21T09:08:12.150756+0000","last_clean":"2024-04-21T09:08:12.150756+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:08:12.150756+0000","last_undegraded":"2024-04-21T09:08:12.150756+0000","last_fullsized":"2024-04-21T09:08:12.150756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325172+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:08:12.325172+0000","last_peered":"2024-04-21T09:08:12.325172+0000","last_clean":"2024-04-21T09:08:12.325172+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:08:12.325172+0000","last_undegraded":"2024-04-21T09:08:12.325172+0000","last_fullsized":"2024-04-21T09:08:12.325172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140181+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:08:12.140181+0000","last_peered":"2024-04-21T09:08:12.140181+0000","last_clean":"2024-04-21T09:08:12.140181+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:08:12.140181+0000","last_undegraded":"2024-04-21T09:08:12.140181+0000","last_fullsized":"2024-04-21T09:08:12.140181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'5","reported_seq":164,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324171+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:08:12.324171+0000","last_peered":"2024-04-21T09:08:12.324171+0000","last_clean":"2024-04-21T09:08:12.324171+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:08:12.324171+0000","last_undegraded":"2024-04-21T09:08:12.324171+0000","last_fullsized":"2024-04-21T09:08:12.324171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324692+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:08:12.324692+0000","last_peered":"2024-04-21T09:08:12.324692+0000","last_clean":"2024-04-21T09:08:12.324692+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:08:12.324692+0000","last_undegraded":"2024-04-21T09:08:12.324692+0000","last_fullsized":"2024-04-21T09:08:12.324692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033848+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:08:12.033848+0000","last_peered":"2024-04-21T09:08:12.033848+0000","last_clean":"2024-04-21T09:08:12.033848+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:08:12.033848+0000","last_undegraded":"2024-04-21T09:08:12.033848+0000","last_fullsized":"2024-04-21T09:08:12.033848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142080+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:08:12.142080+0000","last_peered":"2024-04-21T09:08:12.142080+0000","last_clean":"2024-04-21T09:08:12.142080+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:08:12.142080+0000","last_undegraded":"2024-04-21T09:08:12.142080+0000","last_fullsized":"2024-04-21T09:08:12.142080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150555+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:08:12.150555+0000","last_peered":"2024-04-21T09:08:12.150555+0000","last_clean":"2024-04-21T09:08:12.150555+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:08:12.150555+0000","last_undegraded":"2024-04-21T09:08:12.150555+0000","last_fullsized":"2024-04-21T09:08:12.150555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034523+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:08:12.034523+0000","last_peered":"2024-04-21T09:08:12.034523+0000","last_clean":"2024-04-21T09:08:12.034523+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:08:12.034523+0000","last_undegraded":"2024-04-21T09:08:12.034523+0000","last_fullsized":"2024-04-21T09:08:12.034523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763981+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:08:12.763981+0000","last_peered":"2024-04-21T09:08:12.763981+0000","last_clean":"2024-04-21T09:08:12.763981+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:08:12.763981+0000","last_undegraded":"2024-04-21T09:08:12.763981+0000","last_fullsized":"2024-04-21T09:08:12.763981+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142163+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:08:12.142163+0000","last_peered":"2024-04-21T09:08:12.142163+0000","last_clean":"2024-04-21T09:08:12.142163+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:08:12.142163+0000","last_undegraded":"2024-04-21T09:08:12.142163+0000","last_fullsized":"2024-04-21T09:08:12.142163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139908+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:08:12.139908+0000","last_peered":"2024-04-21T09:08:12.139908+0000","last_clean":"2024-04-21T09:08:12.139908+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:08:12.139908+0000","last_undegraded":"2024-04-21T09:08:12.139908+0000","last_fullsized":"2024-04-21T09:08:12.139908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034862+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:08:12.034862+0000","last_peered":"2024-04-21T09:08:12.034862+0000","last_clean":"2024-04-21T09:08:12.034862+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:08:12.034862+0000","last_undegraded":"2024-04-21T09:08:12.034862+0000","last_fullsized":"2024-04-21T09:08:12.034862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033827+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:08:12.033827+0000","last_peered":"2024-04-21T09:08:12.033827+0000","last_clean":"2024-04-21T09:08:12.033827+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:08:12.033827+0000","last_undegraded":"2024-04-21T09:08:12.033827+0000","last_fullsized":"2024-04-21T09:08:12.033827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325076+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:08:12.325076+0000","last_peered":"2024-04-21T09:08:12.325076+0000","last_clean":"2024-04-21T09:08:12.325076+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:08:12.325076+0000","last_undegraded":"2024-04-21T09:08:12.325076+0000","last_fullsized":"2024-04-21T09:08:12.325076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034855+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:08:12.034855+0000","last_peered":"2024-04-21T09:08:12.034855+0000","last_clean":"2024-04-21T09:08:12.034855+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:08:12.034855+0000","last_undegraded":"2024-04-21T09:08:12.034855+0000","last_fullsized":"2024-04-21T09:08:12.034855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033559+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:08:12.033559+0000","last_peered":"2024-04-21T09:08:12.033559+0000","last_clean":"2024-04-21T09:08:12.033559+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:08:12.033559+0000","last_undegraded":"2024-04-21T09:08:12.033559+0000","last_fullsized":"2024-04-21T09:08:12.033559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150754+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:08:12.150754+0000","last_peered":"2024-04-21T09:08:12.150754+0000","last_clean":"2024-04-21T09:08:12.150754+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:08:12.150754+0000","last_undegraded":"2024-04-21T09:08:12.150754+0000","last_fullsized":"2024-04-21T09:08:12.150754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":161,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033281+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:08:12.033281+0000","last_peered":"2024-04-21T09:08:12.033281+0000","last_clean":"2024-04-21T09:08:12.033281+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:08:12.033281+0000","last_undegraded":"2024-04-21T09:08:12.033281+0000","last_fullsized":"2024-04-21T09:08:12.033281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325490+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:08:12.325490+0000","last_peered":"2024-04-21T09:08:12.325490+0000","last_clean":"2024-04-21T09:08:12.325490+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:08:12.325490+0000","last_undegraded":"2024-04-21T09:08:12.325490+0000","last_fullsized":"2024-04-21T09:08:12.325490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142918+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:08:12.142918+0000","last_peered":"2024-04-21T09:08:12.142918+0000","last_clean":"2024-04-21T09:08:12.142918+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:08:12.142918+0000","last_undegraded":"2024-04-21T09:08:12.142918+0000","last_fullsized":"2024-04-21T09:08:12.142918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280892+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:08:12.280892+0000","last_peered":"2024-04-21T09:08:12.280892+0000","last_clean":"2024-04-21T09:08:12.280892+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:08:12.280892+0000","last_undegraded":"2024-04-21T09:08:12.280892+0000","last_fullsized":"2024-04-21T09:08:12.280892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":369,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324359+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:08:12.324359+0000","last_peered":"2024-04-21T09:08:12.324359+0000","last_clean":"2024-04-21T09:08:12.324359+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:08:12.324359+0000","last_undegraded":"2024-04-21T09:08:12.324359+0000","last_fullsized":"2024-04-21T09:08:12.324359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150742+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:08:12.150742+0000","last_peered":"2024-04-21T09:08:12.150742+0000","last_clean":"2024-04-21T09:08:12.150742+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:08:12.150742+0000","last_undegraded":"2024-04-21T09:08:12.150742+0000","last_fullsized":"2024-04-21T09:08:12.150742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033204+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:08:12.033204+0000","last_peered":"2024-04-21T09:08:12.033204+0000","last_clean":"2024-04-21T09:08:12.033204+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:08:12.033204+0000","last_undegraded":"2024-04-21T09:08:12.033204+0000","last_fullsized":"2024-04-21T09:08:12.033204+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324532+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:08:12.324532+0000","last_peered":"2024-04-21T09:08:12.324532+0000","last_clean":"2024-04-21T09:08:12.324532+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:08:12.324532+0000","last_undegraded":"2024-04-21T09:08:12.324532+0000","last_fullsized":"2024-04-21T09:08:12.324532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763722+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:08:12.763722+0000","last_peered":"2024-04-21T09:08:12.763722+0000","last_clean":"2024-04-21T09:08:12.763722+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:08:12.763722+0000","last_undegraded":"2024-04-21T09:08:12.763722+0000","last_fullsized":"2024-04-21T09:08:12.763722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142352+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:08:12.142352+0000","last_peered":"2024-04-21T09:08:12.142352+0000","last_clean":"2024-04-21T09:08:12.142352+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:08:12.142352+0000","last_undegraded":"2024-04-21T09:08:12.142352+0000","last_fullsized":"2024-04-21T09:08:12.142352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033592+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:08:12.033592+0000","last_peered":"2024-04-21T09:08:12.033592+0000","last_clean":"2024-04-21T09:08:12.033592+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:08:12.033592+0000","last_undegraded":"2024-04-21T09:08:12.033592+0000","last_fullsized":"2024-04-21T09:08:12.033592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142660+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:08:12.142660+0000","last_peered":"2024-04-21T09:08:12.142660+0000","last_clean":"2024-04-21T09:08:12.142660+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:08:12.142660+0000","last_undegraded":"2024-04-21T09:08:12.142660+0000","last_fullsized":"2024-04-21T09:08:12.142660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'45","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.271084+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:08:12.271084+0000","last_peered":"2024-04-21T09:08:12.271084+0000","last_clean":"2024-04-21T09:08:12.271084+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:08:12.271084+0000","last_undegraded":"2024-04-21T09:08:12.271084+0000","last_fullsized":"2024-04-21T09:08:12.271084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143688+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:08:12.143688+0000","last_peered":"2024-04-21T09:08:12.143688+0000","last_clean":"2024-04-21T09:08:12.143688+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:08:12.143688+0000","last_undegraded":"2024-04-21T09:08:12.143688+0000","last_fullsized":"2024-04-21T09:08:12.143688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034657+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:08:12.034657+0000","last_peered":"2024-04-21T09:08:12.034657+0000","last_clean":"2024-04-21T09:08:12.034657+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:08:12.034657+0000","last_undegraded":"2024-04-21T09:08:12.034657+0000","last_fullsized":"2024-04-21T09:08:12.034657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281282+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:08:12.281282+0000","last_peered":"2024-04-21T09:08:12.281282+0000","last_clean":"2024-04-21T09:08:12.281282+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:08:12.281282+0000","last_undegraded":"2024-04-21T09:08:12.281282+0000","last_fullsized":"2024-04-21T09:08:12.281282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970759+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:07:40.970759+0000","last_peered":"2024-04-21T09:07:40.970759+0000","last_clean":"2024-04-21T09:07:40.970759+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:07:40.970759+0000","last_undegraded":"2024-04-21T09:07:40.970759+0000","last_fullsized":"2024-04-21T09:07:40.970759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281130+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:08:12.281130+0000","last_peered":"2024-04-21T09:08:12.281130+0000","last_clean":"2024-04-21T09:08:12.281130+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:08:12.281130+0000","last_undegraded":"2024-04-21T09:08:12.281130+0000","last_fullsized":"2024-04-21T09:08:12.281130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323960+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:08:12.323960+0000","last_peered":"2024-04-21T09:08:12.323960+0000","last_clean":"2024-04-21T09:08:12.323960+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:08:12.323960+0000","last_undegraded":"2024-04-21T09:08:12.323960+0000","last_fullsized":"2024-04-21T09:08:12.323960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.141989+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:08:12.141989+0000","last_peered":"2024-04-21T09:08:12.141989+0000","last_clean":"2024-04-21T09:08:12.141989+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:08:12.141989+0000","last_undegraded":"2024-04-21T09:08:12.141989+0000","last_fullsized":"2024-04-21T09:08:12.141989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140305+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:08:12.140305+0000","last_peered":"2024-04-21T09:08:12.140305+0000","last_clean":"2024-04-21T09:08:12.140305+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:08:12.140305+0000","last_undegraded":"2024-04-21T09:08:12.140305+0000","last_fullsized":"2024-04-21T09:08:12.140305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143035+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:08:12.143035+0000","last_peered":"2024-04-21T09:08:12.143035+0000","last_clean":"2024-04-21T09:08:12.143035+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:08:12.143035+0000","last_undegraded":"2024-04-21T09:08:12.143035+0000","last_fullsized":"2024-04-21T09:08:12.143035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763571+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:08:12.763571+0000","last_peered":"2024-04-21T09:08:12.763571+0000","last_clean":"2024-04-21T09:08:12.763571+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:08:12.763571+0000","last_undegraded":"2024-04-21T09:08:12.763571+0000","last_fullsized":"2024-04-21T09:08:12.763571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281011+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:08:12.281011+0000","last_peered":"2024-04-21T09:08:12.281011+0000","last_clean":"2024-04-21T09:08:12.281011+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:08:12.281011+0000","last_undegraded":"2024-04-21T09:08:12.281011+0000","last_fullsized":"2024-04-21T09:08:12.281011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034866+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:08:12.034866+0000","last_peered":"2024-04-21T09:08:12.034866+0000","last_clean":"2024-04-21T09:08:12.034866+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:08:12.034866+0000","last_undegraded":"2024-04-21T09:08:12.034866+0000","last_fullsized":"2024-04-21T09:08:12.034866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143614+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:08:12.143614+0000","last_peered":"2024-04-21T09:08:12.143614+0000","last_clean":"2024-04-21T09:08:12.143614+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:08:12.143614+0000","last_undegraded":"2024-04-21T09:08:12.143614+0000","last_fullsized":"2024-04-21T09:08:12.143614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763696+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:08:12.763696+0000","last_peered":"2024-04-21T09:08:12.763696+0000","last_clean":"2024-04-21T09:08:12.763696+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:08:12.763696+0000","last_undegraded":"2024-04-21T09:08:12.763696+0000","last_fullsized":"2024-04-21T09:08:12.763696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325194+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:08:12.325194+0000","last_peered":"2024-04-21T09:08:12.325194+0000","last_clean":"2024-04-21T09:08:12.325194+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:08:12.325194+0000","last_undegraded":"2024-04-21T09:08:12.325194+0000","last_fullsized":"2024-04-21T09:08:12.325194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324153+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:08:12.324153+0000","last_peered":"2024-04-21T09:08:12.324153+0000","last_clean":"2024-04-21T09:08:12.324153+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:08:12.324153+0000","last_undegraded":"2024-04-21T09:08:12.324153+0000","last_fullsized":"2024-04-21T09:08:12.324153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324834+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:08:12.324834+0000","last_peered":"2024-04-21T09:08:12.324834+0000","last_clean":"2024-04-21T09:08:12.324834+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:08:12.324834+0000","last_undegraded":"2024-04-21T09:08:12.324834+0000","last_fullsized":"2024-04-21T09:08:12.324834+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325898+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:08:12.325898+0000","last_peered":"2024-04-21T09:08:12.325898+0000","last_clean":"2024-04-21T09:08:12.325898+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:08:12.325898+0000","last_undegraded":"2024-04-21T09:08:12.325898+0000","last_fullsized":"2024-04-21T09:08:12.325898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970437+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:07:40.970437+0000","last_peered":"2024-04-21T09:07:40.970437+0000","last_clean":"2024-04-21T09:07:40.970437+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:07:40.970437+0000","last_undegraded":"2024-04-21T09:07:40.970437+0000","last_fullsized":"2024-04-21T09:07:40.970437+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"96'39","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.264983+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:08:12.264983+0000","last_peered":"2024-04-21T09:08:12.264983+0000","last_clean":"2024-04-21T09:08:12.264983+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:08:12.264983+0000","last_undegraded":"2024-04-21T09:08:12.264983+0000","last_fullsized":"2024-04-21T09:08:12.264983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324272+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:08:12.324272+0000","last_peered":"2024-04-21T09:08:12.324272+0000","last_clean":"2024-04-21T09:08:12.324272+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:08:12.324272+0000","last_undegraded":"2024-04-21T09:08:12.324272+0000","last_fullsized":"2024-04-21T09:08:12.324272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150326+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:08:12.150326+0000","last_peered":"2024-04-21T09:08:12.150326+0000","last_clean":"2024-04-21T09:08:12.150326+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:08:12.150326+0000","last_undegraded":"2024-04-21T09:08:12.150326+0000","last_fullsized":"2024-04-21T09:08:12.150326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033814+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:08:12.033814+0000","last_peered":"2024-04-21T09:08:12.033814+0000","last_clean":"2024-04-21T09:08:12.033814+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:08:12.033814+0000","last_undegraded":"2024-04-21T09:08:12.033814+0000","last_fullsized":"2024-04-21T09:08:12.033814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325958+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:08:12.325958+0000","last_peered":"2024-04-21T09:08:12.325958+0000","last_clean":"2024-04-21T09:08:12.325958+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:08:12.325958+0000","last_undegraded":"2024-04-21T09:08:12.325958+0000","last_fullsized":"2024-04-21T09:08:12.325958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763645+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:08:12.763645+0000","last_peered":"2024-04-21T09:08:12.763645+0000","last_clean":"2024-04-21T09:08:12.763645+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:08:12.763645+0000","last_undegraded":"2024-04-21T09:08:12.763645+0000","last_fullsized":"2024-04-21T09:08:12.763645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034801+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:08:12.034801+0000","last_peered":"2024-04-21T09:08:12.034801+0000","last_clean":"2024-04-21T09:08:12.034801+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:08:12.034801+0000","last_undegraded":"2024-04-21T09:08:12.034801+0000","last_fullsized":"2024-04-21T09:08:12.034801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034715+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:08:12.034715+0000","last_peered":"2024-04-21T09:08:12.034715+0000","last_clean":"2024-04-21T09:08:12.034715+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:08:12.034715+0000","last_undegraded":"2024-04-21T09:08:12.034715+0000","last_fullsized":"2024-04-21T09:08:12.034715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970791+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:07:40.970791+0000","last_peered":"2024-04-21T09:07:40.970791+0000","last_clean":"2024-04-21T09:07:40.970791+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:07:40.970791+0000","last_undegraded":"2024-04-21T09:07:40.970791+0000","last_fullsized":"2024-04-21T09:07:40.970791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033916+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:08:12.033916+0000","last_peered":"2024-04-21T09:08:12.033916+0000","last_clean":"2024-04-21T09:08:12.033916+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:08:12.033916+0000","last_undegraded":"2024-04-21T09:08:12.033916+0000","last_fullsized":"2024-04-21T09:08:12.033916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763506+0000","last_change":"2024-04-21T09:07:40.649825+0000","last_active":"2024-04-21T09:08:12.763506+0000","last_peered":"2024-04-21T09:08:12.763506+0000","last_clean":"2024-04-21T09:08:12.763506+0000","last_became_active":"2024-04-21T09:07:40.649264+0000","last_became_peered":"2024-04-21T09:07:40.649264+0000","last_unstale":"2024-04-21T09:08:12.763506+0000","last_undegraded":"2024-04-21T09:08:12.763506+0000","last_fullsized":"2024-04-21T09:08:12.763506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:38.142948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035249+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:08:12.035249+0000","last_peered":"2024-04-21T09:08:12.035249+0000","last_clean":"2024-04-21T09:08:12.035249+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:08:12.035249+0000","last_undegraded":"2024-04-21T09:08:12.035249+0000","last_fullsized":"2024-04-21T09:08:12.035249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143896+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:08:12.143896+0000","last_peered":"2024-04-21T09:08:12.143896+0000","last_clean":"2024-04-21T09:08:12.143896+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:08:12.143896+0000","last_undegraded":"2024-04-21T09:08:12.143896+0000","last_fullsized":"2024-04-21T09:08:12.143896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150636+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:08:12.150636+0000","last_peered":"2024-04-21T09:08:12.150636+0000","last_clean":"2024-04-21T09:08:12.150636+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:08:12.150636+0000","last_undegraded":"2024-04-21T09:08:12.150636+0000","last_fullsized":"2024-04-21T09:08:12.150636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143204+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:08:12.143204+0000","last_peered":"2024-04-21T09:08:12.143204+0000","last_clean":"2024-04-21T09:08:12.143204+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:08:12.143204+0000","last_undegraded":"2024-04-21T09:08:12.143204+0000","last_fullsized":"2024-04-21T09:08:12.143204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324412+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:08:12.324412+0000","last_peered":"2024-04-21T09:08:12.324412+0000","last_clean":"2024-04-21T09:08:12.324412+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:08:12.324412+0000","last_undegraded":"2024-04-21T09:08:12.324412+0000","last_fullsized":"2024-04-21T09:08:12.324412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033628+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:08:12.033628+0000","last_peered":"2024-04-21T09:08:12.033628+0000","last_clean":"2024-04-21T09:08:12.033628+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:08:12.033628+0000","last_undegraded":"2024-04-21T09:08:12.033628+0000","last_fullsized":"2024-04-21T09:08:12.033628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763507+0000","last_change":"2024-04-21T09:07:40.650222+0000","last_active":"2024-04-21T09:08:12.763507+0000","last_peered":"2024-04-21T09:08:12.763507+0000","last_clean":"2024-04-21T09:08:12.763507+0000","last_became_active":"2024-04-21T09:07:40.649595+0000","last_became_peered":"2024-04-21T09:07:40.649595+0000","last_unstale":"2024-04-21T09:08:12.763507+0000","last_undegraded":"2024-04-21T09:08:12.763507+0000","last_fullsized":"2024-04-21T09:08:12.763507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:25.006017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107013,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596944,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":831,"num_read_kb":525,"num_write":287,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1790832,"data_compressed":80799,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":532061,"data_compressed":29477,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1220608,"data_stored":1328509,"data_compressed":75912,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":939453,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":935255,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1118208,"data_stored":1138669,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":537317,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":605237,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":528839,"data_compressed":29477,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":604801,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1062327,"data_compressed":57756,"data_compressed_allocated":77824,"data_compressed_original":606208,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888142,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":543679,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":1265158,"data_compressed":74281,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68010,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67574,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133467,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":201305,"data_compressed":7787,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:08:14.427 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:08:14.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 -- 172.21.15.66:0/2096786482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d8101b60 msgr2=0x7f32d8101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 --2- 172.21.15.66:0/2096786482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d8101b60 0x7f32d8101fe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009ca0 tx=0x7f32d002c180 comp rx=0 tx=0).stop 2024-04-21T09:08:14.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 -- 172.21.15.66:0/2096786482 shutdown_connections 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 --2- 172.21.15.66:0/2096786482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d8101b60 0x7f32d8101fe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 --2- 172.21.15.66:0/2096786482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d81011c0 0x7f32d8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 --2- 172.21.15.66:0/2096786482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d8100860 0x7f32d8100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 -- 172.21.15.66:0/2096786482 >> 172.21.15.66:0/2096786482 conn(0x7f32d80fc3e0 msgr2=0x7f32d80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 -- 172.21.15.66:0/2096786482 shutdown_connections 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.565+0000 7f32dda6d640 1 -- 172.21.15.66:0/2096786482 wait complete. 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 Processor -- start 2024-04-21T09:08:14.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 -- start start 2024-04-21T09:08:14.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d8100860 0x7f32d819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d81011c0 0x7f32d819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8101b60 0x7f32d81a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810fcb0 con 0x7f32d81011c0 2024-04-21T09:08:14.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810fb30 con 0x7f32d8101b60 2024-04-21T09:08:14.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32dda6d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810fe30 con 0x7f32d8100860 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.566+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d81011c0 0x7f32d819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d81011c0 0x7f32d819a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46658/0 (socket says 172.21.15.66:46658) 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 -- 172.21.15.66:0/1706368162 learned_addr learned my addr 172.21.15.66:0/1706368162 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d77fe640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d8100860 0x7f32d819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d7fff640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8101b60 0x7f32d81a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 -- 172.21.15.66:0/1706368162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d8100860 msgr2=0x7f32d819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d8100860 0x7f32d819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 -- 172.21.15.66:0/1706368162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8101b60 msgr2=0x7f32d81a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8101b60 0x7f32d81a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 -- 172.21.15.66:0/1706368162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d81a1f30 con 0x7f32d81011c0 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d6ffd640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d81011c0 0x7f32d819a850 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00ea60 tx=0x7f32cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d4ff9640 1 -- 172.21.15.66:0/1706368162 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00cde0 con 0x7f32d81011c0 2024-04-21T09:08:14.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d81a2220 con 0x7f32d81011c0 2024-04-21T09:08:14.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d4ff9640 1 -- 172.21.15.66:0/1706368162 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32cc0035f0 con 0x7f32d81011c0 2024-04-21T09:08:14.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d81a27c0 con 0x7f32d81011c0 2024-04-21T09:08:14.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.567+0000 7f32d4ff9640 1 -- 172.21.15.66:0/1706368162 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc005220 con 0x7f32d81011c0 2024-04-21T09:08:14.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.568+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d810b7a0 con 0x7f32d81011c0 2024-04-21T09:08:14.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.570+0000 7f32d4ff9640 1 -- 172.21.15.66:0/1706368162 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f32cc0037a0 con 0x7f32d81011c0 2024-04-21T09:08:14.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.570+0000 7f32d4ff9640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f32c00a3d20 0x7f32c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.570+0000 7f32d77fe640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f32c00a3d20 0x7f32c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.570+0000 7f32d4ff9640 1 -- 172.21.15.66:0/1706368162 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f32cc0d11d0 con 0x7f32d81011c0 2024-04-21T09:08:14.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.571+0000 7f32d77fe640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f32c00a3d20 0x7f32c00a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009bd0 tx=0x7f32c8005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:14.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.578+0000 7f32d4ff9640 1 -- 172.21.15.66:0/1706368162 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32cc09d500 con 0x7f32d81011c0 2024-04-21T09:08:14.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: [21/Apr/2024:09:08:12] ENGINE Serving on http://172.21.15.66:8765 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: [21/Apr/2024:09:08:12] ENGINE Serving on https://172.21.15.66:7150 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: [21/Apr/2024:09:08:12] ENGINE Bus STARTED 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4203144360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3943877073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2205992275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:14 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: [21/Apr/2024:09:08:12] ENGINE Serving on http://172.21.15.66:8765 2024-04-21T09:08:14.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: [21/Apr/2024:09:08:12] ENGINE Serving on https://172.21.15.66:7150 2024-04-21T09:08:14.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: [21/Apr/2024:09:08:12] ENGINE Bus STARTED 2024-04-21T09:08:14.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4203144360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3943877073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2205992275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:14 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:14.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.763+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32d8062390 con 0x7f32c00a3d20 2024-04-21T09:08:14.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.773+0000 7f32d4ff9640 1 -- 172.21.15.66:0/1706368162 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339695 (secure 0 0 0) 0x7f32d8062390 con 0x7f32c00a3d20 2024-04-21T09:08:14.777 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:14.779 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f32c00a3d20 msgr2=0x7f32c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f32c00a3d20 0x7f32c00a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009bd0 tx=0x7f32c8005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d81011c0 msgr2=0x7f32d819a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d81011c0 0x7f32d819a850 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00ea60 tx=0x7f32cc00ef20 comp rx=0 tx=0).stop 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 shutdown_connections 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f32c00a3d20 0x7f32c00a61e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8101b60 0x7f32d81a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32d81011c0 0x7f32d819a850 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 --2- 172.21.15.66:0/1706368162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d8100860 0x7f32d819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 >> 172.21.15.66:0/1706368162 conn(0x7f32d80fc3e0 msgr2=0x7f32d80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:14.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 shutdown_connections 2024-04-21T09:08:14.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.777+0000 7f32dda6d640 1 -- 172.21.15.66:0/1706368162 wait complete. 2024-04-21T09:08:14.789 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T09:08:14.047546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187546,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":967,"num_read_kb":644,"num_write":470,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357988,"kb_used_data":12944,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583836,"statfs":{"total":1151940427776,"available":1151573848064,"internally_reserved":0,"allocated":13254656,"data_stored":10021305,"data_compressed":571306,"data_compressed_allocated":778240,"data_compressed_original":4407296,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989630"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764225+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:08:12.764225+0000","last_peered":"2024-04-21T09:08:12.764225+0000","last_clean":"2024-04-21T09:08:12.764225+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:08:12.764225+0000","last_undegraded":"2024-04-21T09:08:12.764225+0000","last_fullsized":"2024-04-21T09:08:12.764225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970868+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:07:40.970868+0000","last_peered":"2024-04-21T09:07:40.970868+0000","last_clean":"2024-04-21T09:07:40.970868+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:07:40.970868+0000","last_undegraded":"2024-04-21T09:07:40.970868+0000","last_fullsized":"2024-04-21T09:07:40.970868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970770+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:07:40.970770+0000","last_peered":"2024-04-21T09:07:40.970770+0000","last_clean":"2024-04-21T09:07:40.970770+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:07:40.970770+0000","last_undegraded":"2024-04-21T09:07:40.970770+0000","last_fullsized":"2024-04-21T09:07:40.970770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281097+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:08:12.281097+0000","last_peered":"2024-04-21T09:08:12.281097+0000","last_clean":"2024-04-21T09:08:12.281097+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:08:12.281097+0000","last_undegraded":"2024-04-21T09:08:12.281097+0000","last_fullsized":"2024-04-21T09:08:12.281097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142191+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:08:12.142191+0000","last_peered":"2024-04-21T09:08:12.142191+0000","last_clean":"2024-04-21T09:08:12.142191+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:08:12.142191+0000","last_undegraded":"2024-04-21T09:08:12.142191+0000","last_fullsized":"2024-04-21T09:08:12.142191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034608+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:08:12.034608+0000","last_peered":"2024-04-21T09:08:12.034608+0000","last_clean":"2024-04-21T09:08:12.034608+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:08:12.034608+0000","last_undegraded":"2024-04-21T09:08:12.034608+0000","last_fullsized":"2024-04-21T09:08:12.034608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324820+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:08:12.324820+0000","last_peered":"2024-04-21T09:08:12.324820+0000","last_clean":"2024-04-21T09:08:12.324820+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:08:12.324820+0000","last_undegraded":"2024-04-21T09:08:12.324820+0000","last_fullsized":"2024-04-21T09:08:12.324820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142634+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:08:12.142634+0000","last_peered":"2024-04-21T09:08:12.142634+0000","last_clean":"2024-04-21T09:08:12.142634+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:08:12.142634+0000","last_undegraded":"2024-04-21T09:08:12.142634+0000","last_fullsized":"2024-04-21T09:08:12.142634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325536+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:08:12.325536+0000","last_peered":"2024-04-21T09:08:12.325536+0000","last_clean":"2024-04-21T09:08:12.325536+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:08:12.325536+0000","last_undegraded":"2024-04-21T09:08:12.325536+0000","last_fullsized":"2024-04-21T09:08:12.325536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035030+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:08:12.035030+0000","last_peered":"2024-04-21T09:08:12.035030+0000","last_clean":"2024-04-21T09:08:12.035030+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:08:12.035030+0000","last_undegraded":"2024-04-21T09:08:12.035030+0000","last_fullsized":"2024-04-21T09:08:12.035030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325727+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:08:12.325727+0000","last_peered":"2024-04-21T09:08:12.325727+0000","last_clean":"2024-04-21T09:08:12.325727+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:08:12.325727+0000","last_undegraded":"2024-04-21T09:08:12.325727+0000","last_fullsized":"2024-04-21T09:08:12.325727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281072+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:08:12.281072+0000","last_peered":"2024-04-21T09:08:12.281072+0000","last_clean":"2024-04-21T09:08:12.281072+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:08:12.281072+0000","last_undegraded":"2024-04-21T09:08:12.281072+0000","last_fullsized":"2024-04-21T09:08:12.281072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033565+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:08:12.033565+0000","last_peered":"2024-04-21T09:08:12.033565+0000","last_clean":"2024-04-21T09:08:12.033565+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:08:12.033565+0000","last_undegraded":"2024-04-21T09:08:12.033565+0000","last_fullsized":"2024-04-21T09:08:12.033565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970421+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:07:40.970421+0000","last_peered":"2024-04-21T09:07:40.970421+0000","last_clean":"2024-04-21T09:07:40.970421+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:07:40.970421+0000","last_undegraded":"2024-04-21T09:07:40.970421+0000","last_fullsized":"2024-04-21T09:07:40.970421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970476+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:07:40.970476+0000","last_peered":"2024-04-21T09:07:40.970476+0000","last_clean":"2024-04-21T09:07:40.970476+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:07:40.970476+0000","last_undegraded":"2024-04-21T09:07:40.970476+0000","last_fullsized":"2024-04-21T09:07:40.970476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323842+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:08:12.323842+0000","last_peered":"2024-04-21T09:08:12.323842+0000","last_clean":"2024-04-21T09:08:12.323842+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:08:12.323842+0000","last_undegraded":"2024-04-21T09:08:12.323842+0000","last_fullsized":"2024-04-21T09:08:12.323842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764134+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:08:12.764134+0000","last_peered":"2024-04-21T09:08:12.764134+0000","last_clean":"2024-04-21T09:08:12.764134+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:08:12.764134+0000","last_undegraded":"2024-04-21T09:08:12.764134+0000","last_fullsized":"2024-04-21T09:08:12.764134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140490+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:08:12.140490+0000","last_peered":"2024-04-21T09:08:12.140490+0000","last_clean":"2024-04-21T09:08:12.140490+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:08:12.140490+0000","last_undegraded":"2024-04-21T09:08:12.140490+0000","last_fullsized":"2024-04-21T09:08:12.140490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035166+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:08:12.035166+0000","last_peered":"2024-04-21T09:08:12.035166+0000","last_clean":"2024-04-21T09:08:12.035166+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:08:12.035166+0000","last_undegraded":"2024-04-21T09:08:12.035166+0000","last_fullsized":"2024-04-21T09:08:12.035166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034821+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:08:12.034821+0000","last_peered":"2024-04-21T09:08:12.034821+0000","last_clean":"2024-04-21T09:08:12.034821+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:08:12.034821+0000","last_undegraded":"2024-04-21T09:08:12.034821+0000","last_fullsized":"2024-04-21T09:08:12.034821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140286+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:08:12.140286+0000","last_peered":"2024-04-21T09:08:12.140286+0000","last_clean":"2024-04-21T09:08:12.140286+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:08:12.140286+0000","last_undegraded":"2024-04-21T09:08:12.140286+0000","last_fullsized":"2024-04-21T09:08:12.140286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143771+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:08:12.143771+0000","last_peered":"2024-04-21T09:08:12.143771+0000","last_clean":"2024-04-21T09:08:12.143771+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:08:12.143771+0000","last_undegraded":"2024-04-21T09:08:12.143771+0000","last_fullsized":"2024-04-21T09:08:12.143771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280701+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:08:12.280701+0000","last_peered":"2024-04-21T09:08:12.280701+0000","last_clean":"2024-04-21T09:08:12.280701+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:08:12.280701+0000","last_undegraded":"2024-04-21T09:08:12.280701+0000","last_fullsized":"2024-04-21T09:08:12.280701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.144097+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:08:12.144097+0000","last_peered":"2024-04-21T09:08:12.144097+0000","last_clean":"2024-04-21T09:08:12.144097+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:08:12.144097+0000","last_undegraded":"2024-04-21T09:08:12.144097+0000","last_fullsized":"2024-04-21T09:08:12.144097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970613+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:07:40.970613+0000","last_peered":"2024-04-21T09:07:40.970613+0000","last_clean":"2024-04-21T09:07:40.970613+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:07:40.970613+0000","last_undegraded":"2024-04-21T09:07:40.970613+0000","last_fullsized":"2024-04-21T09:07:40.970613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280878+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:08:12.280878+0000","last_peered":"2024-04-21T09:08:12.280878+0000","last_clean":"2024-04-21T09:08:12.280878+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:08:12.280878+0000","last_undegraded":"2024-04-21T09:08:12.280878+0000","last_fullsized":"2024-04-21T09:08:12.280878+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142469+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:08:12.142469+0000","last_peered":"2024-04-21T09:08:12.142469+0000","last_clean":"2024-04-21T09:08:12.142469+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:08:12.142469+0000","last_undegraded":"2024-04-21T09:08:12.142469+0000","last_fullsized":"2024-04-21T09:08:12.142469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325055+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:08:12.325055+0000","last_peered":"2024-04-21T09:08:12.325055+0000","last_clean":"2024-04-21T09:08:12.325055+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:08:12.325055+0000","last_undegraded":"2024-04-21T09:08:12.325055+0000","last_fullsized":"2024-04-21T09:08:12.325055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280726+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:08:12.280726+0000","last_peered":"2024-04-21T09:08:12.280726+0000","last_clean":"2024-04-21T09:08:12.280726+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:08:12.280726+0000","last_undegraded":"2024-04-21T09:08:12.280726+0000","last_fullsized":"2024-04-21T09:08:12.280726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142104+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:08:12.142104+0000","last_peered":"2024-04-21T09:08:12.142104+0000","last_clean":"2024-04-21T09:08:12.142104+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:08:12.142104+0000","last_undegraded":"2024-04-21T09:08:12.142104+0000","last_fullsized":"2024-04-21T09:08:12.142104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150895+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:08:12.150895+0000","last_peered":"2024-04-21T09:08:12.150895+0000","last_clean":"2024-04-21T09:08:12.150895+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:08:12.150895+0000","last_undegraded":"2024-04-21T09:08:12.150895+0000","last_fullsized":"2024-04-21T09:08:12.150895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034947+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:08:12.034947+0000","last_peered":"2024-04-21T09:08:12.034947+0000","last_clean":"2024-04-21T09:08:12.034947+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:08:12.034947+0000","last_undegraded":"2024-04-21T09:08:12.034947+0000","last_fullsized":"2024-04-21T09:08:12.034947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'6","reported_seq":175,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.314168+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:08:12.314168+0000","last_peered":"2024-04-21T09:08:12.314168+0000","last_clean":"2024-04-21T09:08:12.314168+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:08:12.314168+0000","last_undegraded":"2024-04-21T09:08:12.314168+0000","last_fullsized":"2024-04-21T09:08:12.314168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142179+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:08:12.142179+0000","last_peered":"2024-04-21T09:08:12.142179+0000","last_clean":"2024-04-21T09:08:12.142179+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:08:12.142179+0000","last_undegraded":"2024-04-21T09:08:12.142179+0000","last_fullsized":"2024-04-21T09:08:12.142179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035266+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:08:12.035266+0000","last_peered":"2024-04-21T09:08:12.035266+0000","last_clean":"2024-04-21T09:08:12.035266+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:08:12.035266+0000","last_undegraded":"2024-04-21T09:08:12.035266+0000","last_fullsized":"2024-04-21T09:08:12.035266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034693+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:08:12.034693+0000","last_peered":"2024-04-21T09:08:12.034693+0000","last_clean":"2024-04-21T09:08:12.034693+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:08:12.034693+0000","last_undegraded":"2024-04-21T09:08:12.034693+0000","last_fullsized":"2024-04-21T09:08:12.034693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150724+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:08:12.150724+0000","last_peered":"2024-04-21T09:08:12.150724+0000","last_clean":"2024-04-21T09:08:12.150724+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:08:12.150724+0000","last_undegraded":"2024-04-21T09:08:12.150724+0000","last_fullsized":"2024-04-21T09:08:12.150724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763967+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:08:12.763967+0000","last_peered":"2024-04-21T09:08:12.763967+0000","last_clean":"2024-04-21T09:08:12.763967+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:08:12.763967+0000","last_undegraded":"2024-04-21T09:08:12.763967+0000","last_fullsized":"2024-04-21T09:08:12.763967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033440+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:08:12.033440+0000","last_peered":"2024-04-21T09:08:12.033440+0000","last_clean":"2024-04-21T09:08:12.033440+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:08:12.033440+0000","last_undegraded":"2024-04-21T09:08:12.033440+0000","last_fullsized":"2024-04-21T09:08:12.033440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142773+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:08:12.142773+0000","last_peered":"2024-04-21T09:08:12.142773+0000","last_clean":"2024-04-21T09:08:12.142773+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:08:12.142773+0000","last_undegraded":"2024-04-21T09:08:12.142773+0000","last_fullsized":"2024-04-21T09:08:12.142773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970644+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:07:40.970644+0000","last_peered":"2024-04-21T09:07:40.970644+0000","last_clean":"2024-04-21T09:07:40.970644+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:07:40.970644+0000","last_undegraded":"2024-04-21T09:07:40.970644+0000","last_fullsized":"2024-04-21T09:07:40.970644+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150624+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:08:12.150624+0000","last_peered":"2024-04-21T09:08:12.150624+0000","last_clean":"2024-04-21T09:08:12.150624+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:08:12.150624+0000","last_undegraded":"2024-04-21T09:08:12.150624+0000","last_fullsized":"2024-04-21T09:08:12.150624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139585+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:08:12.139585+0000","last_peered":"2024-04-21T09:08:12.139585+0000","last_clean":"2024-04-21T09:08:12.139585+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:08:12.139585+0000","last_undegraded":"2024-04-21T09:08:12.139585+0000","last_fullsized":"2024-04-21T09:08:12.139585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142906+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:08:12.142906+0000","last_peered":"2024-04-21T09:08:12.142906+0000","last_clean":"2024-04-21T09:08:12.142906+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:08:12.142906+0000","last_undegraded":"2024-04-21T09:08:12.142906+0000","last_fullsized":"2024-04-21T09:08:12.142906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":160,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.328407+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:08:12.328407+0000","last_peered":"2024-04-21T09:08:12.328407+0000","last_clean":"2024-04-21T09:08:12.328407+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:08:12.328407+0000","last_undegraded":"2024-04-21T09:08:12.328407+0000","last_fullsized":"2024-04-21T09:08:12.328407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325164+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:08:12.325164+0000","last_peered":"2024-04-21T09:08:12.325164+0000","last_clean":"2024-04-21T09:08:12.325164+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:08:12.325164+0000","last_undegraded":"2024-04-21T09:08:12.325164+0000","last_fullsized":"2024-04-21T09:08:12.325164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139403+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:08:12.139403+0000","last_peered":"2024-04-21T09:08:12.139403+0000","last_clean":"2024-04-21T09:08:12.139403+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:08:12.139403+0000","last_undegraded":"2024-04-21T09:08:12.139403+0000","last_fullsized":"2024-04-21T09:08:12.139403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325630+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:08:12.325630+0000","last_peered":"2024-04-21T09:08:12.325630+0000","last_clean":"2024-04-21T09:08:12.325630+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:08:12.325630+0000","last_undegraded":"2024-04-21T09:08:12.325630+0000","last_fullsized":"2024-04-21T09:08:12.325630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970868+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:07:40.970868+0000","last_peered":"2024-04-21T09:07:40.970868+0000","last_clean":"2024-04-21T09:07:40.970868+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:07:40.970868+0000","last_undegraded":"2024-04-21T09:07:40.970868+0000","last_fullsized":"2024-04-21T09:07:40.970868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150830+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:08:12.150830+0000","last_peered":"2024-04-21T09:08:12.150830+0000","last_clean":"2024-04-21T09:08:12.150830+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:08:12.150830+0000","last_undegraded":"2024-04-21T09:08:12.150830+0000","last_fullsized":"2024-04-21T09:08:12.150830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325080+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:08:12.325080+0000","last_peered":"2024-04-21T09:08:12.325080+0000","last_clean":"2024-04-21T09:08:12.325080+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:08:12.325080+0000","last_undegraded":"2024-04-21T09:08:12.325080+0000","last_fullsized":"2024-04-21T09:08:12.325080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139822+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:08:12.139822+0000","last_peered":"2024-04-21T09:08:12.139822+0000","last_clean":"2024-04-21T09:08:12.139822+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:08:12.139822+0000","last_undegraded":"2024-04-21T09:08:12.139822+0000","last_fullsized":"2024-04-21T09:08:12.139822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":325,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.330114+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:08:12.330114+0000","last_peered":"2024-04-21T09:08:12.330114+0000","last_clean":"2024-04-21T09:08:12.330114+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:08:12.330114+0000","last_undegraded":"2024-04-21T09:08:12.330114+0000","last_fullsized":"2024-04-21T09:08:12.330114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":191,"num_write":40,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142319+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:08:12.142319+0000","last_peered":"2024-04-21T09:08:12.142319+0000","last_clean":"2024-04-21T09:08:12.142319+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:08:12.142319+0000","last_undegraded":"2024-04-21T09:08:12.142319+0000","last_fullsized":"2024-04-21T09:08:12.142319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.151015+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:08:12.151015+0000","last_peered":"2024-04-21T09:08:12.151015+0000","last_clean":"2024-04-21T09:08:12.151015+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:08:12.151015+0000","last_undegraded":"2024-04-21T09:08:12.151015+0000","last_fullsized":"2024-04-21T09:08:12.151015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035051+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:08:12.035051+0000","last_peered":"2024-04-21T09:08:12.035051+0000","last_clean":"2024-04-21T09:08:12.035051+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:08:12.035051+0000","last_undegraded":"2024-04-21T09:08:12.035051+0000","last_fullsized":"2024-04-21T09:08:12.035051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033721+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:08:12.033721+0000","last_peered":"2024-04-21T09:08:12.033721+0000","last_clean":"2024-04-21T09:08:12.033721+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:08:12.033721+0000","last_undegraded":"2024-04-21T09:08:12.033721+0000","last_fullsized":"2024-04-21T09:08:12.033721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150756+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:08:12.150756+0000","last_peered":"2024-04-21T09:08:12.150756+0000","last_clean":"2024-04-21T09:08:12.150756+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:08:12.150756+0000","last_undegraded":"2024-04-21T09:08:12.150756+0000","last_fullsized":"2024-04-21T09:08:12.150756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325172+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:08:12.325172+0000","last_peered":"2024-04-21T09:08:12.325172+0000","last_clean":"2024-04-21T09:08:12.325172+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:08:12.325172+0000","last_undegraded":"2024-04-21T09:08:12.325172+0000","last_fullsized":"2024-04-21T09:08:12.325172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140181+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:08:12.140181+0000","last_peered":"2024-04-21T09:08:12.140181+0000","last_clean":"2024-04-21T09:08:12.140181+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:08:12.140181+0000","last_undegraded":"2024-04-21T09:08:12.140181+0000","last_fullsized":"2024-04-21T09:08:12.140181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'5","reported_seq":164,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324171+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:08:12.324171+0000","last_peered":"2024-04-21T09:08:12.324171+0000","last_clean":"2024-04-21T09:08:12.324171+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:08:12.324171+0000","last_undegraded":"2024-04-21T09:08:12.324171+0000","last_fullsized":"2024-04-21T09:08:12.324171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324692+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:08:12.324692+0000","last_peered":"2024-04-21T09:08:12.324692+0000","last_clean":"2024-04-21T09:08:12.324692+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:08:12.324692+0000","last_undegraded":"2024-04-21T09:08:12.324692+0000","last_fullsized":"2024-04-21T09:08:12.324692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033848+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:08:12.033848+0000","last_peered":"2024-04-21T09:08:12.033848+0000","last_clean":"2024-04-21T09:08:12.033848+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:08:12.033848+0000","last_undegraded":"2024-04-21T09:08:12.033848+0000","last_fullsized":"2024-04-21T09:08:12.033848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142080+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:08:12.142080+0000","last_peered":"2024-04-21T09:08:12.142080+0000","last_clean":"2024-04-21T09:08:12.142080+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:08:12.142080+0000","last_undegraded":"2024-04-21T09:08:12.142080+0000","last_fullsized":"2024-04-21T09:08:12.142080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150555+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:08:12.150555+0000","last_peered":"2024-04-21T09:08:12.150555+0000","last_clean":"2024-04-21T09:08:12.150555+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:08:12.150555+0000","last_undegraded":"2024-04-21T09:08:12.150555+0000","last_fullsized":"2024-04-21T09:08:12.150555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034523+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:08:12.034523+0000","last_peered":"2024-04-21T09:08:12.034523+0000","last_clean":"2024-04-21T09:08:12.034523+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:08:12.034523+0000","last_undegraded":"2024-04-21T09:08:12.034523+0000","last_fullsized":"2024-04-21T09:08:12.034523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763981+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:08:12.763981+0000","last_peered":"2024-04-21T09:08:12.763981+0000","last_clean":"2024-04-21T09:08:12.763981+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:08:12.763981+0000","last_undegraded":"2024-04-21T09:08:12.763981+0000","last_fullsized":"2024-04-21T09:08:12.763981+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142163+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:08:12.142163+0000","last_peered":"2024-04-21T09:08:12.142163+0000","last_clean":"2024-04-21T09:08:12.142163+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:08:12.142163+0000","last_undegraded":"2024-04-21T09:08:12.142163+0000","last_fullsized":"2024-04-21T09:08:12.142163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139908+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:08:12.139908+0000","last_peered":"2024-04-21T09:08:12.139908+0000","last_clean":"2024-04-21T09:08:12.139908+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:08:12.139908+0000","last_undegraded":"2024-04-21T09:08:12.139908+0000","last_fullsized":"2024-04-21T09:08:12.139908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034862+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:08:12.034862+0000","last_peered":"2024-04-21T09:08:12.034862+0000","last_clean":"2024-04-21T09:08:12.034862+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:08:12.034862+0000","last_undegraded":"2024-04-21T09:08:12.034862+0000","last_fullsized":"2024-04-21T09:08:12.034862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033827+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:08:12.033827+0000","last_peered":"2024-04-21T09:08:12.033827+0000","last_clean":"2024-04-21T09:08:12.033827+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:08:12.033827+0000","last_undegraded":"2024-04-21T09:08:12.033827+0000","last_fullsized":"2024-04-21T09:08:12.033827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325076+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:08:12.325076+0000","last_peered":"2024-04-21T09:08:12.325076+0000","last_clean":"2024-04-21T09:08:12.325076+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:08:12.325076+0000","last_undegraded":"2024-04-21T09:08:12.325076+0000","last_fullsized":"2024-04-21T09:08:12.325076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034855+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:08:12.034855+0000","last_peered":"2024-04-21T09:08:12.034855+0000","last_clean":"2024-04-21T09:08:12.034855+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:08:12.034855+0000","last_undegraded":"2024-04-21T09:08:12.034855+0000","last_fullsized":"2024-04-21T09:08:12.034855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033559+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:08:12.033559+0000","last_peered":"2024-04-21T09:08:12.033559+0000","last_clean":"2024-04-21T09:08:12.033559+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:08:12.033559+0000","last_undegraded":"2024-04-21T09:08:12.033559+0000","last_fullsized":"2024-04-21T09:08:12.033559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150754+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:08:12.150754+0000","last_peered":"2024-04-21T09:08:12.150754+0000","last_clean":"2024-04-21T09:08:12.150754+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:08:12.150754+0000","last_undegraded":"2024-04-21T09:08:12.150754+0000","last_fullsized":"2024-04-21T09:08:12.150754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":161,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033281+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:08:12.033281+0000","last_peered":"2024-04-21T09:08:12.033281+0000","last_clean":"2024-04-21T09:08:12.033281+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:08:12.033281+0000","last_undegraded":"2024-04-21T09:08:12.033281+0000","last_fullsized":"2024-04-21T09:08:12.033281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325490+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:08:12.325490+0000","last_peered":"2024-04-21T09:08:12.325490+0000","last_clean":"2024-04-21T09:08:12.325490+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:08:12.325490+0000","last_undegraded":"2024-04-21T09:08:12.325490+0000","last_fullsized":"2024-04-21T09:08:12.325490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142918+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:08:12.142918+0000","last_peered":"2024-04-21T09:08:12.142918+0000","last_clean":"2024-04-21T09:08:12.142918+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:08:12.142918+0000","last_undegraded":"2024-04-21T09:08:12.142918+0000","last_fullsized":"2024-04-21T09:08:12.142918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280892+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:08:12.280892+0000","last_peered":"2024-04-21T09:08:12.280892+0000","last_clean":"2024-04-21T09:08:12.280892+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:08:12.280892+0000","last_undegraded":"2024-04-21T09:08:12.280892+0000","last_fullsized":"2024-04-21T09:08:12.280892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":369,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324359+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:08:12.324359+0000","last_peered":"2024-04-21T09:08:12.324359+0000","last_clean":"2024-04-21T09:08:12.324359+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:08:12.324359+0000","last_undegraded":"2024-04-21T09:08:12.324359+0000","last_fullsized":"2024-04-21T09:08:12.324359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150742+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:08:12.150742+0000","last_peered":"2024-04-21T09:08:12.150742+0000","last_clean":"2024-04-21T09:08:12.150742+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:08:12.150742+0000","last_undegraded":"2024-04-21T09:08:12.150742+0000","last_fullsized":"2024-04-21T09:08:12.150742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033204+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:08:12.033204+0000","last_peered":"2024-04-21T09:08:12.033204+0000","last_clean":"2024-04-21T09:08:12.033204+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:08:12.033204+0000","last_undegraded":"2024-04-21T09:08:12.033204+0000","last_fullsized":"2024-04-21T09:08:12.033204+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324532+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:08:12.324532+0000","last_peered":"2024-04-21T09:08:12.324532+0000","last_clean":"2024-04-21T09:08:12.324532+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:08:12.324532+0000","last_undegraded":"2024-04-21T09:08:12.324532+0000","last_fullsized":"2024-04-21T09:08:12.324532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763722+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:08:12.763722+0000","last_peered":"2024-04-21T09:08:12.763722+0000","last_clean":"2024-04-21T09:08:12.763722+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:08:12.763722+0000","last_undegraded":"2024-04-21T09:08:12.763722+0000","last_fullsized":"2024-04-21T09:08:12.763722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142352+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:08:12.142352+0000","last_peered":"2024-04-21T09:08:12.142352+0000","last_clean":"2024-04-21T09:08:12.142352+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:08:12.142352+0000","last_undegraded":"2024-04-21T09:08:12.142352+0000","last_fullsized":"2024-04-21T09:08:12.142352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033592+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:08:12.033592+0000","last_peered":"2024-04-21T09:08:12.033592+0000","last_clean":"2024-04-21T09:08:12.033592+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:08:12.033592+0000","last_undegraded":"2024-04-21T09:08:12.033592+0000","last_fullsized":"2024-04-21T09:08:12.033592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142660+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:08:12.142660+0000","last_peered":"2024-04-21T09:08:12.142660+0000","last_clean":"2024-04-21T09:08:12.142660+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:08:12.142660+0000","last_undegraded":"2024-04-21T09:08:12.142660+0000","last_fullsized":"2024-04-21T09:08:12.142660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'45","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.271084+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:08:12.271084+0000","last_peered":"2024-04-21T09:08:12.271084+0000","last_clean":"2024-04-21T09:08:12.271084+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:08:12.271084+0000","last_undegraded":"2024-04-21T09:08:12.271084+0000","last_fullsized":"2024-04-21T09:08:12.271084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143688+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:08:12.143688+0000","last_peered":"2024-04-21T09:08:12.143688+0000","last_clean":"2024-04-21T09:08:12.143688+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:08:12.143688+0000","last_undegraded":"2024-04-21T09:08:12.143688+0000","last_fullsized":"2024-04-21T09:08:12.143688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034657+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:08:12.034657+0000","last_peered":"2024-04-21T09:08:12.034657+0000","last_clean":"2024-04-21T09:08:12.034657+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:08:12.034657+0000","last_undegraded":"2024-04-21T09:08:12.034657+0000","last_fullsized":"2024-04-21T09:08:12.034657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281282+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:08:12.281282+0000","last_peered":"2024-04-21T09:08:12.281282+0000","last_clean":"2024-04-21T09:08:12.281282+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:08:12.281282+0000","last_undegraded":"2024-04-21T09:08:12.281282+0000","last_fullsized":"2024-04-21T09:08:12.281282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970759+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:07:40.970759+0000","last_peered":"2024-04-21T09:07:40.970759+0000","last_clean":"2024-04-21T09:07:40.970759+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:07:40.970759+0000","last_undegraded":"2024-04-21T09:07:40.970759+0000","last_fullsized":"2024-04-21T09:07:40.970759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281130+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:08:12.281130+0000","last_peered":"2024-04-21T09:08:12.281130+0000","last_clean":"2024-04-21T09:08:12.281130+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:08:12.281130+0000","last_undegraded":"2024-04-21T09:08:12.281130+0000","last_fullsized":"2024-04-21T09:08:12.281130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323960+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:08:12.323960+0000","last_peered":"2024-04-21T09:08:12.323960+0000","last_clean":"2024-04-21T09:08:12.323960+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:08:12.323960+0000","last_undegraded":"2024-04-21T09:08:12.323960+0000","last_fullsized":"2024-04-21T09:08:12.323960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.141989+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:08:12.141989+0000","last_peered":"2024-04-21T09:08:12.141989+0000","last_clean":"2024-04-21T09:08:12.141989+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:08:12.141989+0000","last_undegraded":"2024-04-21T09:08:12.141989+0000","last_fullsized":"2024-04-21T09:08:12.141989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140305+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:08:12.140305+0000","last_peered":"2024-04-21T09:08:12.140305+0000","last_clean":"2024-04-21T09:08:12.140305+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:08:12.140305+0000","last_undegraded":"2024-04-21T09:08:12.140305+0000","last_fullsized":"2024-04-21T09:08:12.140305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143035+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:08:12.143035+0000","last_peered":"2024-04-21T09:08:12.143035+0000","last_clean":"2024-04-21T09:08:12.143035+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:08:12.143035+0000","last_undegraded":"2024-04-21T09:08:12.143035+0000","last_fullsized":"2024-04-21T09:08:12.143035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763571+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:08:12.763571+0000","last_peered":"2024-04-21T09:08:12.763571+0000","last_clean":"2024-04-21T09:08:12.763571+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:08:12.763571+0000","last_undegraded":"2024-04-21T09:08:12.763571+0000","last_fullsized":"2024-04-21T09:08:12.763571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281011+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:08:12.281011+0000","last_peered":"2024-04-21T09:08:12.281011+0000","last_clean":"2024-04-21T09:08:12.281011+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:08:12.281011+0000","last_undegraded":"2024-04-21T09:08:12.281011+0000","last_fullsized":"2024-04-21T09:08:12.281011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034866+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:08:12.034866+0000","last_peered":"2024-04-21T09:08:12.034866+0000","last_clean":"2024-04-21T09:08:12.034866+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:08:12.034866+0000","last_undegraded":"2024-04-21T09:08:12.034866+0000","last_fullsized":"2024-04-21T09:08:12.034866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143614+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:08:12.143614+0000","last_peered":"2024-04-21T09:08:12.143614+0000","last_clean":"2024-04-21T09:08:12.143614+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:08:12.143614+0000","last_undegraded":"2024-04-21T09:08:12.143614+0000","last_fullsized":"2024-04-21T09:08:12.143614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763696+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:08:12.763696+0000","last_peered":"2024-04-21T09:08:12.763696+0000","last_clean":"2024-04-21T09:08:12.763696+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:08:12.763696+0000","last_undegraded":"2024-04-21T09:08:12.763696+0000","last_fullsized":"2024-04-21T09:08:12.763696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325194+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:08:12.325194+0000","last_peered":"2024-04-21T09:08:12.325194+0000","last_clean":"2024-04-21T09:08:12.325194+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:08:12.325194+0000","last_undegraded":"2024-04-21T09:08:12.325194+0000","last_fullsized":"2024-04-21T09:08:12.325194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324153+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:08:12.324153+0000","last_peered":"2024-04-21T09:08:12.324153+0000","last_clean":"2024-04-21T09:08:12.324153+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:08:12.324153+0000","last_undegraded":"2024-04-21T09:08:12.324153+0000","last_fullsized":"2024-04-21T09:08:12.324153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324834+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:08:12.324834+0000","last_peered":"2024-04-21T09:08:12.324834+0000","last_clean":"2024-04-21T09:08:12.324834+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:08:12.324834+0000","last_undegraded":"2024-04-21T09:08:12.324834+0000","last_fullsized":"2024-04-21T09:08:12.324834+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325898+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:08:12.325898+0000","last_peered":"2024-04-21T09:08:12.325898+0000","last_clean":"2024-04-21T09:08:12.325898+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:08:12.325898+0000","last_undegraded":"2024-04-21T09:08:12.325898+0000","last_fullsized":"2024-04-21T09:08:12.325898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970437+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:07:40.970437+0000","last_peered":"2024-04-21T09:07:40.970437+0000","last_clean":"2024-04-21T09:07:40.970437+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:07:40.970437+0000","last_undegraded":"2024-04-21T09:07:40.970437+0000","last_fullsized":"2024-04-21T09:07:40.970437+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"96'39","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.264983+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:08:12.264983+0000","last_peered":"2024-04-21T09:08:12.264983+0000","last_clean":"2024-04-21T09:08:12.264983+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:08:12.264983+0000","last_undegraded":"2024-04-21T09:08:12.264983+0000","last_fullsized":"2024-04-21T09:08:12.264983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324272+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:08:12.324272+0000","last_peered":"2024-04-21T09:08:12.324272+0000","last_clean":"2024-04-21T09:08:12.324272+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:08:12.324272+0000","last_undegraded":"2024-04-21T09:08:12.324272+0000","last_fullsized":"2024-04-21T09:08:12.324272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150326+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:08:12.150326+0000","last_peered":"2024-04-21T09:08:12.150326+0000","last_clean":"2024-04-21T09:08:12.150326+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:08:12.150326+0000","last_undegraded":"2024-04-21T09:08:12.150326+0000","last_fullsized":"2024-04-21T09:08:12.150326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033814+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:08:12.033814+0000","last_peered":"2024-04-21T09:08:12.033814+0000","last_clean":"2024-04-21T09:08:12.033814+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:08:12.033814+0000","last_undegraded":"2024-04-21T09:08:12.033814+0000","last_fullsized":"2024-04-21T09:08:12.033814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325958+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:08:12.325958+0000","last_peered":"2024-04-21T09:08:12.325958+0000","last_clean":"2024-04-21T09:08:12.325958+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:08:12.325958+0000","last_undegraded":"2024-04-21T09:08:12.325958+0000","last_fullsized":"2024-04-21T09:08:12.325958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763645+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:08:12.763645+0000","last_peered":"2024-04-21T09:08:12.763645+0000","last_clean":"2024-04-21T09:08:12.763645+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:08:12.763645+0000","last_undegraded":"2024-04-21T09:08:12.763645+0000","last_fullsized":"2024-04-21T09:08:12.763645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034801+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:08:12.034801+0000","last_peered":"2024-04-21T09:08:12.034801+0000","last_clean":"2024-04-21T09:08:12.034801+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:08:12.034801+0000","last_undegraded":"2024-04-21T09:08:12.034801+0000","last_fullsized":"2024-04-21T09:08:12.034801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034715+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:08:12.034715+0000","last_peered":"2024-04-21T09:08:12.034715+0000","last_clean":"2024-04-21T09:08:12.034715+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:08:12.034715+0000","last_undegraded":"2024-04-21T09:08:12.034715+0000","last_fullsized":"2024-04-21T09:08:12.034715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970791+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:07:40.970791+0000","last_peered":"2024-04-21T09:07:40.970791+0000","last_clean":"2024-04-21T09:07:40.970791+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:07:40.970791+0000","last_undegraded":"2024-04-21T09:07:40.970791+0000","last_fullsized":"2024-04-21T09:07:40.970791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033916+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:08:12.033916+0000","last_peered":"2024-04-21T09:08:12.033916+0000","last_clean":"2024-04-21T09:08:12.033916+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:08:12.033916+0000","last_undegraded":"2024-04-21T09:08:12.033916+0000","last_fullsized":"2024-04-21T09:08:12.033916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763506+0000","last_change":"2024-04-21T09:07:40.649825+0000","last_active":"2024-04-21T09:08:12.763506+0000","last_peered":"2024-04-21T09:08:12.763506+0000","last_clean":"2024-04-21T09:08:12.763506+0000","last_became_active":"2024-04-21T09:07:40.649264+0000","last_became_peered":"2024-04-21T09:07:40.649264+0000","last_unstale":"2024-04-21T09:08:12.763506+0000","last_undegraded":"2024-04-21T09:08:12.763506+0000","last_fullsized":"2024-04-21T09:08:12.763506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:38.142948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035249+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:08:12.035249+0000","last_peered":"2024-04-21T09:08:12.035249+0000","last_clean":"2024-04-21T09:08:12.035249+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:08:12.035249+0000","last_undegraded":"2024-04-21T09:08:12.035249+0000","last_fullsized":"2024-04-21T09:08:12.035249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143896+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:08:12.143896+0000","last_peered":"2024-04-21T09:08:12.143896+0000","last_clean":"2024-04-21T09:08:12.143896+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:08:12.143896+0000","last_undegraded":"2024-04-21T09:08:12.143896+0000","last_fullsized":"2024-04-21T09:08:12.143896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150636+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:08:12.150636+0000","last_peered":"2024-04-21T09:08:12.150636+0000","last_clean":"2024-04-21T09:08:12.150636+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:08:12.150636+0000","last_undegraded":"2024-04-21T09:08:12.150636+0000","last_fullsized":"2024-04-21T09:08:12.150636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143204+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:08:12.143204+0000","last_peered":"2024-04-21T09:08:12.143204+0000","last_clean":"2024-04-21T09:08:12.143204+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:08:12.143204+0000","last_undegraded":"2024-04-21T09:08:12.143204+0000","last_fullsized":"2024-04-21T09:08:12.143204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324412+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:08:12.324412+0000","last_peered":"2024-04-21T09:08:12.324412+0000","last_clean":"2024-04-21T09:08:12.324412+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:08:12.324412+0000","last_undegraded":"2024-04-21T09:08:12.324412+0000","last_fullsized":"2024-04-21T09:08:12.324412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033628+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:08:12.033628+0000","last_peered":"2024-04-21T09:08:12.033628+0000","last_clean":"2024-04-21T09:08:12.033628+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:08:12.033628+0000","last_undegraded":"2024-04-21T09:08:12.033628+0000","last_fullsized":"2024-04-21T09:08:12.033628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763507+0000","last_change":"2024-04-21T09:07:40.650222+0000","last_active":"2024-04-21T09:08:12.763507+0000","last_peered":"2024-04-21T09:08:12.763507+0000","last_clean":"2024-04-21T09:08:12.763507+0000","last_became_active":"2024-04-21T09:07:40.649595+0000","last_became_peered":"2024-04-21T09:07:40.649595+0000","last_unstale":"2024-04-21T09:08:12.763507+0000","last_undegraded":"2024-04-21T09:08:12.763507+0000","last_fullsized":"2024-04-21T09:08:12.763507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:25.006017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107013,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596944,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":831,"num_read_kb":525,"num_write":287,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1790832,"data_compressed":80799,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":532061,"data_compressed":29477,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1220608,"data_stored":1328509,"data_compressed":75912,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":939453,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":935255,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1118208,"data_stored":1138669,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":537317,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":605237,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":528839,"data_compressed":29477,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":604801,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1062327,"data_compressed":57756,"data_compressed_allocated":77824,"data_compressed_original":606208,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888142,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":543679,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":1265158,"data_compressed":74281,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68010,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67574,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133467,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":201305,"data_compressed":7787,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:08:14.794 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-21T09:08:14.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:08:14.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.933+0000 7fd32b69a640 1 -- 172.21.15.66:0/651644210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 msgr2=0x7fd3241092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.933+0000 7fd32b69a640 1 --2- 172.21.15.66:0/651644210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd3241092a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009ca0 tx=0x7fd31c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.933+0000 7fd32b69a640 1 -- 172.21.15.66:0/651644210 shutdown_connections 2024-04-21T09:08:14.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.933+0000 7fd32b69a640 1 --2- 172.21.15.66:0/651644210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd3241092a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.933+0000 7fd32b69a640 1 --2- 172.21.15.66:0/651644210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd324101fc0 0x7fd324102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.933+0000 7fd32b69a640 1 --2- 172.21.15.66:0/651644210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd324100e70 0x7fd324101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.933+0000 7fd32b69a640 1 -- 172.21.15.66:0/651644210 >> 172.21.15.66:0/651644210 conn(0x7fd3240fc3e0 msgr2=0x7fd3240fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:14.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.934+0000 7fd32b69a640 1 -- 172.21.15.66:0/651644210 shutdown_connections 2024-04-21T09:08:14.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.934+0000 7fd32b69a640 1 -- 172.21.15.66:0/651644210 wait complete. 2024-04-21T09:08:14.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.934+0000 7fd32b69a640 1 Processor -- start 2024-04-21T09:08:14.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.934+0000 7fd32b69a640 1 -- start start 2024-04-21T09:08:14.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd32b69a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd324100e70 0x7fd324197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd32b69a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd324101fc0 0x7fd324197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd32b69a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd32419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd32b69a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd32410c9c0 con 0x7fd324102980 2024-04-21T09:08:14.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd32b69a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd32410c840 con 0x7fd324100e70 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd32b69a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd32410cb40 con 0x7fd324101fc0 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd32419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd32419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60962/0 (socket says 172.21.15.66:60962) 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329436640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd324100e70 0x7fd324197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 -- 172.21.15.66:0/1021238617 learned_addr learned my addr 172.21.15.66:0/1021238617 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd328c35640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd324101fc0 0x7fd324197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 -- 172.21.15.66:0/1021238617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd324101fc0 msgr2=0x7fd324197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd324101fc0 0x7fd324197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 -- 172.21.15.66:0/1021238617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd324100e70 msgr2=0x7fd324197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd324100e70 0x7fd324197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 -- 172.21.15.66:0/1021238617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32419ec30 con 0x7fd324102980 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd329c37640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd32419e5c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd31c02c700 tx=0x7fd31c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:14.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.935+0000 7fd31a7fc640 1 -- 172.21.15.66:0/1021238617 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c041020 con 0x7fd324102980 2024-04-21T09:08:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.936+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3240690e0 con 0x7fd324102980 2024-04-21T09:08:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.936+0000 7fd31a7fc640 1 -- 172.21.15.66:0/1021238617 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31c034d60 con 0x7fd324102980 2024-04-21T09:08:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.936+0000 7fd31a7fc640 1 -- 172.21.15.66:0/1021238617 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c039040 con 0x7fd324102980 2024-04-21T09:08:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.936+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd324069620 con 0x7fd324102980 2024-04-21T09:08:14.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.937+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd324068de0 con 0x7fd324102980 2024-04-21T09:08:14.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.938+0000 7fd31a7fc640 1 -- 172.21.15.66:0/1021238617 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7fd31c044020 con 0x7fd324102980 2024-04-21T09:08:14.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.938+0000 7fd31a7fc640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd3100a3d20 0x7fd3100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:14.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.939+0000 7fd329436640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd3100a3d20 0x7fd3100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:14.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.939+0000 7fd31a7fc640 1 -- 172.21.15.66:0/1021238617 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd31c0f2960 con 0x7fd324102980 2024-04-21T09:08:14.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.939+0000 7fd329436640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd3100a3d20 0x7fd3100a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd320009bd0 tx=0x7fd320005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:14.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:14.944+0000 7fd31a7fc640 1 -- 172.21.15.66:0/1021238617 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd31c04b020 con 0x7fd324102980 2024-04-21T09:08:15.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.162+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd324069880 con 0x7fd3100a3d20 2024-04-21T09:08:15.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.171+0000 7fd31a7fc640 1 -- 172.21.15.66:0/1021238617 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339695 (secure 0 0 0) 0x7fd324069880 con 0x7fd3100a3d20 2024-04-21T09:08:15.175 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:15.177 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:08:15.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd3100a3d20 msgr2=0x7fd3100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd3100a3d20 0x7fd3100a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd320009bd0 tx=0x7fd320005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:15.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 msgr2=0x7fd32419e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd32419e5c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd31c02c700 tx=0x7fd31c0093f0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 shutdown_connections 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd3100a3d20 0x7fd3100a61e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd324102980 0x7fd32419e5c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd324101fc0 0x7fd324197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 --2- 172.21.15.66:0/1021238617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd324100e70 0x7fd324197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 >> 172.21.15.66:0/1021238617 conn(0x7fd3240fc3e0 msgr2=0x7fd324105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 shutdown_connections 2024-04-21T09:08:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.175+0000 7fd32b69a640 1 -- 172.21.15.66:0/1021238617 wait complete. 2024-04-21T09:08:15.187 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T09:08:14.047546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187546,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":967,"num_read_kb":644,"num_write":470,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357988,"kb_used_data":12944,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583836,"statfs":{"total":1151940427776,"available":1151573848064,"internally_reserved":0,"allocated":13254656,"data_stored":10021305,"data_compressed":571306,"data_compressed_allocated":778240,"data_compressed_original":4407296,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989630"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764225+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:08:12.764225+0000","last_peered":"2024-04-21T09:08:12.764225+0000","last_clean":"2024-04-21T09:08:12.764225+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:08:12.764225+0000","last_undegraded":"2024-04-21T09:08:12.764225+0000","last_fullsized":"2024-04-21T09:08:12.764225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970868+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:07:40.970868+0000","last_peered":"2024-04-21T09:07:40.970868+0000","last_clean":"2024-04-21T09:07:40.970868+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:07:40.970868+0000","last_undegraded":"2024-04-21T09:07:40.970868+0000","last_fullsized":"2024-04-21T09:07:40.970868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970770+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:07:40.970770+0000","last_peered":"2024-04-21T09:07:40.970770+0000","last_clean":"2024-04-21T09:07:40.970770+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:07:40.970770+0000","last_undegraded":"2024-04-21T09:07:40.970770+0000","last_fullsized":"2024-04-21T09:07:40.970770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281097+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:08:12.281097+0000","last_peered":"2024-04-21T09:08:12.281097+0000","last_clean":"2024-04-21T09:08:12.281097+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:08:12.281097+0000","last_undegraded":"2024-04-21T09:08:12.281097+0000","last_fullsized":"2024-04-21T09:08:12.281097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142191+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:08:12.142191+0000","last_peered":"2024-04-21T09:08:12.142191+0000","last_clean":"2024-04-21T09:08:12.142191+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:08:12.142191+0000","last_undegraded":"2024-04-21T09:08:12.142191+0000","last_fullsized":"2024-04-21T09:08:12.142191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034608+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:08:12.034608+0000","last_peered":"2024-04-21T09:08:12.034608+0000","last_clean":"2024-04-21T09:08:12.034608+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:08:12.034608+0000","last_undegraded":"2024-04-21T09:08:12.034608+0000","last_fullsized":"2024-04-21T09:08:12.034608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324820+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:08:12.324820+0000","last_peered":"2024-04-21T09:08:12.324820+0000","last_clean":"2024-04-21T09:08:12.324820+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:08:12.324820+0000","last_undegraded":"2024-04-21T09:08:12.324820+0000","last_fullsized":"2024-04-21T09:08:12.324820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142634+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:08:12.142634+0000","last_peered":"2024-04-21T09:08:12.142634+0000","last_clean":"2024-04-21T09:08:12.142634+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:08:12.142634+0000","last_undegraded":"2024-04-21T09:08:12.142634+0000","last_fullsized":"2024-04-21T09:08:12.142634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325536+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:08:12.325536+0000","last_peered":"2024-04-21T09:08:12.325536+0000","last_clean":"2024-04-21T09:08:12.325536+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:08:12.325536+0000","last_undegraded":"2024-04-21T09:08:12.325536+0000","last_fullsized":"2024-04-21T09:08:12.325536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035030+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:08:12.035030+0000","last_peered":"2024-04-21T09:08:12.035030+0000","last_clean":"2024-04-21T09:08:12.035030+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:08:12.035030+0000","last_undegraded":"2024-04-21T09:08:12.035030+0000","last_fullsized":"2024-04-21T09:08:12.035030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325727+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:08:12.325727+0000","last_peered":"2024-04-21T09:08:12.325727+0000","last_clean":"2024-04-21T09:08:12.325727+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:08:12.325727+0000","last_undegraded":"2024-04-21T09:08:12.325727+0000","last_fullsized":"2024-04-21T09:08:12.325727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281072+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:08:12.281072+0000","last_peered":"2024-04-21T09:08:12.281072+0000","last_clean":"2024-04-21T09:08:12.281072+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:08:12.281072+0000","last_undegraded":"2024-04-21T09:08:12.281072+0000","last_fullsized":"2024-04-21T09:08:12.281072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033565+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:08:12.033565+0000","last_peered":"2024-04-21T09:08:12.033565+0000","last_clean":"2024-04-21T09:08:12.033565+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:08:12.033565+0000","last_undegraded":"2024-04-21T09:08:12.033565+0000","last_fullsized":"2024-04-21T09:08:12.033565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970421+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:07:40.970421+0000","last_peered":"2024-04-21T09:07:40.970421+0000","last_clean":"2024-04-21T09:07:40.970421+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:07:40.970421+0000","last_undegraded":"2024-04-21T09:07:40.970421+0000","last_fullsized":"2024-04-21T09:07:40.970421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970476+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:07:40.970476+0000","last_peered":"2024-04-21T09:07:40.970476+0000","last_clean":"2024-04-21T09:07:40.970476+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:07:40.970476+0000","last_undegraded":"2024-04-21T09:07:40.970476+0000","last_fullsized":"2024-04-21T09:07:40.970476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323842+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:08:12.323842+0000","last_peered":"2024-04-21T09:08:12.323842+0000","last_clean":"2024-04-21T09:08:12.323842+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:08:12.323842+0000","last_undegraded":"2024-04-21T09:08:12.323842+0000","last_fullsized":"2024-04-21T09:08:12.323842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764134+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:08:12.764134+0000","last_peered":"2024-04-21T09:08:12.764134+0000","last_clean":"2024-04-21T09:08:12.764134+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:08:12.764134+0000","last_undegraded":"2024-04-21T09:08:12.764134+0000","last_fullsized":"2024-04-21T09:08:12.764134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140490+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:08:12.140490+0000","last_peered":"2024-04-21T09:08:12.140490+0000","last_clean":"2024-04-21T09:08:12.140490+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:08:12.140490+0000","last_undegraded":"2024-04-21T09:08:12.140490+0000","last_fullsized":"2024-04-21T09:08:12.140490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035166+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:08:12.035166+0000","last_peered":"2024-04-21T09:08:12.035166+0000","last_clean":"2024-04-21T09:08:12.035166+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:08:12.035166+0000","last_undegraded":"2024-04-21T09:08:12.035166+0000","last_fullsized":"2024-04-21T09:08:12.035166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034821+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:08:12.034821+0000","last_peered":"2024-04-21T09:08:12.034821+0000","last_clean":"2024-04-21T09:08:12.034821+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:08:12.034821+0000","last_undegraded":"2024-04-21T09:08:12.034821+0000","last_fullsized":"2024-04-21T09:08:12.034821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140286+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:08:12.140286+0000","last_peered":"2024-04-21T09:08:12.140286+0000","last_clean":"2024-04-21T09:08:12.140286+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:08:12.140286+0000","last_undegraded":"2024-04-21T09:08:12.140286+0000","last_fullsized":"2024-04-21T09:08:12.140286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143771+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:08:12.143771+0000","last_peered":"2024-04-21T09:08:12.143771+0000","last_clean":"2024-04-21T09:08:12.143771+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:08:12.143771+0000","last_undegraded":"2024-04-21T09:08:12.143771+0000","last_fullsized":"2024-04-21T09:08:12.143771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280701+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:08:12.280701+0000","last_peered":"2024-04-21T09:08:12.280701+0000","last_clean":"2024-04-21T09:08:12.280701+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:08:12.280701+0000","last_undegraded":"2024-04-21T09:08:12.280701+0000","last_fullsized":"2024-04-21T09:08:12.280701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.144097+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:08:12.144097+0000","last_peered":"2024-04-21T09:08:12.144097+0000","last_clean":"2024-04-21T09:08:12.144097+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:08:12.144097+0000","last_undegraded":"2024-04-21T09:08:12.144097+0000","last_fullsized":"2024-04-21T09:08:12.144097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970613+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:07:40.970613+0000","last_peered":"2024-04-21T09:07:40.970613+0000","last_clean":"2024-04-21T09:07:40.970613+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:07:40.970613+0000","last_undegraded":"2024-04-21T09:07:40.970613+0000","last_fullsized":"2024-04-21T09:07:40.970613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280878+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:08:12.280878+0000","last_peered":"2024-04-21T09:08:12.280878+0000","last_clean":"2024-04-21T09:08:12.280878+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:08:12.280878+0000","last_undegraded":"2024-04-21T09:08:12.280878+0000","last_fullsized":"2024-04-21T09:08:12.280878+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142469+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:08:12.142469+0000","last_peered":"2024-04-21T09:08:12.142469+0000","last_clean":"2024-04-21T09:08:12.142469+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:08:12.142469+0000","last_undegraded":"2024-04-21T09:08:12.142469+0000","last_fullsized":"2024-04-21T09:08:12.142469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325055+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:08:12.325055+0000","last_peered":"2024-04-21T09:08:12.325055+0000","last_clean":"2024-04-21T09:08:12.325055+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:08:12.325055+0000","last_undegraded":"2024-04-21T09:08:12.325055+0000","last_fullsized":"2024-04-21T09:08:12.325055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280726+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:08:12.280726+0000","last_peered":"2024-04-21T09:08:12.280726+0000","last_clean":"2024-04-21T09:08:12.280726+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:08:12.280726+0000","last_undegraded":"2024-04-21T09:08:12.280726+0000","last_fullsized":"2024-04-21T09:08:12.280726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142104+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:08:12.142104+0000","last_peered":"2024-04-21T09:08:12.142104+0000","last_clean":"2024-04-21T09:08:12.142104+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:08:12.142104+0000","last_undegraded":"2024-04-21T09:08:12.142104+0000","last_fullsized":"2024-04-21T09:08:12.142104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150895+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:08:12.150895+0000","last_peered":"2024-04-21T09:08:12.150895+0000","last_clean":"2024-04-21T09:08:12.150895+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:08:12.150895+0000","last_undegraded":"2024-04-21T09:08:12.150895+0000","last_fullsized":"2024-04-21T09:08:12.150895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034947+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:08:12.034947+0000","last_peered":"2024-04-21T09:08:12.034947+0000","last_clean":"2024-04-21T09:08:12.034947+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:08:12.034947+0000","last_undegraded":"2024-04-21T09:08:12.034947+0000","last_fullsized":"2024-04-21T09:08:12.034947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'6","reported_seq":175,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.314168+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:08:12.314168+0000","last_peered":"2024-04-21T09:08:12.314168+0000","last_clean":"2024-04-21T09:08:12.314168+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:08:12.314168+0000","last_undegraded":"2024-04-21T09:08:12.314168+0000","last_fullsized":"2024-04-21T09:08:12.314168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142179+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:08:12.142179+0000","last_peered":"2024-04-21T09:08:12.142179+0000","last_clean":"2024-04-21T09:08:12.142179+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:08:12.142179+0000","last_undegraded":"2024-04-21T09:08:12.142179+0000","last_fullsized":"2024-04-21T09:08:12.142179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035266+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:08:12.035266+0000","last_peered":"2024-04-21T09:08:12.035266+0000","last_clean":"2024-04-21T09:08:12.035266+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:08:12.035266+0000","last_undegraded":"2024-04-21T09:08:12.035266+0000","last_fullsized":"2024-04-21T09:08:12.035266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034693+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:08:12.034693+0000","last_peered":"2024-04-21T09:08:12.034693+0000","last_clean":"2024-04-21T09:08:12.034693+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:08:12.034693+0000","last_undegraded":"2024-04-21T09:08:12.034693+0000","last_fullsized":"2024-04-21T09:08:12.034693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150724+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:08:12.150724+0000","last_peered":"2024-04-21T09:08:12.150724+0000","last_clean":"2024-04-21T09:08:12.150724+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:08:12.150724+0000","last_undegraded":"2024-04-21T09:08:12.150724+0000","last_fullsized":"2024-04-21T09:08:12.150724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763967+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:08:12.763967+0000","last_peered":"2024-04-21T09:08:12.763967+0000","last_clean":"2024-04-21T09:08:12.763967+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:08:12.763967+0000","last_undegraded":"2024-04-21T09:08:12.763967+0000","last_fullsized":"2024-04-21T09:08:12.763967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033440+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:08:12.033440+0000","last_peered":"2024-04-21T09:08:12.033440+0000","last_clean":"2024-04-21T09:08:12.033440+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:08:12.033440+0000","last_undegraded":"2024-04-21T09:08:12.033440+0000","last_fullsized":"2024-04-21T09:08:12.033440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142773+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:08:12.142773+0000","last_peered":"2024-04-21T09:08:12.142773+0000","last_clean":"2024-04-21T09:08:12.142773+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:08:12.142773+0000","last_undegraded":"2024-04-21T09:08:12.142773+0000","last_fullsized":"2024-04-21T09:08:12.142773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970644+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:07:40.970644+0000","last_peered":"2024-04-21T09:07:40.970644+0000","last_clean":"2024-04-21T09:07:40.970644+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:07:40.970644+0000","last_undegraded":"2024-04-21T09:07:40.970644+0000","last_fullsized":"2024-04-21T09:07:40.970644+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150624+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:08:12.150624+0000","last_peered":"2024-04-21T09:08:12.150624+0000","last_clean":"2024-04-21T09:08:12.150624+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:08:12.150624+0000","last_undegraded":"2024-04-21T09:08:12.150624+0000","last_fullsized":"2024-04-21T09:08:12.150624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139585+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:08:12.139585+0000","last_peered":"2024-04-21T09:08:12.139585+0000","last_clean":"2024-04-21T09:08:12.139585+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:08:12.139585+0000","last_undegraded":"2024-04-21T09:08:12.139585+0000","last_fullsized":"2024-04-21T09:08:12.139585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142906+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:08:12.142906+0000","last_peered":"2024-04-21T09:08:12.142906+0000","last_clean":"2024-04-21T09:08:12.142906+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:08:12.142906+0000","last_undegraded":"2024-04-21T09:08:12.142906+0000","last_fullsized":"2024-04-21T09:08:12.142906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":160,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.328407+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:08:12.328407+0000","last_peered":"2024-04-21T09:08:12.328407+0000","last_clean":"2024-04-21T09:08:12.328407+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:08:12.328407+0000","last_undegraded":"2024-04-21T09:08:12.328407+0000","last_fullsized":"2024-04-21T09:08:12.328407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325164+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:08:12.325164+0000","last_peered":"2024-04-21T09:08:12.325164+0000","last_clean":"2024-04-21T09:08:12.325164+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:08:12.325164+0000","last_undegraded":"2024-04-21T09:08:12.325164+0000","last_fullsized":"2024-04-21T09:08:12.325164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139403+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:08:12.139403+0000","last_peered":"2024-04-21T09:08:12.139403+0000","last_clean":"2024-04-21T09:08:12.139403+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:08:12.139403+0000","last_undegraded":"2024-04-21T09:08:12.139403+0000","last_fullsized":"2024-04-21T09:08:12.139403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325630+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:08:12.325630+0000","last_peered":"2024-04-21T09:08:12.325630+0000","last_clean":"2024-04-21T09:08:12.325630+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:08:12.325630+0000","last_undegraded":"2024-04-21T09:08:12.325630+0000","last_fullsized":"2024-04-21T09:08:12.325630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970868+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:07:40.970868+0000","last_peered":"2024-04-21T09:07:40.970868+0000","last_clean":"2024-04-21T09:07:40.970868+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:07:40.970868+0000","last_undegraded":"2024-04-21T09:07:40.970868+0000","last_fullsized":"2024-04-21T09:07:40.970868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150830+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:08:12.150830+0000","last_peered":"2024-04-21T09:08:12.150830+0000","last_clean":"2024-04-21T09:08:12.150830+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:08:12.150830+0000","last_undegraded":"2024-04-21T09:08:12.150830+0000","last_fullsized":"2024-04-21T09:08:12.150830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325080+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:08:12.325080+0000","last_peered":"2024-04-21T09:08:12.325080+0000","last_clean":"2024-04-21T09:08:12.325080+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:08:12.325080+0000","last_undegraded":"2024-04-21T09:08:12.325080+0000","last_fullsized":"2024-04-21T09:08:12.325080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139822+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:08:12.139822+0000","last_peered":"2024-04-21T09:08:12.139822+0000","last_clean":"2024-04-21T09:08:12.139822+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:08:12.139822+0000","last_undegraded":"2024-04-21T09:08:12.139822+0000","last_fullsized":"2024-04-21T09:08:12.139822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":325,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.330114+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:08:12.330114+0000","last_peered":"2024-04-21T09:08:12.330114+0000","last_clean":"2024-04-21T09:08:12.330114+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:08:12.330114+0000","last_undegraded":"2024-04-21T09:08:12.330114+0000","last_fullsized":"2024-04-21T09:08:12.330114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":191,"num_write":40,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142319+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:08:12.142319+0000","last_peered":"2024-04-21T09:08:12.142319+0000","last_clean":"2024-04-21T09:08:12.142319+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:08:12.142319+0000","last_undegraded":"2024-04-21T09:08:12.142319+0000","last_fullsized":"2024-04-21T09:08:12.142319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.151015+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:08:12.151015+0000","last_peered":"2024-04-21T09:08:12.151015+0000","last_clean":"2024-04-21T09:08:12.151015+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:08:12.151015+0000","last_undegraded":"2024-04-21T09:08:12.151015+0000","last_fullsized":"2024-04-21T09:08:12.151015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035051+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:08:12.035051+0000","last_peered":"2024-04-21T09:08:12.035051+0000","last_clean":"2024-04-21T09:08:12.035051+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:08:12.035051+0000","last_undegraded":"2024-04-21T09:08:12.035051+0000","last_fullsized":"2024-04-21T09:08:12.035051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033721+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:08:12.033721+0000","last_peered":"2024-04-21T09:08:12.033721+0000","last_clean":"2024-04-21T09:08:12.033721+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:08:12.033721+0000","last_undegraded":"2024-04-21T09:08:12.033721+0000","last_fullsized":"2024-04-21T09:08:12.033721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150756+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:08:12.150756+0000","last_peered":"2024-04-21T09:08:12.150756+0000","last_clean":"2024-04-21T09:08:12.150756+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:08:12.150756+0000","last_undegraded":"2024-04-21T09:08:12.150756+0000","last_fullsized":"2024-04-21T09:08:12.150756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325172+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:08:12.325172+0000","last_peered":"2024-04-21T09:08:12.325172+0000","last_clean":"2024-04-21T09:08:12.325172+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:08:12.325172+0000","last_undegraded":"2024-04-21T09:08:12.325172+0000","last_fullsized":"2024-04-21T09:08:12.325172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140181+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:08:12.140181+0000","last_peered":"2024-04-21T09:08:12.140181+0000","last_clean":"2024-04-21T09:08:12.140181+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:08:12.140181+0000","last_undegraded":"2024-04-21T09:08:12.140181+0000","last_fullsized":"2024-04-21T09:08:12.140181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'5","reported_seq":164,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324171+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:08:12.324171+0000","last_peered":"2024-04-21T09:08:12.324171+0000","last_clean":"2024-04-21T09:08:12.324171+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:08:12.324171+0000","last_undegraded":"2024-04-21T09:08:12.324171+0000","last_fullsized":"2024-04-21T09:08:12.324171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324692+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:08:12.324692+0000","last_peered":"2024-04-21T09:08:12.324692+0000","last_clean":"2024-04-21T09:08:12.324692+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:08:12.324692+0000","last_undegraded":"2024-04-21T09:08:12.324692+0000","last_fullsized":"2024-04-21T09:08:12.324692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033848+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:08:12.033848+0000","last_peered":"2024-04-21T09:08:12.033848+0000","last_clean":"2024-04-21T09:08:12.033848+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:08:12.033848+0000","last_undegraded":"2024-04-21T09:08:12.033848+0000","last_fullsized":"2024-04-21T09:08:12.033848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142080+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:08:12.142080+0000","last_peered":"2024-04-21T09:08:12.142080+0000","last_clean":"2024-04-21T09:08:12.142080+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:08:12.142080+0000","last_undegraded":"2024-04-21T09:08:12.142080+0000","last_fullsized":"2024-04-21T09:08:12.142080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150555+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:08:12.150555+0000","last_peered":"2024-04-21T09:08:12.150555+0000","last_clean":"2024-04-21T09:08:12.150555+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:08:12.150555+0000","last_undegraded":"2024-04-21T09:08:12.150555+0000","last_fullsized":"2024-04-21T09:08:12.150555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034523+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:08:12.034523+0000","last_peered":"2024-04-21T09:08:12.034523+0000","last_clean":"2024-04-21T09:08:12.034523+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:08:12.034523+0000","last_undegraded":"2024-04-21T09:08:12.034523+0000","last_fullsized":"2024-04-21T09:08:12.034523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763981+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:08:12.763981+0000","last_peered":"2024-04-21T09:08:12.763981+0000","last_clean":"2024-04-21T09:08:12.763981+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:08:12.763981+0000","last_undegraded":"2024-04-21T09:08:12.763981+0000","last_fullsized":"2024-04-21T09:08:12.763981+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142163+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:08:12.142163+0000","last_peered":"2024-04-21T09:08:12.142163+0000","last_clean":"2024-04-21T09:08:12.142163+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:08:12.142163+0000","last_undegraded":"2024-04-21T09:08:12.142163+0000","last_fullsized":"2024-04-21T09:08:12.142163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139908+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:08:12.139908+0000","last_peered":"2024-04-21T09:08:12.139908+0000","last_clean":"2024-04-21T09:08:12.139908+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:08:12.139908+0000","last_undegraded":"2024-04-21T09:08:12.139908+0000","last_fullsized":"2024-04-21T09:08:12.139908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034862+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:08:12.034862+0000","last_peered":"2024-04-21T09:08:12.034862+0000","last_clean":"2024-04-21T09:08:12.034862+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:08:12.034862+0000","last_undegraded":"2024-04-21T09:08:12.034862+0000","last_fullsized":"2024-04-21T09:08:12.034862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033827+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:08:12.033827+0000","last_peered":"2024-04-21T09:08:12.033827+0000","last_clean":"2024-04-21T09:08:12.033827+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:08:12.033827+0000","last_undegraded":"2024-04-21T09:08:12.033827+0000","last_fullsized":"2024-04-21T09:08:12.033827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325076+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:08:12.325076+0000","last_peered":"2024-04-21T09:08:12.325076+0000","last_clean":"2024-04-21T09:08:12.325076+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:08:12.325076+0000","last_undegraded":"2024-04-21T09:08:12.325076+0000","last_fullsized":"2024-04-21T09:08:12.325076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034855+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:08:12.034855+0000","last_peered":"2024-04-21T09:08:12.034855+0000","last_clean":"2024-04-21T09:08:12.034855+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:08:12.034855+0000","last_undegraded":"2024-04-21T09:08:12.034855+0000","last_fullsized":"2024-04-21T09:08:12.034855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033559+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:08:12.033559+0000","last_peered":"2024-04-21T09:08:12.033559+0000","last_clean":"2024-04-21T09:08:12.033559+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:08:12.033559+0000","last_undegraded":"2024-04-21T09:08:12.033559+0000","last_fullsized":"2024-04-21T09:08:12.033559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150754+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:08:12.150754+0000","last_peered":"2024-04-21T09:08:12.150754+0000","last_clean":"2024-04-21T09:08:12.150754+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:08:12.150754+0000","last_undegraded":"2024-04-21T09:08:12.150754+0000","last_fullsized":"2024-04-21T09:08:12.150754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":161,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033281+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:08:12.033281+0000","last_peered":"2024-04-21T09:08:12.033281+0000","last_clean":"2024-04-21T09:08:12.033281+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:08:12.033281+0000","last_undegraded":"2024-04-21T09:08:12.033281+0000","last_fullsized":"2024-04-21T09:08:12.033281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325490+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:08:12.325490+0000","last_peered":"2024-04-21T09:08:12.325490+0000","last_clean":"2024-04-21T09:08:12.325490+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:08:12.325490+0000","last_undegraded":"2024-04-21T09:08:12.325490+0000","last_fullsized":"2024-04-21T09:08:12.325490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142918+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:08:12.142918+0000","last_peered":"2024-04-21T09:08:12.142918+0000","last_clean":"2024-04-21T09:08:12.142918+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:08:12.142918+0000","last_undegraded":"2024-04-21T09:08:12.142918+0000","last_fullsized":"2024-04-21T09:08:12.142918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280892+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:08:12.280892+0000","last_peered":"2024-04-21T09:08:12.280892+0000","last_clean":"2024-04-21T09:08:12.280892+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:08:12.280892+0000","last_undegraded":"2024-04-21T09:08:12.280892+0000","last_fullsized":"2024-04-21T09:08:12.280892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":369,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324359+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:08:12.324359+0000","last_peered":"2024-04-21T09:08:12.324359+0000","last_clean":"2024-04-21T09:08:12.324359+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:08:12.324359+0000","last_undegraded":"2024-04-21T09:08:12.324359+0000","last_fullsized":"2024-04-21T09:08:12.324359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150742+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:08:12.150742+0000","last_peered":"2024-04-21T09:08:12.150742+0000","last_clean":"2024-04-21T09:08:12.150742+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:08:12.150742+0000","last_undegraded":"2024-04-21T09:08:12.150742+0000","last_fullsized":"2024-04-21T09:08:12.150742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033204+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:08:12.033204+0000","last_peered":"2024-04-21T09:08:12.033204+0000","last_clean":"2024-04-21T09:08:12.033204+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:08:12.033204+0000","last_undegraded":"2024-04-21T09:08:12.033204+0000","last_fullsized":"2024-04-21T09:08:12.033204+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324532+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:08:12.324532+0000","last_peered":"2024-04-21T09:08:12.324532+0000","last_clean":"2024-04-21T09:08:12.324532+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:08:12.324532+0000","last_undegraded":"2024-04-21T09:08:12.324532+0000","last_fullsized":"2024-04-21T09:08:12.324532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763722+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:08:12.763722+0000","last_peered":"2024-04-21T09:08:12.763722+0000","last_clean":"2024-04-21T09:08:12.763722+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:08:12.763722+0000","last_undegraded":"2024-04-21T09:08:12.763722+0000","last_fullsized":"2024-04-21T09:08:12.763722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142352+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:08:12.142352+0000","last_peered":"2024-04-21T09:08:12.142352+0000","last_clean":"2024-04-21T09:08:12.142352+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:08:12.142352+0000","last_undegraded":"2024-04-21T09:08:12.142352+0000","last_fullsized":"2024-04-21T09:08:12.142352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033592+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:08:12.033592+0000","last_peered":"2024-04-21T09:08:12.033592+0000","last_clean":"2024-04-21T09:08:12.033592+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:08:12.033592+0000","last_undegraded":"2024-04-21T09:08:12.033592+0000","last_fullsized":"2024-04-21T09:08:12.033592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142660+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:08:12.142660+0000","last_peered":"2024-04-21T09:08:12.142660+0000","last_clean":"2024-04-21T09:08:12.142660+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:08:12.142660+0000","last_undegraded":"2024-04-21T09:08:12.142660+0000","last_fullsized":"2024-04-21T09:08:12.142660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'45","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.271084+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:08:12.271084+0000","last_peered":"2024-04-21T09:08:12.271084+0000","last_clean":"2024-04-21T09:08:12.271084+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:08:12.271084+0000","last_undegraded":"2024-04-21T09:08:12.271084+0000","last_fullsized":"2024-04-21T09:08:12.271084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143688+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:08:12.143688+0000","last_peered":"2024-04-21T09:08:12.143688+0000","last_clean":"2024-04-21T09:08:12.143688+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:08:12.143688+0000","last_undegraded":"2024-04-21T09:08:12.143688+0000","last_fullsized":"2024-04-21T09:08:12.143688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034657+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:08:12.034657+0000","last_peered":"2024-04-21T09:08:12.034657+0000","last_clean":"2024-04-21T09:08:12.034657+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:08:12.034657+0000","last_undegraded":"2024-04-21T09:08:12.034657+0000","last_fullsized":"2024-04-21T09:08:12.034657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281282+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:08:12.281282+0000","last_peered":"2024-04-21T09:08:12.281282+0000","last_clean":"2024-04-21T09:08:12.281282+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:08:12.281282+0000","last_undegraded":"2024-04-21T09:08:12.281282+0000","last_fullsized":"2024-04-21T09:08:12.281282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970759+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:07:40.970759+0000","last_peered":"2024-04-21T09:07:40.970759+0000","last_clean":"2024-04-21T09:07:40.970759+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:07:40.970759+0000","last_undegraded":"2024-04-21T09:07:40.970759+0000","last_fullsized":"2024-04-21T09:07:40.970759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281130+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:08:12.281130+0000","last_peered":"2024-04-21T09:08:12.281130+0000","last_clean":"2024-04-21T09:08:12.281130+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:08:12.281130+0000","last_undegraded":"2024-04-21T09:08:12.281130+0000","last_fullsized":"2024-04-21T09:08:12.281130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323960+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:08:12.323960+0000","last_peered":"2024-04-21T09:08:12.323960+0000","last_clean":"2024-04-21T09:08:12.323960+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:08:12.323960+0000","last_undegraded":"2024-04-21T09:08:12.323960+0000","last_fullsized":"2024-04-21T09:08:12.323960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.141989+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:08:12.141989+0000","last_peered":"2024-04-21T09:08:12.141989+0000","last_clean":"2024-04-21T09:08:12.141989+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:08:12.141989+0000","last_undegraded":"2024-04-21T09:08:12.141989+0000","last_fullsized":"2024-04-21T09:08:12.141989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140305+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:08:12.140305+0000","last_peered":"2024-04-21T09:08:12.140305+0000","last_clean":"2024-04-21T09:08:12.140305+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:08:12.140305+0000","last_undegraded":"2024-04-21T09:08:12.140305+0000","last_fullsized":"2024-04-21T09:08:12.140305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143035+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:08:12.143035+0000","last_peered":"2024-04-21T09:08:12.143035+0000","last_clean":"2024-04-21T09:08:12.143035+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:08:12.143035+0000","last_undegraded":"2024-04-21T09:08:12.143035+0000","last_fullsized":"2024-04-21T09:08:12.143035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763571+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:08:12.763571+0000","last_peered":"2024-04-21T09:08:12.763571+0000","last_clean":"2024-04-21T09:08:12.763571+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:08:12.763571+0000","last_undegraded":"2024-04-21T09:08:12.763571+0000","last_fullsized":"2024-04-21T09:08:12.763571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281011+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:08:12.281011+0000","last_peered":"2024-04-21T09:08:12.281011+0000","last_clean":"2024-04-21T09:08:12.281011+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:08:12.281011+0000","last_undegraded":"2024-04-21T09:08:12.281011+0000","last_fullsized":"2024-04-21T09:08:12.281011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034866+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:08:12.034866+0000","last_peered":"2024-04-21T09:08:12.034866+0000","last_clean":"2024-04-21T09:08:12.034866+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:08:12.034866+0000","last_undegraded":"2024-04-21T09:08:12.034866+0000","last_fullsized":"2024-04-21T09:08:12.034866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143614+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:08:12.143614+0000","last_peered":"2024-04-21T09:08:12.143614+0000","last_clean":"2024-04-21T09:08:12.143614+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:08:12.143614+0000","last_undegraded":"2024-04-21T09:08:12.143614+0000","last_fullsized":"2024-04-21T09:08:12.143614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763696+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:08:12.763696+0000","last_peered":"2024-04-21T09:08:12.763696+0000","last_clean":"2024-04-21T09:08:12.763696+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:08:12.763696+0000","last_undegraded":"2024-04-21T09:08:12.763696+0000","last_fullsized":"2024-04-21T09:08:12.763696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325194+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:08:12.325194+0000","last_peered":"2024-04-21T09:08:12.325194+0000","last_clean":"2024-04-21T09:08:12.325194+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:08:12.325194+0000","last_undegraded":"2024-04-21T09:08:12.325194+0000","last_fullsized":"2024-04-21T09:08:12.325194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324153+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:08:12.324153+0000","last_peered":"2024-04-21T09:08:12.324153+0000","last_clean":"2024-04-21T09:08:12.324153+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:08:12.324153+0000","last_undegraded":"2024-04-21T09:08:12.324153+0000","last_fullsized":"2024-04-21T09:08:12.324153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324834+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:08:12.324834+0000","last_peered":"2024-04-21T09:08:12.324834+0000","last_clean":"2024-04-21T09:08:12.324834+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:08:12.324834+0000","last_undegraded":"2024-04-21T09:08:12.324834+0000","last_fullsized":"2024-04-21T09:08:12.324834+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325898+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:08:12.325898+0000","last_peered":"2024-04-21T09:08:12.325898+0000","last_clean":"2024-04-21T09:08:12.325898+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:08:12.325898+0000","last_undegraded":"2024-04-21T09:08:12.325898+0000","last_fullsized":"2024-04-21T09:08:12.325898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970437+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:07:40.970437+0000","last_peered":"2024-04-21T09:07:40.970437+0000","last_clean":"2024-04-21T09:07:40.970437+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:07:40.970437+0000","last_undegraded":"2024-04-21T09:07:40.970437+0000","last_fullsized":"2024-04-21T09:07:40.970437+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"96'39","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.264983+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:08:12.264983+0000","last_peered":"2024-04-21T09:08:12.264983+0000","last_clean":"2024-04-21T09:08:12.264983+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:08:12.264983+0000","last_undegraded":"2024-04-21T09:08:12.264983+0000","last_fullsized":"2024-04-21T09:08:12.264983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324272+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:08:12.324272+0000","last_peered":"2024-04-21T09:08:12.324272+0000","last_clean":"2024-04-21T09:08:12.324272+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:08:12.324272+0000","last_undegraded":"2024-04-21T09:08:12.324272+0000","last_fullsized":"2024-04-21T09:08:12.324272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150326+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:08:12.150326+0000","last_peered":"2024-04-21T09:08:12.150326+0000","last_clean":"2024-04-21T09:08:12.150326+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:08:12.150326+0000","last_undegraded":"2024-04-21T09:08:12.150326+0000","last_fullsized":"2024-04-21T09:08:12.150326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033814+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:08:12.033814+0000","last_peered":"2024-04-21T09:08:12.033814+0000","last_clean":"2024-04-21T09:08:12.033814+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:08:12.033814+0000","last_undegraded":"2024-04-21T09:08:12.033814+0000","last_fullsized":"2024-04-21T09:08:12.033814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325958+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:08:12.325958+0000","last_peered":"2024-04-21T09:08:12.325958+0000","last_clean":"2024-04-21T09:08:12.325958+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:08:12.325958+0000","last_undegraded":"2024-04-21T09:08:12.325958+0000","last_fullsized":"2024-04-21T09:08:12.325958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763645+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:08:12.763645+0000","last_peered":"2024-04-21T09:08:12.763645+0000","last_clean":"2024-04-21T09:08:12.763645+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:08:12.763645+0000","last_undegraded":"2024-04-21T09:08:12.763645+0000","last_fullsized":"2024-04-21T09:08:12.763645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034801+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:08:12.034801+0000","last_peered":"2024-04-21T09:08:12.034801+0000","last_clean":"2024-04-21T09:08:12.034801+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:08:12.034801+0000","last_undegraded":"2024-04-21T09:08:12.034801+0000","last_fullsized":"2024-04-21T09:08:12.034801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034715+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:08:12.034715+0000","last_peered":"2024-04-21T09:08:12.034715+0000","last_clean":"2024-04-21T09:08:12.034715+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:08:12.034715+0000","last_undegraded":"2024-04-21T09:08:12.034715+0000","last_fullsized":"2024-04-21T09:08:12.034715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T09:07:40.970791+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:07:40.970791+0000","last_peered":"2024-04-21T09:07:40.970791+0000","last_clean":"2024-04-21T09:07:40.970791+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:07:40.970791+0000","last_undegraded":"2024-04-21T09:07:40.970791+0000","last_fullsized":"2024-04-21T09:07:40.970791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033916+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:08:12.033916+0000","last_peered":"2024-04-21T09:08:12.033916+0000","last_clean":"2024-04-21T09:08:12.033916+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:08:12.033916+0000","last_undegraded":"2024-04-21T09:08:12.033916+0000","last_fullsized":"2024-04-21T09:08:12.033916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763506+0000","last_change":"2024-04-21T09:07:40.649825+0000","last_active":"2024-04-21T09:08:12.763506+0000","last_peered":"2024-04-21T09:08:12.763506+0000","last_clean":"2024-04-21T09:08:12.763506+0000","last_became_active":"2024-04-21T09:07:40.649264+0000","last_became_peered":"2024-04-21T09:07:40.649264+0000","last_unstale":"2024-04-21T09:08:12.763506+0000","last_undegraded":"2024-04-21T09:08:12.763506+0000","last_fullsized":"2024-04-21T09:08:12.763506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:38.142948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035249+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:08:12.035249+0000","last_peered":"2024-04-21T09:08:12.035249+0000","last_clean":"2024-04-21T09:08:12.035249+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:08:12.035249+0000","last_undegraded":"2024-04-21T09:08:12.035249+0000","last_fullsized":"2024-04-21T09:08:12.035249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143896+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:08:12.143896+0000","last_peered":"2024-04-21T09:08:12.143896+0000","last_clean":"2024-04-21T09:08:12.143896+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:08:12.143896+0000","last_undegraded":"2024-04-21T09:08:12.143896+0000","last_fullsized":"2024-04-21T09:08:12.143896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150636+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:08:12.150636+0000","last_peered":"2024-04-21T09:08:12.150636+0000","last_clean":"2024-04-21T09:08:12.150636+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:08:12.150636+0000","last_undegraded":"2024-04-21T09:08:12.150636+0000","last_fullsized":"2024-04-21T09:08:12.150636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143204+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:08:12.143204+0000","last_peered":"2024-04-21T09:08:12.143204+0000","last_clean":"2024-04-21T09:08:12.143204+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:08:12.143204+0000","last_undegraded":"2024-04-21T09:08:12.143204+0000","last_fullsized":"2024-04-21T09:08:12.143204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324412+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:08:12.324412+0000","last_peered":"2024-04-21T09:08:12.324412+0000","last_clean":"2024-04-21T09:08:12.324412+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:08:12.324412+0000","last_undegraded":"2024-04-21T09:08:12.324412+0000","last_fullsized":"2024-04-21T09:08:12.324412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033628+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:08:12.033628+0000","last_peered":"2024-04-21T09:08:12.033628+0000","last_clean":"2024-04-21T09:08:12.033628+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:08:12.033628+0000","last_undegraded":"2024-04-21T09:08:12.033628+0000","last_fullsized":"2024-04-21T09:08:12.033628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763507+0000","last_change":"2024-04-21T09:07:40.650222+0000","last_active":"2024-04-21T09:08:12.763507+0000","last_peered":"2024-04-21T09:08:12.763507+0000","last_clean":"2024-04-21T09:08:12.763507+0000","last_became_active":"2024-04-21T09:07:40.649595+0000","last_became_peered":"2024-04-21T09:07:40.649595+0000","last_unstale":"2024-04-21T09:08:12.763507+0000","last_undegraded":"2024-04-21T09:08:12.763507+0000","last_fullsized":"2024-04-21T09:08:12.763507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:25.006017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107013,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596944,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":831,"num_read_kb":525,"num_write":287,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1790832,"data_compressed":80799,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":532061,"data_compressed":29477,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1220608,"data_stored":1328509,"data_compressed":75912,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":939453,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":935255,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1118208,"data_stored":1138669,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":537317,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":605237,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":528839,"data_compressed":29477,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":604801,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1062327,"data_compressed":57756,"data_compressed_allocated":77824,"data_compressed_original":606208,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888142,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":543679,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":1265158,"data_compressed":74281,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68010,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67574,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133467,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":201305,"data_compressed":7787,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:08:15.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 -- 172.21.15.66:0/1935635603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 msgr2=0x7fedf41092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 --2- 172.21.15.66:0/1935635603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf41092a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fedec009ca0 tx=0x7fedec02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 -- 172.21.15.66:0/1935635603 shutdown_connections 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 --2- 172.21.15.66:0/1935635603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf41092a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 --2- 172.21.15.66:0/1935635603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedf4101fc0 0x7fedf4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 --2- 172.21.15.66:0/1935635603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf4100e70 0x7fedf4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 -- 172.21.15.66:0/1935635603 >> 172.21.15.66:0/1935635603 conn(0x7fedf40fc3e0 msgr2=0x7fedf40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:15.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 -- 172.21.15.66:0/1935635603 shutdown_connections 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.340+0000 7fedfb655640 1 -- 172.21.15.66:0/1935635603 wait complete. 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 Processor -- start 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 -- start start 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf4100e70 0x7fedf4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedf4101fc0 0x7fedf4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedf410c9a0 con 0x7fedf4102980 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedf410c820 con 0x7fedf4101fc0 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedfb655640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fedf410cb20 con 0x7fedf4100e70 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60990/0 (socket says 172.21.15.66:60990) 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 -- 172.21.15.66:0/2459585789 learned_addr learned my addr 172.21.15.66:0/2459585789 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:15.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf93f1640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf4100e70 0x7fedf4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf8bf0640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedf4101fc0 0x7fedf4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 -- 172.21.15.66:0/2459585789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf4100e70 msgr2=0x7fedf4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf4100e70 0x7fedf4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 -- 172.21.15.66:0/2459585789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedf4101fc0 msgr2=0x7fedf4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedf4101fc0 0x7fedf4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 -- 172.21.15.66:0/2459585789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf419fd50 con 0x7fedf4102980 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf8bf0640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedf4101fc0 0x7fedf4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.341+0000 7fedf9bf2640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf419f6e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fedec032520 tx=0x7fedec0347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.342+0000 7fedea7fc640 1 -- 172.21.15.66:0/2459585789 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec039040 con 0x7fedf4102980 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.342+0000 7fedea7fc640 1 -- 172.21.15.66:0/2459585789 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedec044070 con 0x7fedf4102980 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.342+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf419ffe0 con 0x7fedf4102980 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.342+0000 7fedea7fc640 1 -- 172.21.15.66:0/2459585789 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec0042a0 con 0x7fedf4102980 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.342+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf41a0440 con 0x7fedf4102980 2024-04-21T09:08:15.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.343+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf41a0700 con 0x7fedf4102980 2024-04-21T09:08:15.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.344+0000 7fedea7fc640 1 -- 172.21.15.66:0/2459585789 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7fedec003970 con 0x7fedf4102980 2024-04-21T09:08:15.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.344+0000 7fedea7fc640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fede00a3d20 0x7fede00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.345+0000 7fedea7fc640 1 -- 172.21.15.66:0/2459585789 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fedec0f2cb0 con 0x7fedf4102980 2024-04-21T09:08:15.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.345+0000 7fedf93f1640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fede00a3d20 0x7fede00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.345+0000 7fedf93f1640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fede00a3d20 0x7fede00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fedf0005d00 tx=0x7fedf000b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:15.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.349+0000 7fedea7fc640 1 -- 172.21.15.66:0/2459585789 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedec035d30 con 0x7fedf4102980 2024-04-21T09:08:15.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.539+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedf41a0a90 con 0x7fedf4102980 2024-04-21T09:08:15.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.540+0000 7fedea7fc640 1 -- 172.21.15.66:0/2459585789 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+25086 (secure 0 0 0) 0x7fedec0bf0f0 con 0x7fedf4102980 2024-04-21T09:08:15.541 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:15.541 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:08:12.028415+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:15.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fede00a3d20 msgr2=0x7fede00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fede00a3d20 0x7fede00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fedf0005d00 tx=0x7fedf000b410 comp rx=0 tx=0).stop 2024-04-21T09:08:15.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 msgr2=0x7fedf419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf419f6e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fedec032520 tx=0x7fedec0347b0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 shutdown_connections 2024-04-21T09:08:15.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fede00a3d20 0x7fede00a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedf4102980 0x7fedf419f6e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedf4101fc0 0x7fedf4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 --2- 172.21.15.66:0/2459585789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf4100e70 0x7fedf4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.542+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 >> 172.21.15.66:0/2459585789 conn(0x7fedf40fc3e0 msgr2=0x7fedf40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:15.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.543+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 shutdown_connections 2024-04-21T09:08:15.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.543+0000 7fedfb655640 1 -- 172.21.15.66:0/2459585789 wait complete. 2024-04-21T09:08:15.553 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:15 smithi066 ceph-mon[27953]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:15.553 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:15 smithi066 ceph-mon[27953]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-21T09:08:15.553 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:15 smithi066 ceph-mon[27953]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:15.553 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:15 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:15.553 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:15 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:15.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:08:15.555 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:08:15.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:15 smithi186 ceph-mon[31505]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:15.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:15 smithi186 ceph-mon[31505]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-21T09:08:15.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:15 smithi186 ceph-mon[31505]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:15 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:15 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:15.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:15 smithi083 ceph-mon[31413]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:15.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:15 smithi083 ceph-mon[31413]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-21T09:08:15.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:15 smithi083 ceph-mon[31413]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:15.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:15 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:15.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:15 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:15.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 -- 172.21.15.66:0/4006641700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888100e90 msgr2=0x7fc8881012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 --2- 172.21.15.66:0/4006641700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888100e90 0x7fc8881012b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc880009860 tx=0x7fc88002bb70 comp rx=0 tx=0).stop 2024-04-21T09:08:15.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 -- 172.21.15.66:0/4006641700 shutdown_connections 2024-04-21T09:08:15.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 --2- 172.21.15.66:0/4006641700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8881029a0 0x7fc8881092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 --2- 172.21.15.66:0/4006641700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc888101fe0 0x7fc888102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 --2- 172.21.15.66:0/4006641700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888100e90 0x7fc8881012b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 -- 172.21.15.66:0/4006641700 >> 172.21.15.66:0/4006641700 conn(0x7fc8880fc400 msgr2=0x7fc8880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:15.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 -- 172.21.15.66:0/4006641700 shutdown_connections 2024-04-21T09:08:15.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.664+0000 7fc890152640 1 -- 172.21.15.66:0/4006641700 wait complete. 2024-04-21T09:08:15.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 Processor -- start 2024-04-21T09:08:15.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 -- start start 2024-04-21T09:08:15.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc888100e90 0x7fc8881981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888101fe0 0x7fc888198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8881029a0 0x7fc88819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810c990 con 0x7fc8881029a0 2024-04-21T09:08:15.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810c810 con 0x7fc888100e90 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc890152640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810cb10 con 0x7fc888101fe0 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc88e6ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8881029a0 0x7fc88819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc88e6ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8881029a0 0x7fc88819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32774/0 (socket says 172.21.15.66:32774) 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc88e6ef640 1 -- 172.21.15.66:0/3347648933 learned_addr learned my addr 172.21.15.66:0/3347648933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc88deee640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc888100e90 0x7fc8881981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.665+0000 7fc88d6ed640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888101fe0 0x7fc888198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88e6ef640 1 -- 172.21.15.66:0/3347648933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888101fe0 msgr2=0x7fc888198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88e6ef640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888101fe0 0x7fc888198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88e6ef640 1 -- 172.21.15.66:0/3347648933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc888100e90 msgr2=0x7fc8881981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88e6ef640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc888100e90 0x7fc8881981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88e6ef640 1 -- 172.21.15.66:0/3347648933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88819fde0 con 0x7fc8881029a0 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88d6ed640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888101fe0 0x7fc888198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88deee640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc888100e90 0x7fc8881981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:15.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.666+0000 7fc88e6ef640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8881029a0 0x7fc88819f770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc87800eb10 tx=0x7fc87800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:15.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.669+0000 7fc886ffd640 1 -- 172.21.15.66:0/3347648933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878004b20 con 0x7fc8881029a0 2024-04-21T09:08:15.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.669+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8881a00d0 con 0x7fc8881029a0 2024-04-21T09:08:15.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.669+0000 7fc886ffd640 1 -- 172.21.15.66:0/3347648933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc878004cc0 con 0x7fc8881029a0 2024-04-21T09:08:15.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.669+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8881a0620 con 0x7fc8881029a0 2024-04-21T09:08:15.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.669+0000 7fc886ffd640 1 -- 172.21.15.66:0/3347648933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878018630 con 0x7fc8881029a0 2024-04-21T09:08:15.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.670+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8881a0930 con 0x7fc8881029a0 2024-04-21T09:08:15.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.671+0000 7fc886ffd640 1 -- 172.21.15.66:0/3347648933 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7fc878005000 con 0x7fc8881029a0 2024-04-21T09:08:15.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.671+0000 7fc886ffd640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8700a3d20 0x7fc8700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.672+0000 7fc886ffd640 1 -- 172.21.15.66:0/3347648933 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc8780d1240 con 0x7fc8881029a0 2024-04-21T09:08:15.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.672+0000 7fc88deee640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8700a3d20 0x7fc8700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.672+0000 7fc88deee640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8700a3d20 0x7fc8700a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc88002c180 tx=0x7fc8800349b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:15.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.676+0000 7fc886ffd640 1 -- 172.21.15.66:0/3347648933 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc87809d570 con 0x7fc8881029a0 2024-04-21T09:08:15.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.864+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8881a0b00 con 0x7fc8881029a0 2024-04-21T09:08:15.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.865+0000 7fc886ffd640 1 -- 172.21.15.66:0/3347648933 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fc87809ce70 con 0x7fc8881029a0 2024-04-21T09:08:15.865 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:08:15.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8700a3d20 msgr2=0x7fc8700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8700a3d20 0x7fc8700a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc88002c180 tx=0x7fc8800349b0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8881029a0 msgr2=0x7fc88819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8881029a0 0x7fc88819f770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc87800eb10 tx=0x7fc87800c2d0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 shutdown_connections 2024-04-21T09:08:15.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8700a3d20 0x7fc8700a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8881029a0 0x7fc88819f770 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc888101fe0 0x7fc888198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 --2- 172.21.15.66:0/3347648933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc888100e90 0x7fc8881981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 >> 172.21.15.66:0/3347648933 conn(0x7fc8880fc400 msgr2=0x7fc8880690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:15.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 shutdown_connections 2024-04-21T09:08:15.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.867+0000 7fc890152640 1 -- 172.21.15.66:0/3347648933 wait complete. 2024-04-21T09:08:15.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:08:15.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 -- 172.21.15.66:0/1838467927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604100e60 msgr2=0x7f1604101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 --2- 172.21.15.66:0/1838467927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604100e60 0x7f1604101280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009900 tx=0x7f15f402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:15.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 -- 172.21.15.66:0/1838467927 shutdown_connections 2024-04-21T09:08:15.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 --2- 172.21.15.66:0/1838467927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604102a00 0x7f1604109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 --2- 172.21.15.66:0/1838467927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1604102040 0x7f16041024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 --2- 172.21.15.66:0/1838467927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604100e60 0x7f1604101280 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 -- 172.21.15.66:0/1838467927 >> 172.21.15.66:0/1838467927 conn(0x7f16040fc440 msgr2=0x7f16040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 -- 172.21.15.66:0/1838467927 shutdown_connections 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 -- 172.21.15.66:0/1838467927 wait complete. 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.984+0000 7f16096d9640 1 Processor -- start 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16096d9640 1 -- start start 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16096d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1604100e60 0x7f160406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16096d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604102040 0x7f160419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16096d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604102a00 0x7f16041a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16096d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f160410c9d0 con 0x7f1604102a00 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16096d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f160410c850 con 0x7f1604102040 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16096d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f160410cb50 con 0x7f1604100e60 2024-04-21T09:08:15.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604102a00 0x7f16041a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604102a00 0x7f16041a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32790/0 (socket says 172.21.15.66:32790) 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 -- 172.21.15.66:0/1241053856 learned_addr learned my addr 172.21.15.66:0/1241053856 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16027fc640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604102040 0x7f160419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f1602ffd640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1604100e60 0x7f160406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 -- 172.21.15.66:0/1241053856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1604100e60 msgr2=0x7f160406b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1604100e60 0x7f160406b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 -- 172.21.15.66:0/1241053856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604102040 msgr2=0x7f160419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604102040 0x7f160419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 -- 172.21.15.66:0/1241053856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16041a2000 con 0x7f1604102a00 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f16037fe640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604102a00 0x7f16041a1810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00eb10 tx=0x7f15fc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.985+0000 7f15fbfff640 1 -- 172.21.15.66:0/1241053856 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00ce20 con 0x7f1604102a00 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.986+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16041a2290 con 0x7f1604102a00 2024-04-21T09:08:15.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.986+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16041a27e0 con 0x7f1604102a00 2024-04-21T09:08:15.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.986+0000 7f15fbfff640 1 -- 172.21.15.66:0/1241053856 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15fc005020 con 0x7f1604102a00 2024-04-21T09:08:15.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.986+0000 7f15fbfff640 1 -- 172.21.15.66:0/1241053856 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc010690 con 0x7f1604102a00 2024-04-21T09:08:15.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.987+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16041082d0 con 0x7f1604102a00 2024-04-21T09:08:15.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.988+0000 7f15fbfff640 1 -- 172.21.15.66:0/1241053856 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f15fc0054b0 con 0x7f1604102a00 2024-04-21T09:08:15.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.988+0000 7f15fbfff640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f15f00a3d50 0x7f15f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:15.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.988+0000 7f1602ffd640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f15f00a3d50 0x7f15f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:15.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.988+0000 7f15fbfff640 1 -- 172.21.15.66:0/1241053856 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f15fc0d10b0 con 0x7f1604102a00 2024-04-21T09:08:15.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.989+0000 7f1602ffd640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f15f00a3d50 0x7f15f00a6210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005ab0 tx=0x7f15f4005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:15.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:15.993+0000 7f15fbfff640 1 -- 172.21.15.66:0/1241053856 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15fc09d3e0 con 0x7f1604102a00 2024-04-21T09:08:16.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.199+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f160402d770 con 0x7f1604102a00 2024-04-21T09:08:16.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.200+0000 7f15fbfff640 1 -- 172.21.15.66:0/1241053856 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f15fc09cce0 con 0x7f1604102a00 2024-04-21T09:08:16.201 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f15f00a3d50 msgr2=0x7f15f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f15f00a3d50 0x7f15f00a6210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005ab0 tx=0x7f15f4005fd0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604102a00 msgr2=0x7f16041a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604102a00 0x7f16041a1810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00eb10 tx=0x7f15fc00ef10 comp rx=0 tx=0).stop 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 shutdown_connections 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f15f00a3d50 0x7f15f00a6210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1604102a00 0x7f16041a1810 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604102040 0x7f160419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 --2- 172.21.15.66:0/1241053856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1604100e60 0x7f160406b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 >> 172.21.15.66:0/1241053856 conn(0x7f16040fc440 msgr2=0x7f16041054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:16.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 shutdown_connections 2024-04-21T09:08:16.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.202+0000 7f16096d9640 1 -- 172.21.15.66:0/1241053856 wait complete. 2024-04-21T09:08:16.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:16.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 -- 172.21.15.66:0/2323652970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e0101b60 msgr2=0x7f08e0101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 --2- 172.21.15.66:0/2323652970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e0101b60 0x7f08e0101fe0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009ca0 tx=0x7f08d802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 -- 172.21.15.66:0/2323652970 shutdown_connections 2024-04-21T09:08:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 --2- 172.21.15.66:0/2323652970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e0101b60 0x7f08e0101fe0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 --2- 172.21.15.66:0/2323652970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08e01011c0 0x7f08e0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 --2- 172.21.15.66:0/2323652970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 0x7f08e0100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 -- 172.21.15.66:0/2323652970 >> 172.21.15.66:0/2323652970 conn(0x7f08e00fc400 msgr2=0x7f08e00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 -- 172.21.15.66:0/2323652970 shutdown_connections 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.326+0000 7f08e8481640 1 -- 172.21.15.66:0/2323652970 wait complete. 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 Processor -- start 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 -- start start 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 0x7f08e019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e01011c0 0x7f08e019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08e0101b60 0x7f08e01a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08e010fce0 con 0x7f08e01011c0 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08e010fb60 con 0x7f08e0100860 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e8481640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f08e010fe60 con 0x7f08e0101b60 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e5a1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e01011c0 0x7f08e019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e621d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 0x7f08e019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e5a1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e01011c0 0x7f08e019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32800/0 (socket says 172.21.15.66:32800) 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e621d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 0x7f08e019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:56996/0 (socket says 172.21.15.66:56996) 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e5a1c640 1 -- 172.21.15.66:0/1159584404 learned_addr learned my addr 172.21.15.66:0/1159584404 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e6a1e640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08e0101b60 0x7f08e01a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e621d640 1 -- 172.21.15.66:0/1159584404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08e0101b60 msgr2=0x7f08e01a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.327+0000 7f08e621d640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08e0101b60 0x7f08e01a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08e621d640 1 -- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e01011c0 msgr2=0x7f08e019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08e621d640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e01011c0 0x7f08e019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08e621d640 1 -- 172.21.15.66:0/1159584404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e01a1fc0 con 0x7f08e0100860 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08e5a1c640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e01011c0 0x7f08e019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08e621d640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 0x7f08e019a3a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e9f0 tx=0x7f08dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08d77fe640 1 -- 172.21.15.66:0/1159584404 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00ccc0 con 0x7f08e0100860 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e01a22b0 con 0x7f08e0100860 2024-04-21T09:08:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08d77fe640 1 -- 172.21.15.66:0/1159584404 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08dc0108e0 con 0x7f08e0100860 2024-04-21T09:08:16.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08d77fe640 1 -- 172.21.15.66:0/1159584404 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc018880 con 0x7f08e0100860 2024-04-21T09:08:16.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.328+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e01a2800 con 0x7f08e0100860 2024-04-21T09:08:16.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.329+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e010b780 con 0x7f08e0100860 2024-04-21T09:08:16.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.330+0000 7f08d77fe640 1 -- 172.21.15.66:0/1159584404 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f08dc010430 con 0x7f08e0100860 2024-04-21T09:08:16.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.331+0000 7f08d77fe640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08cc0a3d20 0x7f08cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.331+0000 7f08e5a1c640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08cc0a3d20 0x7f08cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.331+0000 7f08d77fe640 1 -- 172.21.15.66:0/1159584404 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f08dc0d0f40 con 0x7f08e0100860 2024-04-21T09:08:16.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.331+0000 7f08e5a1c640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08cc0a3d20 0x7f08cc0a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08e019b8b0 tx=0x7f08d0005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:16.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.334+0000 7f08d77fe640 1 -- 172.21.15.66:0/1159584404 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08dc09cc20 con 0x7f08e0100860 2024-04-21T09:08:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.525+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f08e0062b50 con 0x7f08e0100860 2024-04-21T09:08:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.525+0000 7f08d77fe640 1 -- 172.21.15.66:0/1159584404 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f08dc0a0f20 con 0x7f08e0100860 2024-04-21T09:08:16.526 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08cc0a3d20 msgr2=0x7f08cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08cc0a3d20 0x7f08cc0a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08e019b8b0 tx=0x7f08d0005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 msgr2=0x7f08e019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 0x7f08e019a3a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e9f0 tx=0x7f08dc00eeb0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 shutdown_connections 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08cc0a3d20 0x7f08cc0a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08e0101b60 0x7f08e01a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e01011c0 0x7f08e019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 --2- 172.21.15.66:0/1159584404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0100860 0x7f08e019a3a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 >> 172.21.15.66:0/1159584404 conn(0x7f08e00fc400 msgr2=0x7f08e00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:16.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 shutdown_connections 2024-04-21T09:08:16.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.527+0000 7f08e8481640 1 -- 172.21.15.66:0/1159584404 wait complete. 2024-04-21T09:08:16.539 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:16.539 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:16.539 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.539 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.539 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.539 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.539 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.540 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2459585789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.541 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:08:16.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:08:16.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3347648933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:16.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1241053856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:16.543 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-21T09:08:16.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2459585789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:08:16.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:08:16.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3347648933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:16.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1241053856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-21T09:08:16.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2459585789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:08:16.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:08:16.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:08:16.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3347648933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:16.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1241053856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 -- 172.21.15.66:0/2010200519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec10c510 msgr2=0x7f37ec10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 --2- 172.21.15.66:0/2010200519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec10c510 0x7f37ec10e9b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009ca0 tx=0x7f37e402c230 comp rx=0 tx=0).stop 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 -- 172.21.15.66:0/2010200519 shutdown_connections 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 --2- 172.21.15.66:0/2010200519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec10c510 0x7f37ec10e9b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 --2- 172.21.15.66:0/2010200519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec103af0 0x7f37ec10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 --2- 172.21.15.66:0/2010200519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec1031d0 0x7f37ec1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 -- 172.21.15.66:0/2010200519 >> 172.21.15.66:0/2010200519 conn(0x7f37ec077f50 msgr2=0x7f37ec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 -- 172.21.15.66:0/2010200519 shutdown_connections 2024-04-21T09:08:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.653+0000 7f37f0c70640 1 -- 172.21.15.66:0/2010200519 wait complete. 2024-04-21T09:08:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 Processor -- start 2024-04-21T09:08:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 -- start start 2024-04-21T09:08:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1031d0 0x7f37ec100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec103af0 0x7f37ec101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec10c510 0x7f37ec1018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec112180 con 0x7f37ec103af0 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec112000 con 0x7f37ec1031d0 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37f0c70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec112300 con 0x7f37ec10c510 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37e9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec103af0 0x7f37ec101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37ea59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1031d0 0x7f37ec100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37e9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec103af0 0x7f37ec101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32828/0 (socket says 172.21.15.66:32828) 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37e9d9b640 1 -- 172.21.15.66:0/1189356061 learned_addr learned my addr 172.21.15.66:0/1189356061 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37ead9d640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec10c510 0x7f37ec1018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37ea59c640 1 -- 172.21.15.66:0/1189356061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec10c510 msgr2=0x7f37ec1018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37ea59c640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec10c510 0x7f37ec1018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37ea59c640 1 -- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec103af0 msgr2=0x7f37ec101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37ea59c640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec103af0 0x7f37ec101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.654+0000 7f37ea59c640 1 -- 172.21.15.66:0/1189356061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec1a6be0 con 0x7f37ec1031d0 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.655+0000 7f37ea59c640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1031d0 0x7f37ec100e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d6d0 tx=0x7f37dc00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.655+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc017070 con 0x7f37ec1031d0 2024-04-21T09:08:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.655+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37dc009820 con 0x7f37ec1031d0 2024-04-21T09:08:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.655+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1a6ed0 con 0x7f37ec1031d0 2024-04-21T09:08:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.655+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc004c40 con 0x7f37ec1031d0 2024-04-21T09:08:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.655+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a7390 con 0x7f37ec1031d0 2024-04-21T09:08:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.656+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec068d70 con 0x7f37ec1031d0 2024-04-21T09:08:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.657+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f37dc005220 con 0x7f37ec1031d0 2024-04-21T09:08:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.658+0000 7f37db7fe640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37d40a3d20 0x7f37d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.658+0000 7f37e9d9b640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37d40a3d20 0x7f37d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.658+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f37dc0d07c0 con 0x7f37ec1031d0 2024-04-21T09:08:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.658+0000 7f37e9d9b640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37d40a3d20 0x7f37d40a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37ec1028b0 tx=0x7f37e0005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.661+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37dc09c460 con 0x7f37ec1031d0 2024-04-21T09:08:16.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.683+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f37dc098ad0 con 0x7f37ec1031d0 2024-04-21T09:08:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.876+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f37ec107690 con 0x7f37ec1031d0 2024-04-21T09:08:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.877+0000 7f37db7fe640 1 -- 172.21.15.66:0/1189356061 <== mon.2 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f37dc09cb60 con 0x7f37ec1031d0 2024-04-21T09:08:16.878 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37d40a3d20 msgr2=0x7f37d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37d40a3d20 0x7f37d40a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37ec1028b0 tx=0x7f37e0005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1031d0 msgr2=0x7f37ec100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1031d0 0x7f37ec100e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d6d0 tx=0x7f37dc00db90 comp rx=0 tx=0).stop 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 shutdown_connections 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37d40a3d20 0x7f37d40a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec10c510 0x7f37ec1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ec103af0 0x7f37ec101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 --2- 172.21.15.66:0/1189356061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1031d0 0x7f37ec100e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 >> 172.21.15.66:0/1189356061 conn(0x7f37ec077f50 msgr2=0x7f37ec0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 shutdown_connections 2024-04-21T09:08:16.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.879+0000 7f37f0c70640 1 -- 172.21.15.66:0/1189356061 wait complete. 2024-04-21T09:08:16.891 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 -- 172.21.15.66:0/491682352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78100e50 msgr2=0x7f8b781012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/491682352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78100e50 0x7f8b781012b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009900 tx=0x7f8b6c02bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 -- 172.21.15.66:0/491682352 shutdown_connections 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/491682352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b781017f0 0x7f8b7810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/491682352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78100e50 0x7f8b781012b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/491682352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b78106f10 0x7f8b781072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 -- 172.21.15.66:0/491682352 >> 172.21.15.66:0/491682352 conn(0x7f8b78077f80 msgr2=0x7f8b78078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 -- 172.21.15.66:0/491682352 shutdown_connections 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 -- 172.21.15.66:0/491682352 wait complete. 2024-04-21T09:08:16.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 Processor -- start 2024-04-21T09:08:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.994+0000 7f8b7e53d640 1 -- start start 2024-04-21T09:08:16.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7e53d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b78100e50 0x7f8b78198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7e53d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b781017f0 0x7f8b78198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7e53d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78106f10 0x7f8b7819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7e53d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781121d0 con 0x7f8b78106f10 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7e53d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b78112050 con 0x7f8b78100e50 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7e53d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8b78112350 con 0x7f8b781017f0 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78106f10 0x7f8b7819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78106f10 0x7f8b7819cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32850/0 (socket says 172.21.15.66:32850) 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 -- 172.21.15.66:0/2577246335 learned_addr learned my addr 172.21.15.66:0/2577246335 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b77fff640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b78100e50 0x7f8b78198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b777fe640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b781017f0 0x7f8b78198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 -- 172.21.15.66:0/2577246335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b781017f0 msgr2=0x7f8b78198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b781017f0 0x7f8b78198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 -- 172.21.15.66:0/2577246335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b78100e50 msgr2=0x7f8b78198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b78100e50 0x7f8b78198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b7cada640 1 -- 172.21.15.66:0/2577246335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7819d6b0 con 0x7f8b78106f10 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.995+0000 7f8b77fff640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b78100e50 0x7f8b78198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.996+0000 7f8b7cada640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78106f10 0x7f8b7819cf80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000da40 tx=0x7f8b7000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.996+0000 7f8b757fa640 1 -- 172.21.15.66:0/2577246335 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70017070 con 0x7f8b78106f10 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.996+0000 7f8b757fa640 1 -- 172.21.15.66:0/2577246335 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b70003e20 con 0x7f8b78106f10 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.996+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7819d9a0 con 0x7f8b78106f10 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.996+0000 7f8b757fa640 1 -- 172.21.15.66:0/2577246335 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70010460 con 0x7f8b78106f10 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.996+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b781a5230 con 0x7f8b78106f10 2024-04-21T09:08:16.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.997+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b78068550 con 0x7f8b78106f10 2024-04-21T09:08:17.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.998+0000 7f8b757fa640 1 -- 172.21.15.66:0/2577246335 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f8b70004000 con 0x7f8b78106f10 2024-04-21T09:08:17.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.999+0000 7f8b757fa640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b600a3db0 0x7f8b600a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.999+0000 7f8b77fff640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b600a3db0 0x7f8b600a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.999+0000 7f8b757fa640 1 -- 172.21.15.66:0/2577246335 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8b700d0690 con 0x7f8b78106f10 2024-04-21T09:08:17.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:16.999+0000 7f8b77fff640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b600a3db0 0x7f8b600a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009f40 tx=0x7f8b68005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.004+0000 7f8b757fa640 1 -- 172.21.15.66:0/2577246335 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b7009c200 con 0x7f8b78106f10 2024-04-21T09:08:17.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.233+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b78109360 con 0x7f8b78106f10 2024-04-21T09:08:17.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.233+0000 7f8b757fa640 1 -- 172.21.15.66:0/2577246335 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f8b700a0500 con 0x7f8b78106f10 2024-04-21T09:08:17.234 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:17.234 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:08:17.234 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b600a3db0 msgr2=0x7f8b600a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b600a3db0 0x7f8b600a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009f40 tx=0x7f8b68005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78106f10 msgr2=0x7f8b7819cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78106f10 0x7f8b7819cf80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000da40 tx=0x7f8b7000df00 comp rx=0 tx=0).stop 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 shutdown_connections 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b600a3db0 0x7f8b600a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b78106f10 0x7f8b7819cf80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b781017f0 0x7f8b78198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 --2- 172.21.15.66:0/2577246335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b78100e50 0x7f8b78198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 >> 172.21.15.66:0/2577246335 conn(0x7f8b78077f80 msgr2=0x7f8b780ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 shutdown_connections 2024-04-21T09:08:17.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.235+0000 7f8b7e53d640 1 -- 172.21.15.66:0/2577246335 wait complete. 2024-04-21T09:08:17.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:17.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 -- 172.21.15.66:0/3798840065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c901031d0 msgr2=0x7f4c901035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 --2- 172.21.15.66:0/3798840065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c901031d0 0x7f4c901035b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c8402bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 -- 172.21.15.66:0/3798840065 shutdown_connections 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 --2- 172.21.15.66:0/3798840065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c9010c5a0 0x7f4c9010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 --2- 172.21.15.66:0/3798840065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90103af0 0x7f4c9010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 --2- 172.21.15.66:0/3798840065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c901031d0 0x7f4c901035b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 -- 172.21.15.66:0/3798840065 >> 172.21.15.66:0/3798840065 conn(0x7f4c900771d0 msgr2=0x7f4c900775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 -- 172.21.15.66:0/3798840065 shutdown_connections 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 -- 172.21.15.66:0/3798840065 wait complete. 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.349+0000 7f4c95c21640 1 Processor -- start 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c95c21640 1 -- start start 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c95c21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c901031d0 0x7f4c9019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c95c21640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c90103af0 0x7f4c9019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c95c21640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c9010c5a0 0x7f4c901985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c95c21640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901121f0 con 0x7f4c90103af0 2024-04-21T09:08:17.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c95c21640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90112070 con 0x7f4c901031d0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c95c21640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90112370 con 0x7f4c9010c5a0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c90103af0 0x7f4c9019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c90103af0 0x7f4c9019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32866/0 (socket says 172.21.15.66:32866) 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 -- 172.21.15.66:0/2608037512 learned_addr learned my addr 172.21.15.66:0/2608037512 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8ffff640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c9010c5a0 0x7f4c901985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8f7fe640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c901031d0 0x7f4c9019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 -- 172.21.15.66:0/2608037512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c9010c5a0 msgr2=0x7f4c901985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c9010c5a0 0x7f4c901985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 -- 172.21.15.66:0/2608037512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c901031d0 msgr2=0x7f4c9019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c901031d0 0x7f4c9019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 -- 172.21.15.66:0/2608037512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c90198d30 con 0x7f4c90103af0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8effd640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c90103af0 0x7f4c9019e940 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d950 tx=0x7f4c8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.350+0000 7f4c8cff9640 1 -- 172.21.15.66:0/2608037512 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80017070 con 0x7f4c90103af0 2024-04-21T09:08:17.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.351+0000 7f4c8cff9640 1 -- 172.21.15.66:0/2608037512 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c80009c30 con 0x7f4c90103af0 2024-04-21T09:08:17.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.351+0000 7f4c8cff9640 1 -- 172.21.15.66:0/2608037512 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000f620 con 0x7f4c90103af0 2024-04-21T09:08:17.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.351+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c90199020 con 0x7f4c90103af0 2024-04-21T09:08:17.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.351+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c901a52c0 con 0x7f4c90103af0 2024-04-21T09:08:17.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.352+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c90069230 con 0x7f4c90103af0 2024-04-21T09:08:17.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.353+0000 7f4c8cff9640 1 -- 172.21.15.66:0/2608037512 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f4c80010460 con 0x7f4c90103af0 2024-04-21T09:08:17.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.353+0000 7f4c8cff9640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c780a3e30 0x7f4c780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.353+0000 7f4c8f7fe640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c780a3e30 0x7f4c780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.353+0000 7f4c8cff9640 1 -- 172.21.15.66:0/2608037512 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4c800d0840 con 0x7f4c90103af0 2024-04-21T09:08:17.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.354+0000 7f4c8f7fe640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c780a3e30 0x7f4c780a62f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c84006010 tx=0x7f4c84005e40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.358+0000 7f4c8cff9640 1 -- 172.21.15.66:0/2608037512 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c80009de0 con 0x7f4c90103af0 2024-04-21T09:08:17.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.545+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c901076c0 con 0x7f4c90103af0 2024-04-21T09:08:17.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.547+0000 7f4c8cff9640 1 -- 172.21.15.66:0/2608037512 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+25086 (secure 0 0 0) 0x7f4c8009c3b0 con 0x7f4c90103af0 2024-04-21T09:08:17.547 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:17.547 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:08:12.028415+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:17.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c780a3e30 msgr2=0x7f4c780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c780a3e30 0x7f4c780a62f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c84006010 tx=0x7f4c84005e40 comp rx=0 tx=0).stop 2024-04-21T09:08:17.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c90103af0 msgr2=0x7f4c9019e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c90103af0 0x7f4c9019e940 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d950 tx=0x7f4c8000de10 comp rx=0 tx=0).stop 2024-04-21T09:08:17.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 shutdown_connections 2024-04-21T09:08:17.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c780a3e30 0x7f4c780a62f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c9010c5a0 0x7f4c901985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c90103af0 0x7f4c9019e940 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 --2- 172.21.15.66:0/2608037512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c901031d0 0x7f4c9019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 >> 172.21.15.66:0/2608037512 conn(0x7f4c900771d0 msgr2=0x7f4c900ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 shutdown_connections 2024-04-21T09:08:17.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.549+0000 7f4c95c21640 1 -- 172.21.15.66:0/2608037512 wait complete. 2024-04-21T09:08:17.559 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Unable to set osd_memory_target on smithi066 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1159584404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1189356061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.561 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2577246335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:17.563 DEBUG:tasks.fwd_scrub:Yielding 2024-04-21T09:08:17.563 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T09:08:17.573 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T09:08:17.573 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:08:17.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-21T09:08:17.573 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-21T09:08:17.573 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:08:17.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:17.574 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Unable to set osd_memory_target on smithi066 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1159584404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1189356061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:17.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:08:17.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:08:17.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:08:17.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2577246335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Unable to set osd_memory_target on smithi066 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1159584404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: Updating smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1189356061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:17.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:08:17.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:08:17.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:08:17.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:08:17.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2577246335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 -- 172.21.15.66:0/2724397464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 msgr2=0x7f98d0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 --2- 172.21.15.66:0/2724397464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d0101140 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009900 tx=0x7f98cc02bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 -- 172.21.15.66:0/2724397464 shutdown_connections 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 --2- 172.21.15.66:0/2724397464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d010e910 0x7f98d0110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 --2- 172.21.15.66:0/2724397464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d0101680 0x7f98d010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 --2- 172.21.15.66:0/2724397464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d0101140 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 -- 172.21.15.66:0/2724397464 >> 172.21.15.66:0/2724397464 conn(0x7f98d0077fd0 msgr2=0x7f98d00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 -- 172.21.15.66:0/2724397464 shutdown_connections 2024-04-21T09:08:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f98d7281640 1 -- 172.21.15.66:0/2724397464 wait complete. 2024-04-21T09:08:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f1975def640 1 -- 172.21.15.66:0/3973517163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 msgr2=0x7f1970101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f1975def640 1 --2- 172.21.15.66:0/3973517163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 0x7f1970101470 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1960009900 tx=0x7f196002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f1975def640 1 -- 172.21.15.66:0/3973517163 shutdown_connections 2024-04-21T09:08:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f1975def640 1 --2- 172.21.15.66:0/3973517163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1970106160 0x7f197010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f1975def640 1 --2- 172.21.15.66:0/3973517163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1970101a40 0x7f1970105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f1975def640 1 --2- 172.21.15.66:0/3973517163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 0x7f1970101470 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.679+0000 7f1975def640 1 -- 172.21.15.66:0/3973517163 >> 172.21.15.66:0/3973517163 conn(0x7f1970077f50 msgr2=0x7f1970078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f1975def640 1 -- 172.21.15.66:0/3973517163 shutdown_connections 2024-04-21T09:08:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f1975def640 1 -- 172.21.15.66:0/3973517163 wait complete. 2024-04-21T09:08:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 Processor -- start 2024-04-21T09:08:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 -- start start 2024-04-21T09:08:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d0101680 0x7f98d01a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f1975def640 1 Processor -- start 2024-04-21T09:08:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d010e910 0x7f98d019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98d01143e0 con 0x7f98d0100d60 2024-04-21T09:08:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0114260 con 0x7f98d010e910 2024-04-21T09:08:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d7281640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0114560 con 0x7f98d0101680 2024-04-21T09:08:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d501d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d501d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d01a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32894/0 (socket says 172.21.15.66:32894) 2024-04-21T09:08:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d501d640 1 -- 172.21.15.66:0/2183987044 learned_addr learned my addr 172.21.15.66:0/2183987044 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f1975def640 1 -- start start 2024-04-21T09:08:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d581e640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d010e910 0x7f98d019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.680+0000 7f98d481c640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d0101680 0x7f98d01a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d501d640 1 -- 172.21.15.66:0/2183987044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d0101680 msgr2=0x7f98d01a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d501d640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d0101680 0x7f98d01a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d501d640 1 -- 172.21.15.66:0/2183987044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d010e910 msgr2=0x7f98d019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d501d640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d010e910 0x7f98d019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d501d640 1 -- 172.21.15.66:0/2183987044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d019afe0 con 0x7f98d0100d60 2024-04-21T09:08:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f1975def640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 0x7f197010e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f1975def640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1970106160 0x7f197010ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f1975def640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f197010f140 0x7f19701a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f1975def640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19701121e0 con 0x7f1970101090 2024-04-21T09:08:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f1975def640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1970112060 con 0x7f1970106160 2024-04-21T09:08:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f1975def640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1970112360 con 0x7f197010f140 2024-04-21T09:08:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 0x7f197010e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 0x7f197010e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32898/0 (socket says 172.21.15.66:32898) 2024-04-21T09:08:17.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196f7fe640 1 -- 172.21.15.66:0/2520475950 learned_addr learned my addr 172.21.15.66:0/2520475950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196effd640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1970106160 0x7f197010ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d501d640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d01a0660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005940 tx=0x7f98cc02c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196ffff640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f197010f140 0x7f19701a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98c67fc640 1 -- 172.21.15.66:0/2183987044 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc043020 con 0x7f98d0100d60 2024-04-21T09:08:17.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196effd640 1 -- 172.21.15.66:0/2520475950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f197010f140 msgr2=0x7f19701a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196effd640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f197010f140 0x7f19701a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196effd640 1 -- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 msgr2=0x7f197010e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196effd640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 0x7f197010e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98c67fc640 1 -- 172.21.15.66:0/2183987044 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc0042d0 con 0x7f98d0100d60 2024-04-21T09:08:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98c67fc640 1 -- 172.21.15.66:0/2183987044 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc003ab0 con 0x7f98d0100d60 2024-04-21T09:08:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196effd640 1 -- 172.21.15.66:0/2520475950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19701a7050 con 0x7f1970106160 2024-04-21T09:08:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d019b270 con 0x7f98d0100d60 2024-04-21T09:08:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d01a7400 con 0x7f98d0100d60 2024-04-21T09:08:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.681+0000 7f196effd640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1970106160 0x7f197010ec00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f196400e9c0 tx=0x7f196400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.682+0000 7f196cff9640 1 -- 172.21.15.66:0/2520475950 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196400ccc0 con 0x7f1970106160 2024-04-21T09:08:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.682+0000 7f196cff9640 1 -- 172.21.15.66:0/2520475950 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640108e0 con 0x7f1970106160 2024-04-21T09:08:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.682+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701a7340 con 0x7f1970106160 2024-04-21T09:08:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.682+0000 7f196cff9640 1 -- 172.21.15.66:0/2520475950 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640187c0 con 0x7f1970106160 2024-04-21T09:08:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.682+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701a78e0 con 0x7f1970106160 2024-04-21T09:08:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.682+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d0069e00 con 0x7f98d0100d60 2024-04-21T09:08:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 Processor -- start 2024-04-21T09:08:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 -- start start 2024-04-21T09:08:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f98c67fc640 1 -- 172.21.15.66:0/2183987044 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f98cc046020 con 0x7f98d0100d60 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce6ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6d290 0x55771ce75840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce75e10 0x55771ce78260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55771cdd0f00 con 0x55771ce75e10 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55771cd6e590 con 0x55771ce6c8e0 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.684+0000 7f19789da740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55771cdccec0 con 0x55771ce6d290 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce6ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196cff9640 1 -- 172.21.15.66:0/2520475950 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f1964010430 con 0x7f1970106160 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce6ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:57106/0 (socket says 172.21.15.66:57106) 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 -- 172.21.15.66:0/983558027 learned_addr learned my addr 172.21.15.66:0/983558027 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196f7fe640 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6d290 0x55771ce75840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196effd640 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce75e10 0x55771ce78260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 -- 172.21.15.66:0/983558027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6d290 msgr2=0x55771ce75840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6d290 0x55771ce75840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 -- 172.21.15.66:0/983558027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce75e10 msgr2=0x55771ce78260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce75e10 0x55771ce78260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 -- 172.21.15.66:0/983558027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771cbab7e0 con 0x55771ce6c8e0 2024-04-21T09:08:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196effd640 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce75e10 0x55771ce78260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f98c67fc640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3ee0 0x7f98bc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196ffff640 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce6ccc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1968009f50 tx=0x7f19680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9adcb2c5b34969a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f1955ffb640 1 -- 172.21.15.66:0/983558027 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968004d20 con 0x55771ce6c8e0 2024-04-21T09:08:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f98d481c640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3ee0 0x7f98bc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f98c67fc640 1 -- 172.21.15.66:0/2183987044 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f98cc0f2c40 con 0x7f98d0100d60 2024-04-21T09:08:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f1955ffb640 1 -- 172.21.15.66:0/983558027 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1968004050 con 0x55771ce6c8e0 2024-04-21T09:08:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f1955ffb640 1 -- 172.21.15.66:0/983558027 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968005280 con 0x55771ce6c8e0 2024-04-21T09:08:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.685+0000 7f196cff9640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19580a3e60 0x7f19580a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19580a3e60 0x7f19580a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f196cff9640 1 -- 172.21.15.66:0/2520475950 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f19640d0fa0 con 0x7f1970106160 2024-04-21T09:08:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f98d481c640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3ee0 0x7f98bc0a63a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d019bf70 tx=0x7f98c0005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 -- 172.21.15.66:0/983558027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 msgr2=0x55771ce6ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce6ccc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1968009f50 tx=0x7f19680118d0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19580a3e60 0x7f19580a6320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1960000c00 tx=0x7f19600058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 -- 172.21.15.66:0/983558027 shutdown_connections 2024-04-21T09:08:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce75e10 0x55771ce78260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6d290 0x55771ce75840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 --2- 172.21.15.66:0/983558027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce6ccc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 -- 172.21.15.66:0/983558027 >> 172.21.15.66:0/983558027 conn(0x55771ce66fb0 msgr2=0x55771ce67390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 -- 172.21.15.66:0/983558027 shutdown_connections 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.686+0000 7f19789da740 1 -- 172.21.15.66:0/983558027 wait complete. 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 -- start start 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 Processor -- start 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce72eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce73930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55771ce78d80 con 0x55771ce6d290 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55771cd8a580 con 0x55771ce6c8e0 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55771ce78f00 con 0x55771ce75e10 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f19789da740 10 client.0 _ll_register_callbacks cb 0x55771cd90650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f196effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f196f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce72eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f196effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32920/0 (socket says 172.21.15.66:32920) 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f196f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce72eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:57118/0 (socket says 172.21.15.66:57118) 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f196effd640 1 -- 172.21.15.66:0/2910300471 learned_addr learned my addr 172.21.15.66:0/2910300471 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.687+0000 7f196ffff640 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce73930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f196effd640 1 -- 172.21.15.66:0/2910300471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 msgr2=0x55771ce73930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f196effd640 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce73930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f196effd640 1 -- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 msgr2=0x55771ce72eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f196effd640 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce72eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f196effd640 1 -- 172.21.15.66:0/2910300471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771cc0bf70 con 0x55771ce6d290 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce72eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f196effd640 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce733f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1964009560 tx=0x7f19640107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f1d731a107db41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.688+0000 7f194effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.690+0000 7f194effd640 1 -- 172.21.15.66:0/2910300471 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640a5690 con 0x55771ce6d290 2024-04-21T09:08:17.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.690+0000 7f194effd640 1 -- 172.21.15.66:0/2910300471 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f19640bbbe0 con 0x55771ce6d290 2024-04-21T09:08:17.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1968004050 con 0x55771ce6d290 2024-04-21T09:08:17.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f194effd640 1 -- 172.21.15.66:0/2910300471 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19640bbde0 con 0x55771ce6d290 2024-04-21T09:08:17.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f98c67fc640 1 -- 172.21.15.66:0/2183987044 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98cc04d020 con 0x7f98d0100d60 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55771caa3170 con 0x55771ce6d290 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f194effd640 1 -- 172.21.15.66:0/2910300471 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1968004050 con 0x55771ce6d290 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f194effd640 1 -- 172.21.15.66:0/2910300471 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1964098820 con 0x55771ce6d290 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 20 client.15660 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ce862e0 0x55771ce886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 4 client.15660 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:17.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55771cdde920 con 0x55771ce862e0 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.691+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ce862e0 0x55771ce886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.692+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ce862e0 0x55771ce886d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dbfe9b8ee14f34c server_cookie=b3ad4be59df12655 in_seq=0 out_seq=0 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.692+0000 7f194effd640 10 client.15660 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f194effd640 1 -- 172.21.15.66:0/2910300471 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55771cdde920 con 0x55771ce862e0 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f194effd640 10 client.15660 handle_command_reply: tid=0 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 client.15660 shutdown 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 msgr2=0x55771ce733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce733f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1964009560 tx=0x7f19640107e0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 shutdown_connections 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ce862e0 0x55771ce886d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce73930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce733f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f194effd640 0 client.15660 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 --2- 172.21.15.66:0/2910300471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6c8e0 0x55771ce72eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 >> 172.21.15.66:0/2910300471 conn(0x55771ce66fb0 msgr2=0x55771ce68d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 shutdown_connections 2024-04-21T09:08:17.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 1 -- 172.21.15.66:0/2910300471 wait complete. 2024-04-21T09:08:17.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.693+0000 7f19789da740 20 client.15660 trim_cache size 0 max 16384 2024-04-21T09:08:17.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 Processor -- start 2024-04-21T09:08:17.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 -- start start 2024-04-21T09:08:17.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6c8e0 0x55771ce755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6d290 0x55771ce81c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce68ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19701b3de0 con 0x55771ce6c8e0 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19701b16b0 con 0x55771ce6d290 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f19789da740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19701af4e0 con 0x55771ce75e10 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6c8e0 0x55771ce755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6c8e0 0x55771ce755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32930/0 (socket says 172.21.15.66:32930) 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 -- 172.21.15.66:0/4261953613 learned_addr learned my addr 172.21.15.66:0/4261953613 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196effd640 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6d290 0x55771ce81c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196ffff640 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce68ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 -- 172.21.15.66:0/4261953613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 msgr2=0x55771ce68ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce68ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 -- 172.21.15.66:0/4261953613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6d290 msgr2=0x55771ce81c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6d290 0x55771ce81c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.711+0000 7f196f7fe640 1 -- 172.21.15.66:0/4261953613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771ce71e60 con 0x55771ce6c8e0 2024-04-21T09:08:17.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.712+0000 7f196f7fe640 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6c8e0 0x55771ce755e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1960006010 tx=0x7f1960044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdc527b33e5cf01 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.712+0000 7f19557fa640 1 -- 172.21.15.66:0/4261953613 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196002cc40 con 0x55771ce6c8e0 2024-04-21T09:08:17.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.712+0000 7f19557fa640 1 -- 172.21.15.66:0/4261953613 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196005a070 con 0x55771ce6c8e0 2024-04-21T09:08:17.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.712+0000 7f19557fa640 1 -- 172.21.15.66:0/4261953613 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19600046c0 con 0x55771ce6c8e0 2024-04-21T09:08:17.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.712+0000 7f19789da740 1 -- 172.21.15.66:0/4261953613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6c8e0 msgr2=0x55771ce755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.712+0000 7f19789da740 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6c8e0 0x55771ce755e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1960006010 tx=0x7f1960044000 comp rx=0 tx=0).stop 2024-04-21T09:08:17.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- 172.21.15.66:0/4261953613 shutdown_connections 2024-04-21T09:08:17.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce75e10 0x55771ce68ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce6d290 0x55771ce81c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 --2- 172.21.15.66:0/4261953613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6c8e0 0x55771ce755e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- 172.21.15.66:0/4261953613 >> 172.21.15.66:0/4261953613 conn(0x55771ce66fb0 msgr2=0x55771ce683b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- 172.21.15.66:0/4261953613 shutdown_connections 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- 172.21.15.66:0/4261953613 wait complete. 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- start start 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 Processor -- start 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6c8e0 0x55771ce6fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce70330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce75e10 0x55771ce70870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19701afb70 con 0x55771ce6d290 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19701b0fe0 con 0x55771ce75e10 2024-04-21T09:08:17.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19701b0240 con 0x55771ce6c8e0 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.713+0000 7f19789da740 10 client.0 _ll_register_callbacks cb 0x7f19640ce110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce70330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce70330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32940/0 (socket says 172.21.15.66:32940) 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 -- 172.21.15.66:0/2974940921 learned_addr learned my addr 172.21.15.66:0/2974940921 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196ffff640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce75e10 0x55771ce70870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 -- 172.21.15.66:0/2974940921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6c8e0 msgr2=0x55771ce6fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6c8e0 0x55771ce6fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6c8e0 0x55771ce6fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 -- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce75e10 msgr2=0x55771ce70870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce75e10 0x55771ce70870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 -- 172.21.15.66:0/2974940921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771ceeba80 con 0x55771ce6d290 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196ffff640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce75e10 0x55771ce70870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f196effd640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce70330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1964098d10 tx=0x7f196409afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b61ee70c757dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f194effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f194effd640 1 -- 172.21.15.66:0/2974940921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196409c8a0 con 0x55771ce6d290 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f194effd640 1 -- 172.21.15.66:0/2974940921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f19640ce740 con 0x55771ce6d290 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196005a070 con 0x55771ce6d290 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f194effd640 1 -- 172.21.15.66:0/2974940921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19640bd070 con 0x55771ce6d290 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f19789da740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.714+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55771ceecda0 con 0x55771ce6d290 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f194effd640 1 -- 172.21.15.66:0/2974940921 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196005a070 con 0x55771ce6d290 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f194effd640 1 -- 172.21.15.66:0/2974940921 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f19640cea90 con 0x55771ce6d290 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f19789da740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f19789da740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f19789da740 20 client.15666 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f19789da740 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ceed330 0x55771ceef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f19789da740 4 client.15666 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:08:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55771cdd8790 con 0x55771ceed330 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ceed330 0x55771ceef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f194effd640 10 client.15666 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.715+0000 7f196f7fe640 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ceed330 0x55771ceef720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406df29e62f0445b server_cookie=5894883f1ca900ab in_seq=0 out_seq=0 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f194effd640 1 -- 172.21.15.66:0/2974940921 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55771cdd8790 con 0x55771ceed330 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f194effd640 10 client.15666 handle_command_reply: tid=0 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 client.15666 shutdown 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 msgr2=0x55771ce70330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce70330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1964098d10 tx=0x7f196409afe0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 shutdown_connections 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55771ceed330 0x55771ceef720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55771ce75e10 0x55771ce70870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55771ce6d290 0x55771ce70330 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 --2- 172.21.15.66:0/2974940921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55771ce6c8e0 0x55771ce6fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f194effd640 0 client.15666 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 >> 172.21.15.66:0/2974940921 conn(0x55771ce66fb0 msgr2=0x55771ce73700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 shutdown_connections 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 1 -- 172.21.15.66:0/2974940921 wait complete. 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.717+0000 7f19789da740 20 client.15666 trim_cache size 0 max 16384 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "dbc43119-7654-4f8f-9e8b-3c9ab403fe7f", 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:08:17.723 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19580a3e60 msgr2=0x7f19580a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19580a3e60 0x7f19580a6320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1960000c00 tx=0x7f19600058d0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1970106160 msgr2=0x7f197010ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1970106160 0x7f197010ec00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f196400e9c0 tx=0x7f196400ee80 comp rx=0 tx=0).stop 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 shutdown_connections 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19580a3e60 0x7f19580a6320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f197010f140 0x7f19701a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1970106160 0x7f197010ec00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 --2- 172.21.15.66:0/2520475950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1970101090 0x7f197010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 >> 172.21.15.66:0/2520475950 conn(0x7f1970077f50 msgr2=0x7f19700ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 shutdown_connections 2024-04-21T09:08:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.718+0000 7f1975def640 1 -- 172.21.15.66:0/2520475950 wait complete. 2024-04-21T09:08:17.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:08:17.729 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:08:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.833+0000 7faa4a631640 1 -- 172.21.15.66:0/161379782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 msgr2=0x7faa4410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.833+0000 7faa4a631640 1 --2- 172.21.15.66:0/161379782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa4410e9a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009ca0 tx=0x7faa3c02c230 comp rx=0 tx=0).stop 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.833+0000 7faa4a631640 1 -- 172.21.15.66:0/161379782 shutdown_connections 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.833+0000 7faa4a631640 1 --2- 172.21.15.66:0/161379782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa4410e9a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.833+0000 7faa4a631640 1 --2- 172.21.15.66:0/161379782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa44103ae0 0x7faa4410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.833+0000 7faa4a631640 1 --2- 172.21.15.66:0/161379782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa441031c0 0x7faa441035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.833+0000 7faa4a631640 1 -- 172.21.15.66:0/161379782 >> 172.21.15.66:0/161379782 conn(0x7faa44078000 msgr2=0x7faa44078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 -- 172.21.15.66:0/161379782 shutdown_connections 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 -- 172.21.15.66:0/161379782 wait complete. 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 Processor -- start 2024-04-21T09:08:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 -- start start 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031c0 0x7faa44100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103ae0 0x7faa44101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa44101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa44112170 con 0x7faa4410c500 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa44111ff0 con 0x7faa44103ae0 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.834+0000 7faa4a631640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faa441122f0 con 0x7faa441031c0 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa44101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa44101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32946/0 (socket says 172.21.15.66:32946) 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 -- 172.21.15.66:0/850076281 learned_addr learned my addr 172.21.15.66:0/850076281 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa43fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031c0 0x7faa44100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa437fe640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103ae0 0x7faa44101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 -- 172.21.15.66:0/850076281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031c0 msgr2=0x7faa44100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031c0 0x7faa44100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 -- 172.21.15.66:0/850076281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103ae0 msgr2=0x7faa44101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103ae0 0x7faa44101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 -- 172.21.15.66:0/850076281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa441a6bb0 con 0x7faa4410c500 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa48bce640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa44101890 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faa3c02c840 tx=0x7faa3c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa417fa640 1 -- 172.21.15.66:0/850076281 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c039040 con 0x7faa4410c500 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa417fa640 1 -- 172.21.15.66:0/850076281 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c044070 con 0x7faa4410c500 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa441a6e40 con 0x7faa4410c500 2024-04-21T09:08:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa417fa640 1 -- 172.21.15.66:0/850076281 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0329c0 con 0x7faa4410c500 2024-04-21T09:08:17.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.835+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa441a7300 con 0x7faa4410c500 2024-04-21T09:08:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.838+0000 7faa417fa640 1 -- 172.21.15.66:0/850076281 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7faa3c032b60 con 0x7faa4410c500 2024-04-21T09:08:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.838+0000 7faa417fa640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa2c0a3db0 0x7faa2c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.838+0000 7faa43fff640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa2c0a3db0 0x7faa2c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.838+0000 7faa417fa640 1 -- 172.21.15.66:0/850076281 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7faa3c0f3120 con 0x7faa4410c500 2024-04-21T09:08:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 Processor -- start 2024-04-21T09:08:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa43fff640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa2c0a3db0 0x7faa2c0a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa3800bbe0 tx=0x7faa3800b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 -- start start 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda875960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875ea0 0x55fcda87e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda87ea80 0x55fcda880e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fcda7da1e0 con 0x55fcda875ea0 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fcda777390 con 0x55fcda87ea80 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa4d226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fcda7d61a0 con 0x55fcda875580 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875ea0 0x55fcda87e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875ea0 0x55fcda87e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32960/0 (socket says 172.21.15.66:32960) 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa437fe640 1 -- 172.21.15.66:0/2959421505 learned_addr learned my addr 172.21.15.66:0/2959421505 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa48bce640 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda87ea80 0x55fcda880e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.839+0000 7faa43fff640 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda875960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa437fe640 1 -- 172.21.15.66:0/2959421505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 msgr2=0x55fcda875960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa437fe640 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda875960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa437fe640 1 -- 172.21.15.66:0/2959421505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda87ea80 msgr2=0x55fcda880e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa437fe640 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda87ea80 0x55fcda880e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa437fe640 1 -- 172.21.15.66:0/2959421505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcda5b67d0 con 0x55fcda875ea0 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa48bce640 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda87ea80 0x55fcda880e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa437fe640 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875ea0 0x55fcda87e540 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faa3400bc10 tx=0x7faa3400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf1c9ab9817af10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa2a7fc640 1 -- 172.21.15.66:0/2959421505 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34015070 con 0x55fcda875ea0 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa2a7fc640 1 -- 172.21.15.66:0/2959421505 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa340118f0 con 0x55fcda875ea0 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.840+0000 7faa2a7fc640 1 -- 172.21.15.66:0/2959421505 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa340107d0 con 0x55fcda875ea0 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- 172.21.15.66:0/2959421505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875ea0 msgr2=0x55fcda87e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875ea0 0x55fcda87e540 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faa3400bc10 tx=0x7faa3400f730 comp rx=0 tx=0).stop 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- 172.21.15.66:0/2959421505 shutdown_connections 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda87ea80 0x55fcda880e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875ea0 0x55fcda87e540 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 --2- 172.21.15.66:0/2959421505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda875960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- 172.21.15.66:0/2959421505 >> 172.21.15.66:0/2959421505 conn(0x55fcda86fcb0 msgr2=0x55fcda870090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- 172.21.15.66:0/2959421505 shutdown_connections 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- 172.21.15.66:0/2959421505 wait complete. 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- start start 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 Processor -- start 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda87bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda87c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda87c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fcda881870 con 0x55fcda87ea80 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fcda7932f0 con 0x55fcda875ea0 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.841+0000 7faa4d226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fcda8819f0 con 0x55fcda875580 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa4d226740 10 client.0 _ll_register_callbacks cb 0x55fcda7a0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda87c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda87c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32966/0 (socket says 172.21.15.66:32966) 2024-04-21T09:08:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 -- 172.21.15.66:0/2711849236 learned_addr learned my addr 172.21.15.66:0/2711849236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa437fe640 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda87c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa43fff640 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda87bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 -- 172.21.15.66:0/2711849236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 msgr2=0x55fcda87bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda87bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 -- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 msgr2=0x55fcda87c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda87c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 -- 172.21.15.66:0/2711849236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcda4ec3f0 con 0x55fcda87ea80 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa48bce640 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda87c550 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0be9f0 tx=0x7faa3c0be650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c68cabf560bff3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa237fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa237fe640 1 -- 172.21.15.66:0/2711849236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0d8060 con 0x55fcda87ea80 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa237fe640 1 -- 172.21.15.66:0/2711849236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7faa3c0bef50 con 0x55fcda87ea80 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa340118f0 con 0x55fcda87ea80 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa237fe640 1 -- 172.21.15.66:0/2711849236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa3c0f0900 con 0x55fcda87ea80 2024-04-21T09:08:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa4d226740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.842+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fcda4ac0f0 con 0x55fcda87ea80 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa237fe640 1 -- 172.21.15.66:0/2711849236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa340118f0 con 0x55fcda87ea80 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa237fe640 1 -- 172.21.15.66:0/2711849236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faa3c03c030 con 0x55fcda87ea80 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa4d226740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa4d226740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa4d226740 20 client.15678 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa4d226740 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda890d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa4d226740 4 client.15678 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcda7e7c00 con 0x55fcda88e920 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa43fff640 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda890d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa43fff640 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda890d10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55dd47060d4ff868 server_cookie=2ad8ad41c34b48ae in_seq=0 out_seq=0 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.843+0000 7faa237fe640 10 client.15678 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa237fe640 1 -- 172.21.15.66:0/2711849236 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcda7e7c00 con 0x55fcda88e920 2024-04-21T09:08:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa237fe640 10 client.15678 handle_command_reply: tid=0 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 client.15678 shutdown 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 msgr2=0x55fcda87c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda87c550 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0be9f0 tx=0x7faa3c0be650 comp rx=0 tx=0).stop 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 shutdown_connections 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda890d10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda87c550 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda87c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa237fe640 0 client.15678 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 --2- 172.21.15.66:0/2711849236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875580 0x55fcda87bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 >> 172.21.15.66:0/2711849236 conn(0x55fcda86fcb0 msgr2=0x55fcda871a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 shutdown_connections 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.844+0000 7faa4d226740 1 -- 172.21.15.66:0/2711849236 wait complete. 2024-04-21T09:08:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.845+0000 7faa4d226740 20 client.15678 trim_cache size 0 max 16384 2024-04-21T09:08:17.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 Processor -- start 2024-04-21T09:08:17.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 -- start start 2024-04-21T09:08:17.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875580 0x55fcda88a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda86ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda87ea80 0x55fcda87c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa441b3ce0 con 0x55fcda875580 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa441b15b0 con 0x55fcda875ea0 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa4d226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faa441af3e0 con 0x55fcda87ea80 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875580 0x55fcda88a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875580 0x55fcda88a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32968/0 (socket says 172.21.15.66:32968) 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 -- 172.21.15.66:0/908165349 learned_addr learned my addr 172.21.15.66:0/908165349 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa437fe640 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda86ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa48bce640 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda87ea80 0x55fcda87c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 -- 172.21.15.66:0/908165349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda87ea80 msgr2=0x55fcda87c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda87ea80 0x55fcda87c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 -- 172.21.15.66:0/908165349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 msgr2=0x55fcda86ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda86ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa43fff640 1 -- 172.21.15.66:0/908165349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcda875cd0 con 0x55fcda875580 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa48bce640 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda87ea80 0x55fcda87c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.885+0000 7faa437fe640 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda86ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa43fff640 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875580 0x55fcda88a730 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faa380117d0 tx=0x7faa38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0594ee50d33b575 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa29ffb640 1 -- 172.21.15.66:0/908165349 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38011e10 con 0x55fcda875580 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa29ffb640 1 -- 172.21.15.66:0/908165349 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa38021e80 con 0x55fcda875580 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa29ffb640 1 -- 172.21.15.66:0/908165349 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38005570 con 0x55fcda875580 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 -- 172.21.15.66:0/908165349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875580 msgr2=0x55fcda88a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875580 0x55fcda88a730 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faa380117d0 tx=0x7faa38013000 comp rx=0 tx=0).stop 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 -- 172.21.15.66:0/908165349 shutdown_connections 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda87ea80 0x55fcda87c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875ea0 0x55fcda86ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 --2- 172.21.15.66:0/908165349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda875580 0x55fcda88a730 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 -- 172.21.15.66:0/908165349 >> 172.21.15.66:0/908165349 conn(0x55fcda86fcb0 msgr2=0x55fcda87ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 -- 172.21.15.66:0/908165349 shutdown_connections 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.886+0000 7faa4d226740 1 -- 172.21.15.66:0/908165349 wait complete. 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 -- start start 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 Processor -- start 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875580 0x55fcda878e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875ea0 0x55fcda879380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda8f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa441afa70 con 0x55fcda87ea80 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa441b0ee0 con 0x55fcda875580 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faa441b0140 con 0x55fcda875ea0 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa4d226740 10 client.0 _ll_register_callbacks cb 0x7faa3c0f03e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa48bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda8f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa48bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda8f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32982/0 (socket says 172.21.15.66:32982) 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa48bce640 1 -- 172.21.15.66:0/3492450837 learned_addr learned my addr 172.21.15.66:0/3492450837 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.887+0000 7faa43fff640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875580 0x55fcda878e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa437fe640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875ea0 0x55fcda879380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa48bce640 1 -- 172.21.15.66:0/3492450837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875ea0 msgr2=0x55fcda879380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa48bce640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875ea0 0x55fcda879380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa48bce640 1 -- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875580 msgr2=0x55fcda878e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa48bce640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875580 0x55fcda878e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa48bce640 1 -- 172.21.15.66:0/3492450837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcda8f42d0 con 0x55fcda87ea80 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa437fe640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875ea0 0x55fcda879380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa48bce640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda8f3d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faa3c03d040 tx=0x7faa3c004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bded3c634c7d2f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa237fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa237fe640 1 -- 172.21.15.66:0/3492450837 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c035e20 con 0x55fcda87ea80 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa237fe640 1 -- 172.21.15.66:0/3492450837 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7faa3c0d5c60 con 0x55fcda87ea80 2024-04-21T09:08:17.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcda86f3f0 con 0x55fcda87ea80 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa237fe640 1 -- 172.21.15.66:0/3492450837 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa3c0beac0 con 0x55fcda87ea80 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa4d226740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fcda8f42d0 con 0x55fcda87ea80 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.888+0000 7faa237fe640 1 -- 172.21.15.66:0/3492450837 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fcda86f3f0 con 0x55fcda87ea80 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa237fe640 1 -- 172.21.15.66:0/3492450837 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faa3c0be140 con 0x55fcda87ea80 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa4d226740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa4d226740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa4d226740 20 client.15684 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa4d226740 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda8f7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa4d226740 4 client.15684 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fcda7e1a70 con 0x55fcda88e920 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa43fff640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda8f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa237fe640 10 client.15684 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa43fff640 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda8f7840 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8397e519c4269865 server_cookie=d0fd08ed64b98403 in_seq=0 out_seq=0 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa237fe640 1 -- 172.21.15.66:0/3492450837 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fcda7e1a70 con 0x55fcda88e920 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa237fe640 10 client.15684 handle_command_reply: tid=0 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.889+0000 7faa4d226740 1 client.15684 shutdown 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 msgr2=0x55fcda8f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda8f3d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faa3c03d040 tx=0x7faa3c004820 comp rx=0 tx=0).stop 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 shutdown_connections 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fcda88e920 0x55fcda8f7840 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcda87ea80 0x55fcda8f3d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fcda875ea0 0x55fcda879380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 --2- 172.21.15.66:0/3492450837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcda875580 0x55fcda878e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 >> 172.21.15.66:0/3492450837 conn(0x55fcda86fcb0 msgr2=0x55fcda86ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa237fe640 0 client.15684 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 shutdown_connections 2024-04-21T09:08:17.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 1 -- 172.21.15.66:0/3492450837 wait complete. 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4d226740 20 client.15684 trim_cache size 0 max 16384 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "dbc43119-7654-4f8f-9e8b-3c9ab403fe7f": { 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "dbc43119-7654-4f8f-9e8b-3c9ab403fe7f", 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:08:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:08:17.899 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:08:17.899 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:17.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa2c0a3db0 msgr2=0x7faa2c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa2c0a3db0 0x7faa2c0a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa3800bbe0 tx=0x7faa3800b390 comp rx=0 tx=0).stop 2024-04-21T09:08:17.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 msgr2=0x7faa44101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa44101890 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faa3c02c840 tx=0x7faa3c003c20 comp rx=0 tx=0).stop 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 shutdown_connections 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa2c0a3db0 0x7faa2c0a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa4410c500 0x7faa44101890 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa44103ae0 0x7faa44101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 --2- 172.21.15.66:0/850076281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa441031c0 0x7faa44100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.890+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 >> 172.21.15.66:0/850076281 conn(0x7faa44078000 msgr2=0x7faa440fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.891+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 shutdown_connections 2024-04-21T09:08:17.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.891+0000 7faa4a631640 1 -- 172.21.15.66:0/850076281 wait complete. 2024-04-21T09:08:17.902 INFO:tasks.cephfs.filesystem:scrub status for tag:dbc43119-7654-4f8f-9e8b-3c9ab403fe7f - {'path': '/', 'tag': 'dbc43119-7654-4f8f-9e8b-3c9ab403fe7f', 'options': 'recursive,force'} 2024-04-21T09:08:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.933+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f98d0062390 con 0x7f98bc0a3ee0 2024-04-21T09:08:17.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:17.999+0000 7f98c67fc640 1 -- 172.21.15.66:0/2183987044 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f98d0062390 con 0x7f98bc0a3ee0 2024-04-21T09:08:18.000 INFO:teuthology.orchestra.run.smithi066.stdout:Schedule set for path / 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3ee0 msgr2=0x7f98bc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3ee0 0x7f98bc0a63a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d019bf70 tx=0x7f98c0005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 msgr2=0x7f98d01a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d01a0660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005940 tx=0x7f98cc02c5f0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 shutdown_connections 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3ee0 0x7f98bc0a63a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d010e910 0x7f98d019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d0101680 0x7f98d01a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 --2- 172.21.15.66:0/2183987044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d0100d60 0x7f98d01a0660 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 >> 172.21.15.66:0/2183987044 conn(0x7f98d0077fd0 msgr2=0x7f98d010ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:18.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 shutdown_connections 2024-04-21T09:08:18.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.001+0000 7f98d7281640 1 -- 172.21.15.66:0/2183987044 wait complete. 2024-04-21T09:08:18.012 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-21T09:08:18.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/3900696925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841010f0 msgr2=0x7f9a841014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/3900696925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841010f0 0x7f9a841014d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009900 tx=0x7f9a7402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:18.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/3900696925 shutdown_connections 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/3900696925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a84106170 0x7f9a8410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/3900696925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a84101aa0 0x7f9a84105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/3900696925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841010f0 0x7f9a841014d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/3900696925 >> 172.21.15.66:0/3900696925 conn(0x7f9a84077fd0 msgr2=0x7f9a840783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/3900696925 shutdown_connections 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.156+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/3900696925 wait complete. 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 Processor -- start 2024-04-21T09:08:18.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 -- start start 2024-04-21T09:08:18.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a841010f0 0x7f9a841994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a84106170 0x7f9a84199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a8419a000 0x7f9a841a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841110e0 con 0x7f9a8419a000 2024-04-21T09:08:18.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84110f60 con 0x7f9a84106170 2024-04-21T09:08:18.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.157+0000 7f9a8a8e4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84111260 con 0x7f9a841010f0 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a8419a000 0x7f9a841a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a8419a000 0x7f9a841a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32998/0 (socket says 172.21.15.66:32998) 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 -- 172.21.15.66:0/2396788728 learned_addr learned my addr 172.21.15.66:0/2396788728 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a83fff640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a841010f0 0x7f9a841994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a837fe640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a84106170 0x7f9a84199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 -- 172.21.15.66:0/2396788728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a841010f0 msgr2=0x7f9a841994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a841010f0 0x7f9a841994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 -- 172.21.15.66:0/2396788728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a84106170 msgr2=0x7f9a84199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a84106170 0x7f9a84199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 -- 172.21.15.66:0/2396788728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a841a4d80 con 0x7f9a8419a000 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a837fe640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a84106170 0x7f9a84199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a83fff640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a841010f0 0x7f9a841994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a88e81640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a8419a000 0x7f9a841a4560 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00eaa0 tx=0x7f9a7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:18.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a817fa640 1 -- 172.21.15.66:0/2396788728 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00cdb0 con 0x7f9a8419a000 2024-04-21T09:08:18.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a817fa640 1 -- 172.21.15.66:0/2396788728 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a7c005020 con 0x7f9a8419a000 2024-04-21T09:08:18.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a841a5070 con 0x7f9a8419a000 2024-04-21T09:08:18.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a817fa640 1 -- 172.21.15.66:0/2396788728 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0116d0 con 0x7f9a8419a000 2024-04-21T09:08:18.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.158+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a841a55c0 con 0x7f9a8419a000 2024-04-21T09:08:18.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.159+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8404fce0 con 0x7f9a8419a000 2024-04-21T09:08:18.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.160+0000 7f9a817fa640 1 -- 172.21.15.66:0/2396788728 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f9a7c0051d0 con 0x7f9a8419a000 2024-04-21T09:08:18.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.161+0000 7f9a817fa640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9a6c0a3e60 0x7f9a6c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.161+0000 7f9a83fff640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9a6c0a3e60 0x7f9a6c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.161+0000 7f9a817fa640 1 -- 172.21.15.66:0/2396788728 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f9a7c0d0f00 con 0x7f9a8419a000 2024-04-21T09:08:18.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.161+0000 7f9a83fff640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9a6c0a3e60 0x7f9a6c0a6320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a74005ab0 tx=0x7f9a74005a20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:18.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.166+0000 7f9a817fa640 1 -- 172.21.15.66:0/2396788728 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a7c016030 con 0x7f9a8419a000 2024-04-21T09:08:18.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.376+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f9a84101aa0 con 0x7f9a6c0a3e60 2024-04-21T09:08:18.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.435+0000 7f9a817fa640 1 -- 172.21.15.66:0/2396788728 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f9a84101aa0 con 0x7f9a6c0a3e60 2024-04-21T09:08:18.435 INFO:teuthology.orchestra.run.smithi066.stdout:Retention added to path / 2024-04-21T09:08:18.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9a6c0a3e60 msgr2=0x7f9a6c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9a6c0a3e60 0x7f9a6c0a6320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a74005ab0 tx=0x7f9a74005a20 comp rx=0 tx=0).stop 2024-04-21T09:08:18.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a8419a000 msgr2=0x7f9a841a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a8419a000 0x7f9a841a4560 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00eaa0 tx=0x7f9a7c00ef60 comp rx=0 tx=0).stop 2024-04-21T09:08:18.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 shutdown_connections 2024-04-21T09:08:18.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9a6c0a3e60 0x7f9a6c0a6320 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a8419a000 0x7f9a841a4560 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a84106170 0x7f9a84199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 --2- 172.21.15.66:0/2396788728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a841010f0 0x7f9a841994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 >> 172.21.15.66:0/2396788728 conn(0x7f9a84077fd0 msgr2=0x7f9a840ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:18.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 shutdown_connections 2024-04-21T09:08:18.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.437+0000 7f9a8a8e4640 1 -- 172.21.15.66:0/2396788728 wait complete. 2024-04-21T09:08:18.448 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T09:08:18.533 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:18 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2608037512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:18.533 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:18 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:08:18.533 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:18 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:18.533 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:18 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:08:18.533 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:18 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 -- 172.21.15.66:0/3734591798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1341011a0 msgr2=0x7ff134101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 --2- 172.21.15.66:0/3734591798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1341011a0 0x7ff134101620 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff124009900 tx=0x7ff12402bc80 comp rx=0 tx=0).stop 2024-04-21T09:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 -- 172.21.15.66:0/3734591798 shutdown_connections 2024-04-21T09:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 --2- 172.21.15.66:0/3734591798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff134101b60 0x7ff13410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 --2- 172.21.15.66:0/3734591798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1341011a0 0x7ff134101620 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 --2- 172.21.15.66:0/3734591798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff134106fb0 0x7ff134107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 -- 172.21.15.66:0/3734591798 >> 172.21.15.66:0/3734591798 conn(0x7ff134076c50 msgr2=0x7ff134077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 -- 172.21.15.66:0/3734591798 shutdown_connections 2024-04-21T09:08:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.550+0000 7ff13a146640 1 -- 172.21.15.66:0/3734591798 wait complete. 2024-04-21T09:08:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 Processor -- start 2024-04-21T09:08:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 -- start start 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341011a0 0x7ff1341985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff134101b60 0x7ff134198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff134106fb0 0x7ff13419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341110b0 con 0x7ff134106fb0 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff134110f30 con 0x7ff1341011a0 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff13a146640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff134111230 con 0x7ff134101b60 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.551+0000 7ff133fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff134106fb0 0x7ff13419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff134106fb0 0x7ff13419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33010/0 (socket says 172.21.15.66:33010) 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 -- 172.21.15.66:0/3281145606 learned_addr learned my addr 172.21.15.66:0/3281145606 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff132ffd640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff134101b60 0x7ff134198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff1337fe640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341011a0 0x7ff1341985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 -- 172.21.15.66:0/3281145606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff134101b60 msgr2=0x7ff134198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff134101b60 0x7ff134198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 -- 172.21.15.66:0/3281145606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341011a0 msgr2=0x7ff1341985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341011a0 0x7ff1341985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 -- 172.21.15.66:0/3281145606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13419d5c0 con 0x7ff134106fb0 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff1337fe640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341011a0 0x7ff1341985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff133fff640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff134106fb0 0x7ff13419ce90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00eaa0 tx=0x7ff12c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff130ff9640 1 -- 172.21.15.66:0/3281145606 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00cdb0 con 0x7ff134106fb0 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff130ff9640 1 -- 172.21.15.66:0/3281145606 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c005020 con 0x7ff134106fb0 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13419d8b0 con 0x7ff134106fb0 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff130ff9640 1 -- 172.21.15.66:0/3281145606 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0116d0 con 0x7ff134106fb0 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.552+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341a5140 con 0x7ff134106fb0 2024-04-21T09:08:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.553+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff134102550 con 0x7ff134106fb0 2024-04-21T09:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.554+0000 7ff130ff9640 1 -- 172.21.15.66:0/3281145606 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7ff12c0051d0 con 0x7ff134106fb0 2024-04-21T09:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.555+0000 7ff130ff9640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff11c0a3ee0 0x7ff11c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.555+0000 7ff1337fe640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff11c0a3ee0 0x7ff11c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.555+0000 7ff130ff9640 1 -- 172.21.15.66:0/3281145606 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff12c0d1300 con 0x7ff134106fb0 2024-04-21T09:08:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.556+0000 7ff1337fe640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff11c0a3ee0 0x7ff11c0a63a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff128005d00 tx=0x7ff128009550 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:18.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.559+0000 7ff130ff9640 1 -- 172.21.15.66:0/3281145606 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff12c016030 con 0x7ff134106fb0 2024-04-21T09:08:18.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:18 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2608037512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:18.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:18 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:08:18.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:18 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:18.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:18 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:08:18.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:18 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:18.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:18 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2608037512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:18 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:08:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:18 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:18 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:08:18.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:18 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:18.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.771+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff134104c50 con 0x7ff11c0a3ee0 2024-04-21T09:08:18.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.778+0000 7ff130ff9640 1 -- 172.21.15.66:0/3281145606 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7ff134104c50 con 0x7ff11c0a3ee0 2024-04-21T09:08:18.779 INFO:teuthology.orchestra.run.smithi066.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T09:08:17", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff11c0a3ee0 msgr2=0x7ff11c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff11c0a3ee0 0x7ff11c0a63a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff128005d00 tx=0x7ff128009550 comp rx=0 tx=0).stop 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff134106fb0 msgr2=0x7ff13419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff134106fb0 0x7ff13419ce90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00eaa0 tx=0x7ff12c00ef60 comp rx=0 tx=0).stop 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 shutdown_connections 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff11c0a3ee0 0x7ff11c0a63a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff134106fb0 0x7ff13419ce90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff134101b60 0x7ff134198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 --2- 172.21.15.66:0/3281145606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341011a0 0x7ff1341985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 >> 172.21.15.66:0/3281145606 conn(0x7ff134076c50 msgr2=0x7ff134100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:18.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 shutdown_connections 2024-04-21T09:08:18.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.780+0000 7ff13a146640 1 -- 172.21.15.66:0/3281145606 wait complete. 2024-04-21T09:08:18.791 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T09:08:18.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 -- 172.21.15.66:0/3754103821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad0106170 msgr2=0x7faad010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 --2- 172.21.15.66:0/3754103821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad0106170 0x7faad010dd40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faacc009ca0 tx=0x7faacc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 -- 172.21.15.66:0/3754103821 shutdown_connections 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 --2- 172.21.15.66:0/3754103821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad0106170 0x7faad010dd40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 --2- 172.21.15.66:0/3754103821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad0101aa0 0x7faad0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 --2- 172.21.15.66:0/3754103821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad01010f0 0x7faad01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 -- 172.21.15.66:0/3754103821 >> 172.21.15.66:0/3754103821 conn(0x7faad0077fd0 msgr2=0x7faad00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 -- 172.21.15.66:0/3754103821 shutdown_connections 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 -- 172.21.15.66:0/3754103821 wait complete. 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 Processor -- start 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.894+0000 7faad81fd640 1 -- start start 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad01010f0 0x7faad01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad0106170 0x7faad0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad019cfb0 0x7faad019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faad01121f0 con 0x7faad019cfb0 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faad0112070 con 0x7faad01010f0 2024-04-21T09:08:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faad0112370 con 0x7faad0106170 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad019cfb0 0x7faad019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad019cfb0 0x7faad019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33016/0 (socket says 172.21.15.66:33016) 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 -- 172.21.15.66:0/376814163 learned_addr learned my addr 172.21.15.66:0/376814163 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad5f99640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad01010f0 0x7faad01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad5798640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad0106170 0x7faad0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 -- 172.21.15.66:0/376814163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad0106170 msgr2=0x7faad0198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad0106170 0x7faad0198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 -- 172.21.15.66:0/376814163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad01010f0 msgr2=0x7faad01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad01010f0 0x7faad01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 -- 172.21.15.66:0/376814163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad019daa0 con 0x7faad019cfb0 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad679a640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad019cfb0 0x7faad019d460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faacc005e30 tx=0x7faacc034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faac6ffd640 1 -- 172.21.15.66:0/376814163 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc039040 con 0x7faad019cfb0 2024-04-21T09:08:18.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faac6ffd640 1 -- 172.21.15.66:0/376814163 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faacc044070 con 0x7faad019cfb0 2024-04-21T09:08:18.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad019dd30 con 0x7faad019cfb0 2024-04-21T09:08:18.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faac6ffd640 1 -- 172.21.15.66:0/376814163 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc003e60 con 0x7faad019cfb0 2024-04-21T09:08:18.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.895+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01a5550 con 0x7faad019cfb0 2024-04-21T09:08:18.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.896+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad004fce0 con 0x7faad019cfb0 2024-04-21T09:08:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.898+0000 7faac6ffd640 1 -- 172.21.15.66:0/376814163 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7faacc0324c0 con 0x7faad019cfb0 2024-04-21T09:08:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.898+0000 7faac6ffd640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faabc0a3ee0 0x7faabc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.898+0000 7faac6ffd640 1 -- 172.21.15.66:0/376814163 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7faacc0f37f0 con 0x7faad019cfb0 2024-04-21T09:08:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.898+0000 7faad5f99640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faabc0a3ee0 0x7faabc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:18.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.899+0000 7faad5f99640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faabc0a3ee0 0x7faabc0a63a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faac800bbe0 tx=0x7faac800b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:18.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:18.902+0000 7faac6ffd640 1 -- 172.21.15.66:0/376814163 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faacc0bfac0 con 0x7faad019cfb0 2024-04-21T09:08:19.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.112+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7faad0101aa0 con 0x7faabc0a3ee0 2024-04-21T09:08:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.119+0000 7faac6ffd640 1 -- 172.21.15.66:0/376814163 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7faad0101aa0 con 0x7faabc0a3ee0 2024-04-21T09:08:19.120 INFO:teuthology.orchestra.run.smithi066.stdout:/ 1m 6m3h 2024-04-21T09:08:19.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faabc0a3ee0 msgr2=0x7faabc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faabc0a3ee0 0x7faabc0a63a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faac800bbe0 tx=0x7faac800b390 comp rx=0 tx=0).stop 2024-04-21T09:08:19.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad019cfb0 msgr2=0x7faad019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad019cfb0 0x7faad019d460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faacc005e30 tx=0x7faacc034760 comp rx=0 tx=0).stop 2024-04-21T09:08:19.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 shutdown_connections 2024-04-21T09:08:19.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faabc0a3ee0 0x7faabc0a63a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad019cfb0 0x7faad019d460 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad0106170 0x7faad0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 --2- 172.21.15.66:0/376814163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad01010f0 0x7faad01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 >> 172.21.15.66:0/376814163 conn(0x7faad0077fd0 msgr2=0x7faad00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:19.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 shutdown_connections 2024-04-21T09:08:19.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.121+0000 7faad81fd640 1 -- 172.21.15.66:0/376814163 wait complete. 2024-04-21T09:08:19.133 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-21T09:08:19.158 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-21T09:08:19.168 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-21T09:08:19.212 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:08:19.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-21T09:08:19.215 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-21T09:08:19.263 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 -- 172.21.15.66:0/1412899334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 msgr2=0x7ff7541062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 --2- 172.21.15.66:0/1412899334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff7541062a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009900 tx=0x7ff74c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 -- 172.21.15.66:0/1412899334 shutdown_connections 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 --2- 172.21.15.66:0/1412899334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff754102030 0x7ff75410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 --2- 172.21.15.66:0/1412899334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff754101670 0x7ff754101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 --2- 172.21.15.66:0/1412899334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff7541062a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 -- 172.21.15.66:0/1412899334 >> 172.21.15.66:0/1412899334 conn(0x7ff754077480 msgr2=0x7ff7540fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 -- 172.21.15.66:0/1412899334 shutdown_connections 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.365+0000 7ff75bce4640 1 -- 172.21.15.66:0/1412899334 wait complete. 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 Processor -- start 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 -- start start 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff754101670 0x7ff754198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff754102030 0x7ff754198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff75419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541110f0 con 0x7ff754105ec0 2024-04-21T09:08:19.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff754110f70 con 0x7ff754102030 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75bce4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff754111270 con 0x7ff754101670 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff75419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff75419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33026/0 (socket says 172.21.15.66:33026) 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 -- 172.21.15.66:0/2495672434 learned_addr learned my addr 172.21.15.66:0/2495672434 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff759a80640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff754101670 0x7ff754198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75927f640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff754102030 0x7ff754198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 -- 172.21.15.66:0/2495672434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff754101670 msgr2=0x7ff754198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff754101670 0x7ff754198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 -- 172.21.15.66:0/2495672434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff754102030 msgr2=0x7ff754198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff754102030 0x7ff754198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 -- 172.21.15.66:0/2495672434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75419d720 con 0x7ff754105ec0 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff75a281640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff75419cff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff744009fd0 tx=0x7ff74400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.366+0000 7ff752ffd640 1 -- 172.21.15.66:0/2495672434 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744017070 con 0x7ff754105ec0 2024-04-21T09:08:19.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.367+0000 7ff752ffd640 1 -- 172.21.15.66:0/2495672434 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff744003e20 con 0x7ff754105ec0 2024-04-21T09:08:19.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.367+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75419da10 con 0x7ff754105ec0 2024-04-21T09:08:19.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.367+0000 7ff752ffd640 1 -- 172.21.15.66:0/2495672434 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744010460 con 0x7ff754105ec0 2024-04-21T09:08:19.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.367+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7541a52a0 con 0x7ff754105ec0 2024-04-21T09:08:19.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.367+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff754069230 con 0x7ff754105ec0 2024-04-21T09:08:19.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.369+0000 7ff752ffd640 1 -- 172.21.15.66:0/2495672434 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7ff744003970 con 0x7ff754105ec0 2024-04-21T09:08:19.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.369+0000 7ff752ffd640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff73c0a3e30 0x7ff73c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.369+0000 7ff759a80640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff73c0a3e30 0x7ff73c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.369+0000 7ff752ffd640 1 -- 172.21.15.66:0/2495672434 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff7440d0a50 con 0x7ff754105ec0 2024-04-21T09:08:19.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.370+0000 7ff759a80640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff73c0a3e30 0x7ff73c0a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff74c006010 tx=0x7ff74c02c6c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:19.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.374+0000 7ff752ffd640 1 -- 172.21.15.66:0/2495672434 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff74409cc10 con 0x7ff754105ec0 2024-04-21T09:08:19.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.563+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7541a5540 con 0x7ff754105ec0 2024-04-21T09:08:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.564+0000 7ff752ffd640 1 -- 172.21.15.66:0/2495672434 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+25086 (secure 0 0 0) 0x7ff74409c510 con 0x7ff754105ec0 2024-04-21T09:08:19.565 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:19.565 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:08:12.028415+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:19.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff73c0a3e30 msgr2=0x7ff73c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff73c0a3e30 0x7ff73c0a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff74c006010 tx=0x7ff74c02c6c0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 msgr2=0x7ff75419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff75419cff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff744009fd0 tx=0x7ff74400b260 comp rx=0 tx=0).stop 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 shutdown_connections 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff73c0a3e30 0x7ff73c0a62f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff754105ec0 0x7ff75419cff0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff754102030 0x7ff754198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 --2- 172.21.15.66:0/2495672434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff754101670 0x7ff754198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 >> 172.21.15.66:0/2495672434 conn(0x7ff754077480 msgr2=0x7ff754100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 shutdown_connections 2024-04-21T09:08:19.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.567+0000 7ff75bce4640 1 -- 172.21.15.66:0/2495672434 wait complete. 2024-04-21T09:08:19.577 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:19 smithi066 ceph-mon[27953]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:19.577 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:19 smithi066 ceph-mon[27953]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:19.577 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:19 smithi066 ceph-mon[27953]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:19.578 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:08:19.579 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:08:19.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:19 smithi186 ceph-mon[31505]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:19 smithi186 ceph-mon[31505]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:19 smithi186 ceph-mon[31505]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:19 smithi083 ceph-mon[31413]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:19 smithi083 ceph-mon[31413]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:08:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:19 smithi083 ceph-mon[31413]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/1165760840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc06b1f0 msgr2=0x7fb7cc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/1165760840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc06b1f0 0x7fb7cc106b90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009900 tx=0x7fb7b802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/1165760840 shutdown_connections 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/1165760840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc107270 0x7fb7cc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/1165760840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc06b1f0 0x7fb7cc106b90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/1165760840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7cc06a840 0x7fb7cc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/1165760840 >> 172.21.15.66:0/1165760840 conn(0x7fb7cc077eb0 msgr2=0x7fb7cc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/1165760840 shutdown_connections 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.683+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/1165760840 wait complete. 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 Processor -- start 2024-04-21T09:08:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 -- start start 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7cc06a840 0x7fb7cc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc107270 0x7fb7cc198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc19cf90 0x7fb7cc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc112190 con 0x7fb7cc107270 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc112010 con 0x7fb7cc06a840 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d2a8c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc112310 con 0x7fb7cc19cf90 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc107270 0x7fb7cc198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc107270 0x7fb7cc198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33050/0 (socket says 172.21.15.66:33050) 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 -- 172.21.15.66:0/617795386 learned_addr learned my addr 172.21.15.66:0/617795386 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d1029640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc19cf90 0x7fb7cc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7d0828640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7cc06a840 0x7fb7cc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 -- 172.21.15.66:0/617795386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc19cf90 msgr2=0x7fb7cc19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc19cf90 0x7fb7cc19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 -- 172.21.15.66:0/617795386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7cc06a840 msgr2=0x7fb7cc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7cc06a840 0x7fb7cc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.684+0000 7fb7cbfff640 1 -- 172.21.15.66:0/617795386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc19da80 con 0x7fb7cc107270 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.685+0000 7fb7cbfff640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc107270 0x7fb7cc198c10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8036040 tx=0x7fb7b80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.685+0000 7fb7c9ffb640 1 -- 172.21.15.66:0/617795386 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8043020 con 0x7fb7cc107270 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.685+0000 7fb7c9ffb640 1 -- 172.21.15.66:0/617795386 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b802cb70 con 0x7fb7cc107270 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.685+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc19dd10 con 0x7fb7cc107270 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.685+0000 7fb7c9ffb640 1 -- 172.21.15.66:0/617795386 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8003bf0 con 0x7fb7cc107270 2024-04-21T09:08:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.685+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1a5560 con 0x7fb7cc107270 2024-04-21T09:08:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.686+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc0691c0 con 0x7fb7cc107270 2024-04-21T09:08:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.687+0000 7fb7c9ffb640 1 -- 172.21.15.66:0/617795386 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fb7b8046020 con 0x7fb7cc107270 2024-04-21T09:08:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.688+0000 7fb7c9ffb640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb7bc0a3e30 0x7fb7bc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.688+0000 7fb7d0828640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb7bc0a3e30 0x7fb7bc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.688+0000 7fb7c9ffb640 1 -- 172.21.15.66:0/617795386 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb7b80f2d30 con 0x7fb7cc107270 2024-04-21T09:08:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.688+0000 7fb7d0828640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb7bc0a3e30 0x7fb7bc0a62f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005fd0 tx=0x7fb7c000b590 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:19.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.692+0000 7fb7c9ffb640 1 -- 172.21.15.66:0/617795386 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7b804d020 con 0x7fb7cc107270 2024-04-21T09:08:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.879+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7cc06b1f0 con 0x7fb7cc107270 2024-04-21T09:08:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.880+0000 7fb7c9ffb640 1 -- 172.21.15.66:0/617795386 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fb7b80bf000 con 0x7fb7cc107270 2024-04-21T09:08:19.880 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:08:19.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb7bc0a3e30 msgr2=0x7fb7bc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb7bc0a3e30 0x7fb7bc0a62f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005fd0 tx=0x7fb7c000b590 comp rx=0 tx=0).stop 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc107270 msgr2=0x7fb7cc198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc107270 0x7fb7cc198c10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8036040 tx=0x7fb7b80045e0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 shutdown_connections 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb7bc0a3e30 0x7fb7bc0a62f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc19cf90 0x7fb7cc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc107270 0x7fb7cc198c10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 --2- 172.21.15.66:0/617795386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7cc06a840 0x7fb7cc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 >> 172.21.15.66:0/617795386 conn(0x7fb7cc077eb0 msgr2=0x7fb7cc108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 shutdown_connections 2024-04-21T09:08:19.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.882+0000 7fb7d2a8c640 1 -- 172.21.15.66:0/617795386 wait complete. 2024-04-21T09:08:19.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:08:20.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 -- 172.21.15.66:0/1510114102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d8102190 msgr2=0x7f25d8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/1510114102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d8102190 0x7f25d8102570 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009ca0 tx=0x7f25c802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 -- 172.21.15.66:0/1510114102 shutdown_connections 2024-04-21T09:08:20.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/1510114102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d8107260 0x7f25d810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/1510114102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d8102b40 0x7f25d8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/1510114102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d8102190 0x7f25d8102570 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 -- 172.21.15.66:0/1510114102 >> 172.21.15.66:0/1510114102 conn(0x7f25d80fd990 msgr2=0x7f25d80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 -- 172.21.15.66:0/1510114102 shutdown_connections 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.997+0000 7f25ddd8f640 1 -- 172.21.15.66:0/1510114102 wait complete. 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.998+0000 7f25ddd8f640 1 Processor -- start 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.998+0000 7f25ddd8f640 1 -- start start 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.998+0000 7f25ddd8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d8102190 0x7f25d819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.998+0000 7f25ddd8f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d8107260 0x7f25d819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.998+0000 7f25ddd8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d81985c0 0x7f25d8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25ddd8f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81121c0 con 0x7f25d81985c0 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25ddd8f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8112040 con 0x7f25d8102190 2024-04-21T09:08:20.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25ddd8f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8112340 con 0x7f25d8107260 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d81985c0 0x7f25d8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d81985c0 0x7f25d8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33064/0 (socket says 172.21.15.66:33064) 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 -- 172.21.15.66:0/502863915 learned_addr learned my addr 172.21.15.66:0/502863915 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d6ffd640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d8107260 0x7f25d819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d77fe640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d8102190 0x7f25d819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 -- 172.21.15.66:0/502863915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d8107260 msgr2=0x7f25d819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d8107260 0x7f25d819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 -- 172.21.15.66:0/502863915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d8102190 msgr2=0x7f25d819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d8102190 0x7f25d819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 -- 172.21.15.66:0/502863915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d81990e0 con 0x7f25d81985c0 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d77fe640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d8102190 0x7f25d819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d7fff640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d81985c0 0x7f25d8198a70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d950 tx=0x7f25d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d4ff9640 1 -- 172.21.15.66:0/502863915 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0017030 con 0x7f25d81985c0 2024-04-21T09:08:20.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d4ff9640 1 -- 172.21.15.66:0/502863915 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0004020 con 0x7f25d81985c0 2024-04-21T09:08:20.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d81993d0 con 0x7f25d81985c0 2024-04-21T09:08:20.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25d4ff9640 1 -- 172.21.15.66:0/502863915 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0003b50 con 0x7f25d81985c0 2024-04-21T09:08:20.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:19.999+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d81a5620 con 0x7f25d81985c0 2024-04-21T09:08:20.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.000+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d80691c0 con 0x7f25d81985c0 2024-04-21T09:08:20.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.002+0000 7f25d4ff9640 1 -- 172.21.15.66:0/502863915 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f25d001a070 con 0x7f25d81985c0 2024-04-21T09:08:20.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.002+0000 7f25d4ff9640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f25c00a3e30 0x7f25c00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.002+0000 7f25d77fe640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f25c00a3e30 0x7f25c00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.002+0000 7f25d4ff9640 1 -- 172.21.15.66:0/502863915 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f25d00d0740 con 0x7f25d81985c0 2024-04-21T09:08:20.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.003+0000 7f25d77fe640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f25c00a3e30 0x7f25c00a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009a60 tx=0x7f25c8005d80 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.006+0000 7f25d4ff9640 1 -- 172.21.15.66:0/502863915 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25d0021070 con 0x7f25d81985c0 2024-04-21T09:08:20.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.190+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f25d8102b40 con 0x7f25d81985c0 2024-04-21T09:08:20.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.191+0000 7f25d4ff9640 1 -- 172.21.15.66:0/502863915 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f25d009c980 con 0x7f25d81985c0 2024-04-21T09:08:20.192 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:20.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f25c00a3e30 msgr2=0x7f25c00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f25c00a3e30 0x7f25c00a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009a60 tx=0x7f25c8005d80 comp rx=0 tx=0).stop 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d81985c0 msgr2=0x7f25d8198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d81985c0 0x7f25d8198a70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d950 tx=0x7f25d000de10 comp rx=0 tx=0).stop 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 shutdown_connections 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f25c00a3e30 0x7f25c00a62f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d81985c0 0x7f25d8198a70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d8107260 0x7f25d819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 --2- 172.21.15.66:0/502863915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d8102190 0x7f25d819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 >> 172.21.15.66:0/502863915 conn(0x7f25d80fd990 msgr2=0x7f25d80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 shutdown_connections 2024-04-21T09:08:20.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.193+0000 7f25ddd8f640 1 -- 172.21.15.66:0/502863915 wait complete. 2024-04-21T09:08:20.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 -- 172.21.15.66:0/901571294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe0102190 msgr2=0x7fefe0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 --2- 172.21.15.66:0/901571294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe0102190 0x7fefe0102570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009ca0 tx=0x7fefd002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 -- 172.21.15.66:0/901571294 shutdown_connections 2024-04-21T09:08:20.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 --2- 172.21.15.66:0/901571294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0107260 0x7fefe010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 --2- 172.21.15.66:0/901571294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe0102b40 0x7fefe0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 --2- 172.21.15.66:0/901571294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe0102190 0x7fefe0102570 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 -- 172.21.15.66:0/901571294 >> 172.21.15.66:0/901571294 conn(0x7fefe00793a0 msgr2=0x7fefe00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 -- 172.21.15.66:0/901571294 shutdown_connections 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 -- 172.21.15.66:0/901571294 wait complete. 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 Processor -- start 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 -- start start 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.311+0000 7fefe604e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0102190 0x7fefe01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefe604e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe0107260 0x7fefe01a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefe604e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe019a830 0x7fefe019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefe604e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01121d0 con 0x7fefe019a830 2024-04-21T09:08:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefe604e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0112050 con 0x7fefe0107260 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefe604e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0112350 con 0x7fefe0102190 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe019a830 0x7fefe019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe019a830 0x7fefe019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33076/0 (socket says 172.21.15.66:33076) 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 -- 172.21.15.66:0/1135180927 learned_addr learned my addr 172.21.15.66:0/1135180927 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdf7fe640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0102190 0x7fefe01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdeffd640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe0107260 0x7fefe01a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 -- 172.21.15.66:0/1135180927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0102190 msgr2=0x7fefe01a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0102190 0x7fefe01a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 -- 172.21.15.66:0/1135180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe0107260 msgr2=0x7fefe01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe0107260 0x7fefe01a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 -- 172.21.15.66:0/1135180927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe019b3c0 con 0x7fefe019a830 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdeffd640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe0107260 0x7fefe01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdf7fe640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0102190 0x7fefe01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdffff640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe019a830 0x7fefe019ace0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fefd800e970 tx=0x7fefd800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdcff9640 1 -- 172.21.15.66:0/1135180927 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd800ccc0 con 0x7fefe019a830 2024-04-21T09:08:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe019b6b0 con 0x7fefe019a830 2024-04-21T09:08:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdcff9640 1 -- 172.21.15.66:0/1135180927 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd800ce60 con 0x7fefe019a830 2024-04-21T09:08:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefdcff9640 1 -- 172.21.15.66:0/1135180927 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8010430 con 0x7fefe019a830 2024-04-21T09:08:20.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.312+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe01a78e0 con 0x7fefe019a830 2024-04-21T09:08:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.313+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe0103660 con 0x7fefe019a830 2024-04-21T09:08:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.314+0000 7fefdcff9640 1 -- 172.21.15.66:0/1135180927 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fefd8004f90 con 0x7fefe019a830 2024-04-21T09:08:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.315+0000 7fefdcff9640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fefcc0a3db0 0x7fefcc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.315+0000 7fefdf7fe640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fefcc0a3db0 0x7fefcc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.315+0000 7fefdcff9640 1 -- 172.21.15.66:0/1135180927 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fefd80d10f0 con 0x7fefe019a830 2024-04-21T09:08:20.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.316+0000 7fefdf7fe640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fefcc0a3db0 0x7fefcc0a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004af0 tx=0x7fefd0005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.320+0000 7fefdcff9640 1 -- 172.21.15.66:0/1135180927 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefe019b6b0 con 0x7fefe019a830 2024-04-21T09:08:20.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.505+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fefe0102ba0 con 0x7fefe019a830 2024-04-21T09:08:20.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.506+0000 7fefdcff9640 1 -- 172.21.15.66:0/1135180927 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7fefd809d2b0 con 0x7fefe019a830 2024-04-21T09:08:20.506 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:20.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fefcc0a3db0 msgr2=0x7fefcc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fefcc0a3db0 0x7fefcc0a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004af0 tx=0x7fefd0005c50 comp rx=0 tx=0).stop 2024-04-21T09:08:20.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe019a830 msgr2=0x7fefe019ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe019a830 0x7fefe019ace0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fefd800e970 tx=0x7fefd800ed70 comp rx=0 tx=0).stop 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 shutdown_connections 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fefcc0a3db0 0x7fefcc0a6270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefe019a830 0x7fefe019ace0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe0107260 0x7fefe01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 --2- 172.21.15.66:0/1135180927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0102190 0x7fefe01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 >> 172.21.15.66:0/1135180927 conn(0x7fefe00793a0 msgr2=0x7fefe01000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 shutdown_connections 2024-04-21T09:08:20.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.508+0000 7fefe604e640 1 -- 172.21.15.66:0/1135180927 wait complete. 2024-04-21T09:08:20.518 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:20 smithi066 ceph-mon[27953]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:20.518 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:20 smithi066 ceph-mon[27953]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:20.518 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:20 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2495672434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:20.518 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:20 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/617795386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:20.518 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:20 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/502863915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:20.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:20.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:20 smithi083 ceph-mon[31413]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:20.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:20 smithi083 ceph-mon[31413]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:20.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:20 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2495672434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:20.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:20 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/617795386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:20.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:20 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/502863915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:20.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:20 smithi186 ceph-mon[31505]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:20.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:20 smithi186 ceph-mon[31505]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:08:20.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:20 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2495672434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:20.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:20 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/617795386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:20.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:20 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/502863915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:20.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 -- 172.21.15.66:0/2606839035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd410c4f0 msgr2=0x7fbbd410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 --2- 172.21.15.66:0/2606839035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd410c4f0 0x7fbbd410e9b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009ca0 tx=0x7fbbcc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 -- 172.21.15.66:0/2606839035 shutdown_connections 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 --2- 172.21.15.66:0/2606839035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd410c4f0 0x7fbbd410e9b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 --2- 172.21.15.66:0/2606839035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd41016d0 0x7fbbd410be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 --2- 172.21.15.66:0/2606839035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd4100d20 0x7fbbd4101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 -- 172.21.15.66:0/2606839035 >> 172.21.15.66:0/2606839035 conn(0x7fbbd4076150 msgr2=0x7fbbd4076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 -- 172.21.15.66:0/2606839035 shutdown_connections 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.623+0000 7fbbdba69640 1 -- 172.21.15.66:0/2606839035 wait complete. 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 Processor -- start 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 -- start start 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd4100d20 0x7fbbd4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd410c4f0 0x7fbbd4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd419cf70 0x7fbbd419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41121c0 con 0x7fbbd419cf70 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4112040 con 0x7fbbd410c4f0 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbdba69640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4112340 con 0x7fbbd4100d20 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbda006640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd419cf70 0x7fbbd419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbda006640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd419cf70 0x7fbbd419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33094/0 (socket says 172.21.15.66:33094) 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbd9805640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd4100d20 0x7fbbd4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbda006640 1 -- 172.21.15.66:0/3093565853 learned_addr learned my addr 172.21.15.66:0/3093565853 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.624+0000 7fbbd9004640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd410c4f0 0x7fbbd4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbda006640 1 -- 172.21.15.66:0/3093565853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd4100d20 msgr2=0x7fbbd4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbda006640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd4100d20 0x7fbbd4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbda006640 1 -- 172.21.15.66:0/3093565853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd410c4f0 msgr2=0x7fbbd4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbda006640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd410c4f0 0x7fbbd4198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbda006640 1 -- 172.21.15.66:0/3093565853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd419da60 con 0x7fbbd419cf70 2024-04-21T09:08:20.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbda006640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd419cf70 0x7fbbd419d420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc005e30 tx=0x7fbbcc034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbcaffd640 1 -- 172.21.15.66:0/3093565853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc039040 con 0x7fbbd419cf70 2024-04-21T09:08:20.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbcaffd640 1 -- 172.21.15.66:0/3093565853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbcc044070 con 0x7fbbd419cf70 2024-04-21T09:08:20.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd419dcf0 con 0x7fbbd419cf70 2024-04-21T09:08:20.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbcaffd640 1 -- 172.21.15.66:0/3093565853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc003e60 con 0x7fbbd419cf70 2024-04-21T09:08:20.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.625+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd41a54b0 con 0x7fbbd419cf70 2024-04-21T09:08:20.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.626+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd40691c0 con 0x7fbbd419cf70 2024-04-21T09:08:20.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.627+0000 7fbbcaffd640 1 -- 172.21.15.66:0/3093565853 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fbbcc0324c0 con 0x7fbbd419cf70 2024-04-21T09:08:20.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.628+0000 7fbbcaffd640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbc00a3db0 0x7fbbc00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.628+0000 7fbbcaffd640 1 -- 172.21.15.66:0/3093565853 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbbcc0f2e30 con 0x7fbbd419cf70 2024-04-21T09:08:20.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.628+0000 7fbbd9805640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbc00a3db0 0x7fbbc00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.628+0000 7fbbd9805640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbc00a3db0 0x7fbbc00a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005d00 tx=0x7fbbd000b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.632+0000 7fbbcaffd640 1 -- 172.21.15.66:0/3093565853 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbcc0bf100 con 0x7fbbd419cf70 2024-04-21T09:08:20.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.817+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbbd41016d0 con 0x7fbbd419cf70 2024-04-21T09:08:20.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.818+0000 7fbbcaffd640 1 -- 172.21.15.66:0/3093565853 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7fbbcc0bea00 con 0x7fbbd419cf70 2024-04-21T09:08:20.818 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbc00a3db0 msgr2=0x7fbbc00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbc00a3db0 0x7fbbc00a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005d00 tx=0x7fbbd000b410 comp rx=0 tx=0).stop 2024-04-21T09:08:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd419cf70 msgr2=0x7fbbd419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd419cf70 0x7fbbd419d420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc005e30 tx=0x7fbbcc034760 comp rx=0 tx=0).stop 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 shutdown_connections 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbc00a3db0 0x7fbbc00a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd419cf70 0x7fbbd419d420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd410c4f0 0x7fbbd4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 --2- 172.21.15.66:0/3093565853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd4100d20 0x7fbbd4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 >> 172.21.15.66:0/3093565853 conn(0x7fbbd4076150 msgr2=0x7fbbd40fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 shutdown_connections 2024-04-21T09:08:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.820+0000 7fbbdba69640 1 -- 172.21.15.66:0/3093565853 wait complete. 2024-04-21T09:08:20.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.932+0000 7fd5c2175640 1 -- 172.21.15.66:0/3786586701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc0690e0 msgr2=0x7fd5bc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.932+0000 7fd5c2175640 1 --2- 172.21.15.66:0/3786586701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc0690e0 0x7fd5bc069560 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009930 tx=0x7fd5b002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 -- 172.21.15.66:0/3786586701 shutdown_connections 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 --2- 172.21.15.66:0/3786586701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc069aa0 0x7fd5bc10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 --2- 172.21.15.66:0/3786586701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc0690e0 0x7fd5bc069560 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 --2- 172.21.15.66:0/3786586701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5bc107fd0 0x7fd5bc1083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 -- 172.21.15.66:0/3786586701 >> 172.21.15.66:0/3786586701 conn(0x7fd5bc077e30 msgr2=0x7fd5bc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 -- 172.21.15.66:0/3786586701 shutdown_connections 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 -- 172.21.15.66:0/3786586701 wait complete. 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 Processor -- start 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.933+0000 7fd5c2175640 1 -- start start 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5bc0690e0 0x7fd5bc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc069aa0 0x7fd5bc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc107fd0 0x7fd5bc19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc112140 con 0x7fd5bc069aa0 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc111fc0 con 0x7fd5bc0690e0 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1122c0 con 0x7fd5bc107fd0 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc069aa0 0x7fd5bc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc069aa0 0x7fd5bc198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33110/0 (socket says 172.21.15.66:33110) 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 -- 172.21.15.66:0/1376485406 learned_addr learned my addr 172.21.15.66:0/1376485406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5bb7fe640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5bc0690e0 0x7fd5bc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5bbfff640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc107fd0 0x7fd5bc19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 -- 172.21.15.66:0/1376485406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc107fd0 msgr2=0x7fd5bc19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc107fd0 0x7fd5bc19cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 -- 172.21.15.66:0/1376485406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5bc0690e0 msgr2=0x7fd5bc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5bc0690e0 0x7fd5bc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 -- 172.21.15.66:0/1376485406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc19d6a0 con 0x7fd5bc069aa0 2024-04-21T09:08:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5baffd640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc069aa0 0x7fd5bc198c40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009230 tx=0x7fd5b002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5b8ff9640 1 -- 172.21.15.66:0/1376485406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0043020 con 0x7fd5bc069aa0 2024-04-21T09:08:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5b8ff9640 1 -- 172.21.15.66:0/1376485406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b002caf0 con 0x7fd5bc069aa0 2024-04-21T09:08:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc19d930 con 0x7fd5bc069aa0 2024-04-21T09:08:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5b8ff9640 1 -- 172.21.15.66:0/1376485406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003af0 con 0x7fd5bc069aa0 2024-04-21T09:08:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.934+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc1a5220 con 0x7fd5bc069aa0 2024-04-21T09:08:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.936+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc068de0 con 0x7fd5bc069aa0 2024-04-21T09:08:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.937+0000 7fd5b8ff9640 1 -- 172.21.15.66:0/1376485406 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fd5b0046020 con 0x7fd5bc069aa0 2024-04-21T09:08:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.938+0000 7fd5b8ff9640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd5a40a3e30 0x7fd5a40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.938+0000 7fd5b8ff9640 1 -- 172.21.15.66:0/1376485406 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd5b00f3030 con 0x7fd5bc069aa0 2024-04-21T09:08:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.938+0000 7fd5bb7fe640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd5a40a3e30 0x7fd5a40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.939+0000 7fd5bb7fe640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd5a40a3e30 0x7fd5a40a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009bd0 tx=0x7fd5ac005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:20.942+0000 7fd5b8ff9640 1 -- 172.21.15.66:0/1376485406 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5b004d020 con 0x7fd5bc069aa0 2024-04-21T09:08:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.131+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5bc199a00 con 0x7fd5bc069aa0 2024-04-21T09:08:21.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.133+0000 7fd5b8ff9640 1 -- 172.21.15.66:0/1376485406 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+25086 (secure 0 0 0) 0x7fd5bc199a00 con 0x7fd5bc069aa0 2024-04-21T09:08:21.133 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:21.133 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:08:12.028415+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd5a40a3e30 msgr2=0x7fd5a40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd5a40a3e30 0x7fd5a40a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009bd0 tx=0x7fd5ac005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc069aa0 msgr2=0x7fd5bc198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc069aa0 0x7fd5bc198c40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009230 tx=0x7fd5b002c710 comp rx=0 tx=0).stop 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 shutdown_connections 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd5a40a3e30 0x7fd5a40a62f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc107fd0 0x7fd5bc19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5bc069aa0 0x7fd5bc198c40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 --2- 172.21.15.66:0/1376485406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5bc0690e0 0x7fd5bc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 >> 172.21.15.66:0/1376485406 conn(0x7fd5bc077e30 msgr2=0x7fd5bc100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 shutdown_connections 2024-04-21T09:08:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.135+0000 7fd5c2175640 1 -- 172.21.15.66:0/1376485406 wait complete. 2024-04-21T09:08:21.146 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:08:21.147 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:08:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.251+0000 7f18d6f2b640 1 -- 172.21.15.66:0/2803186791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d006b1d0 msgr2=0x7f18d0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.251+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/2803186791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d006b1d0 0x7f18d0106b90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009900 tx=0x7f18c002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.251+0000 7f18d6f2b640 1 -- 172.21.15.66:0/2803186791 shutdown_connections 2024-04-21T09:08:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.251+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/2803186791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d0107270 0x7f18d010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.251+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/2803186791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d006b1d0 0x7f18d0106b90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.251+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/2803186791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d006a820 0x7f18d006ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.251+0000 7f18d6f2b640 1 -- 172.21.15.66:0/2803186791 >> 172.21.15.66:0/2803186791 conn(0x7f18d0077eb0 msgr2=0x7f18d00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 -- 172.21.15.66:0/2803186791 shutdown_connections 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 -- 172.21.15.66:0/2803186791 wait complete. 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 Processor -- start 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 -- start start 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d006a820 0x7f18d0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d0107270 0x7f18d0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d019cfd0 0x7f18d019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112190 con 0x7f18d0107270 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112010 con 0x7f18d019cfd0 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18d6f2b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112310 con 0x7f18d006a820 2024-04-21T09:08:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d0107270 0x7f18d0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d0107270 0x7f18d0198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33134/0 (socket says 172.21.15.66:33134) 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.252+0000 7f18cffff640 1 -- 172.21.15.66:0/1831340429 learned_addr learned my addr 172.21.15.66:0/1831340429 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18d54c8640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d019cfd0 0x7f18d019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18d4cc7640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d006a820 0x7f18d0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cffff640 1 -- 172.21.15.66:0/1831340429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d006a820 msgr2=0x7f18d0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cffff640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d006a820 0x7f18d0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cffff640 1 -- 172.21.15.66:0/1831340429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d019cfd0 msgr2=0x7f18d019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cffff640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d019cfd0 0x7f18d019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cffff640 1 -- 172.21.15.66:0/1831340429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d019dac0 con 0x7f18d0107270 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cffff640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d0107270 0x7f18d0198c50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f18c0036040 tx=0x7f18c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cdffb640 1 -- 172.21.15.66:0/1831340429 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0043020 con 0x7f18d0107270 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cdffb640 1 -- 172.21.15.66:0/1831340429 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c002ca70 con 0x7f18d0107270 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18cdffb640 1 -- 172.21.15.66:0/1831340429 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0003ac0 con 0x7f18d0107270 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d019dd50 con 0x7f18d0107270 2024-04-21T09:08:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.253+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a5570 con 0x7f18d0107270 2024-04-21T09:08:21.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.254+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d00691c0 con 0x7f18d0107270 2024-04-21T09:08:21.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.255+0000 7f18cdffb640 1 -- 172.21.15.66:0/1831340429 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f18c0046020 con 0x7f18d0107270 2024-04-21T09:08:21.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.256+0000 7f18cdffb640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f18bc0a3db0 0x7f18bc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.256+0000 7f18cdffb640 1 -- 172.21.15.66:0/1831340429 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f18c00f2d30 con 0x7f18d0107270 2024-04-21T09:08:21.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.256+0000 7f18d4cc7640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f18bc0a3db0 0x7f18bc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.257+0000 7f18d4cc7640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f18bc0a3db0 0x7f18bc0a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009b60 tx=0x7f18b8005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:21.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.261+0000 7f18cdffb640 1 -- 172.21.15.66:0/1831340429 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18c004d020 con 0x7f18d0107270 2024-04-21T09:08:21.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.446+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f18d006b1d0 con 0x7f18d0107270 2024-04-21T09:08:21.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.446+0000 7f18cdffb640 1 -- 172.21.15.66:0/1831340429 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f18c002cc20 con 0x7f18d0107270 2024-04-21T09:08:21.447 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:08:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f18bc0a3db0 msgr2=0x7f18bc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f18bc0a3db0 0x7f18bc0a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009b60 tx=0x7f18b8005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d0107270 msgr2=0x7f18d0198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d0107270 0x7f18d0198c50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f18c0036040 tx=0x7f18c00045e0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 shutdown_connections 2024-04-21T09:08:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f18bc0a3db0 0x7f18bc0a6270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d019cfd0 0x7f18d019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18d0107270 0x7f18d0198c50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 --2- 172.21.15.66:0/1831340429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d006a820 0x7f18d0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 >> 172.21.15.66:0/1831340429 conn(0x7f18d0077eb0 msgr2=0x7f18d0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 shutdown_connections 2024-04-21T09:08:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.448+0000 7f18d6f2b640 1 -- 172.21.15.66:0/1831340429 wait complete. 2024-04-21T09:08:21.458 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:21 smithi066 ceph-mon[27953]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 93 KiB/s wr, 47 op/s 2024-04-21T09:08:21.459 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:21 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1135180927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:21.459 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:21 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3093565853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:21.459 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:21 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1376485406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:21.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:08:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 -- 172.21.15.66:0/842933912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd654102b90 msgr2=0x7fd654106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 --2- 172.21.15.66:0/842933912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd654102b90 0x7fd654106bd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009900 tx=0x7fd63c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 -- 172.21.15.66:0/842933912 shutdown_connections 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 --2- 172.21.15.66:0/842933912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6541072b0 0x7fd65410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 --2- 172.21.15.66:0/842933912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd654102b90 0x7fd654106bd0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 --2- 172.21.15.66:0/842933912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6541021e0 0x7fd6541025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 -- 172.21.15.66:0/842933912 >> 172.21.15.66:0/842933912 conn(0x7fd6540fda60 msgr2=0x7fd6540ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 -- 172.21.15.66:0/842933912 shutdown_connections 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.566+0000 7fd65af8f640 1 -- 172.21.15.66:0/842933912 wait complete. 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 Processor -- start 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 -- start start 2024-04-21T09:08:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6541021e0 0x7fd6541987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6541072b0 0x7fd654198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65419d010 0x7fd65419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd654112220 con 0x7fd65419d010 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6541120a0 con 0x7fd6541021e0 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65af8f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd6541123a0 con 0x7fd6541072b0 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65419d010 0x7fd65419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd658d2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6541021e0 0x7fd6541987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65419d010 0x7fd65419d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33160/0 (socket says 172.21.15.66:33160) 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 -- 172.21.15.66:0/1357741933 learned_addr learned my addr 172.21.15.66:0/1357741933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd653fff640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6541072b0 0x7fd654198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 -- 172.21.15.66:0/1357741933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6541072b0 msgr2=0x7fd654198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6541072b0 0x7fd654198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 -- 172.21.15.66:0/1357741933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6541021e0 msgr2=0x7fd6541987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6541021e0 0x7fd6541987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd65952c640 1 -- 172.21.15.66:0/1357741933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65419db00 con 0x7fd65419d010 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd653fff640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6541072b0 0x7fd654198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.567+0000 7fd658d2b640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6541021e0 0x7fd6541987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.568+0000 7fd65952c640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65419d010 0x7fd65419d4c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00eb10 tx=0x7fd64c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.568+0000 7fd651ffb640 1 -- 172.21.15.66:0/1357741933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00ce20 con 0x7fd65419d010 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.568+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65419ddf0 con 0x7fd65419d010 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.568+0000 7fd651ffb640 1 -- 172.21.15.66:0/1357741933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd64c005020 con 0x7fd65419d010 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.568+0000 7fd651ffb640 1 -- 172.21.15.66:0/1357741933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c010690 con 0x7fd65419d010 2024-04-21T09:08:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.568+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6541a5560 con 0x7fd65419d010 2024-04-21T09:08:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.569+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6540691c0 con 0x7fd65419d010 2024-04-21T09:08:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.570+0000 7fd651ffb640 1 -- 172.21.15.66:0/1357741933 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fd64c005510 con 0x7fd65419d010 2024-04-21T09:08:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.571+0000 7fd651ffb640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd6400a3db0 0x7fd6400a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.571+0000 7fd658d2b640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd6400a3db0 0x7fd6400a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.571+0000 7fd651ffb640 1 -- 172.21.15.66:0/1357741933 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd64c0d12c0 con 0x7fd65419d010 2024-04-21T09:08:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.571+0000 7fd658d2b640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd6400a3db0 0x7fd6400a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd64400afb0 tx=0x7fd64400a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.576+0000 7fd651ffb640 1 -- 172.21.15.66:0/1357741933 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd64c09d480 con 0x7fd65419d010 2024-04-21T09:08:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:21 smithi186 ceph-mon[31505]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 93 KiB/s wr, 47 op/s 2024-04-21T09:08:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:21 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1135180927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:21 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3093565853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:21 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1376485406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:21.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:21 smithi083 ceph-mon[31413]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 93 KiB/s wr, 47 op/s 2024-04-21T09:08:21.611 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:21 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1135180927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:21.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:21 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3093565853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:21.612 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:21 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1376485406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:21.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.763+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd65410a4e0 con 0x7fd65419d010 2024-04-21T09:08:21.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.763+0000 7fd651ffb640 1 -- 172.21.15.66:0/1357741933 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7fd64c09cd80 con 0x7fd65419d010 2024-04-21T09:08:21.764 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:21.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd6400a3db0 msgr2=0x7fd6400a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd6400a3db0 0x7fd6400a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd64400afb0 tx=0x7fd64400a2d0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65419d010 msgr2=0x7fd65419d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65419d010 0x7fd65419d4c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00eb10 tx=0x7fd64c00ef10 comp rx=0 tx=0).stop 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 shutdown_connections 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd6400a3db0 0x7fd6400a6270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65419d010 0x7fd65419d4c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6541072b0 0x7fd654198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 --2- 172.21.15.66:0/1357741933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6541021e0 0x7fd6541987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 >> 172.21.15.66:0/1357741933 conn(0x7fd6540fda60 msgr2=0x7fd6540fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 shutdown_connections 2024-04-21T09:08:21.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.766+0000 7fd65af8f640 1 -- 172.21.15.66:0/1357741933 wait complete. 2024-04-21T09:08:21.777 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 -- 172.21.15.66:0/2791274486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa90102190 msgr2=0x7faa90102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 --2- 172.21.15.66:0/2791274486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa90102190 0x7faa90102570 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009900 tx=0x7faa8c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 -- 172.21.15.66:0/2791274486 shutdown_connections 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 --2- 172.21.15.66:0/2791274486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa90107260 0x7faa9010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 --2- 172.21.15.66:0/2791274486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa90102b40 0x7faa90106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 --2- 172.21.15.66:0/2791274486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa90102190 0x7faa90102570 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 -- 172.21.15.66:0/2791274486 >> 172.21.15.66:0/2791274486 conn(0x7faa900fd990 msgr2=0x7faa900ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 -- 172.21.15.66:0/2791274486 shutdown_connections 2024-04-21T09:08:21.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.881+0000 7faa979b6640 1 -- 172.21.15.66:0/2791274486 wait complete. 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 Processor -- start 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 -- start start 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa90102190 0x7faa9019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa90107260 0x7faa9019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa901985d0 0x7faa90198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa901121f0 con 0x7faa901985d0 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa90112070 con 0x7faa90107260 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa979b6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faa90112370 con 0x7faa90102190 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa95f53640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa901985d0 0x7faa90198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa95f53640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa901985d0 0x7faa90198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33178/0 (socket says 172.21.15.66:33178) 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa95f53640 1 -- 172.21.15.66:0/3228000166 learned_addr learned my addr 172.21.15.66:0/3228000166 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa95752640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa90102190 0x7faa9019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.882+0000 7faa94f51640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa90107260 0x7faa9019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa95f53640 1 -- 172.21.15.66:0/3228000166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa90102190 msgr2=0x7faa9019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa95f53640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa90102190 0x7faa9019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa95f53640 1 -- 172.21.15.66:0/3228000166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa90107260 msgr2=0x7faa9019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa95f53640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa90107260 0x7faa9019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa95f53640 1 -- 172.21.15.66:0/3228000166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa901990f0 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa95f53640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa901985d0 0x7faa90198a80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faa8800eaa0 tx=0x7faa8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa867fc640 1 -- 172.21.15.66:0/3228000166 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8800cdb0 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa867fc640 1 -- 172.21.15.66:0/3228000166 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa88005020 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa901993e0 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa867fc640 1 -- 172.21.15.66:0/3228000166 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880116d0 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.883+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a5680 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.884+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa900691c0 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.885+0000 7faa867fc640 1 -- 172.21.15.66:0/3228000166 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7faa880051d0 con 0x7faa901985d0 2024-04-21T09:08:21.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.886+0000 7faa867fc640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa7c0a3e30 0x7faa7c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:21.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.886+0000 7faa95752640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa7c0a3e30 0x7faa7c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:21.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.886+0000 7faa867fc640 1 -- 172.21.15.66:0/3228000166 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7faa880d0dd0 con 0x7faa901985d0 2024-04-21T09:08:21.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.886+0000 7faa95752640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa7c0a3e30 0x7faa7c0a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa8c006010 tx=0x7faa8c005e40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:21.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:21.891+0000 7faa867fc640 1 -- 172.21.15.66:0/3228000166 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa880d4030 con 0x7faa901985d0 2024-04-21T09:08:22.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.077+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa90102b40 con 0x7faa901985d0 2024-04-21T09:08:22.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.077+0000 7faa867fc640 1 -- 172.21.15.66:0/3228000166 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7faa88016030 con 0x7faa901985d0 2024-04-21T09:08:22.078 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:22.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa7c0a3e30 msgr2=0x7faa7c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa7c0a3e30 0x7faa7c0a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa8c006010 tx=0x7faa8c005e40 comp rx=0 tx=0).stop 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa901985d0 msgr2=0x7faa90198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa901985d0 0x7faa90198a80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faa8800eaa0 tx=0x7faa8800ef60 comp rx=0 tx=0).stop 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 shutdown_connections 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7faa7c0a3e30 0x7faa7c0a62f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa901985d0 0x7faa90198a80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa90107260 0x7faa9019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 --2- 172.21.15.66:0/3228000166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa90102190 0x7faa9019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 >> 172.21.15.66:0/3228000166 conn(0x7faa900fd990 msgr2=0x7faa900fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 shutdown_connections 2024-04-21T09:08:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.079+0000 7faa979b6640 1 -- 172.21.15.66:0/3228000166 wait complete. 2024-04-21T09:08:22.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 -- 172.21.15.66:0/4148547522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e01010a0 msgr2=0x7f38e0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 --2- 172.21.15.66:0/4148547522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e01010a0 0x7f38e0101480 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009900 tx=0x7f38d002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 -- 172.21.15.66:0/4148547522 shutdown_connections 2024-04-21T09:08:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 --2- 172.21.15.66:0/4148547522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e0106170 0x7f38e010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 --2- 172.21.15.66:0/4148547522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e0101a50 0x7f38e0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 --2- 172.21.15.66:0/4148547522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e01010a0 0x7f38e0101480 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 -- 172.21.15.66:0/4148547522 >> 172.21.15.66:0/4148547522 conn(0x7f38e0077eb0 msgr2=0x7f38e00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 -- 172.21.15.66:0/4148547522 shutdown_connections 2024-04-21T09:08:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.195+0000 7f38e5081640 1 -- 172.21.15.66:0/4148547522 wait complete. 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 Processor -- start 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 -- start start 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e01010a0 0x7f38e0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e0106170 0x7f38e0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e019d010 0x7f38e019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01110e0 con 0x7f38e019d010 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38e0110f60 con 0x7f38e0106170 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38e5081640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f38e0111260 con 0x7f38e01010a0 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e019d010 0x7f38e019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e019d010 0x7f38e019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33196/0 (socket says 172.21.15.66:33196) 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 -- 172.21.15.66:0/2104094628 learned_addr learned my addr 172.21.15.66:0/2104094628 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38ded9d640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e01010a0 0x7f38e0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38d7fff640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e0106170 0x7f38e0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 -- 172.21.15.66:0/2104094628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e01010a0 msgr2=0x7f38e0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e01010a0 0x7f38e0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 -- 172.21.15.66:0/2104094628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e0106170 msgr2=0x7f38e0198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e0106170 0x7f38e0198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 -- 172.21.15.66:0/2104094628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e019db00 con 0x7f38e019d010 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.196+0000 7f38df59e640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e019d010 0x7f38e019d4c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f38d800cad0 tx=0x7f38d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.197+0000 7f38dcd99640 1 -- 172.21.15.66:0/2104094628 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d800ae20 con 0x7f38e019d010 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.197+0000 7f38dcd99640 1 -- 172.21.15.66:0/2104094628 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d8005020 con 0x7f38e019d010 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.197+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e019ddf0 con 0x7f38e019d010 2024-04-21T09:08:22.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.197+0000 7f38dcd99640 1 -- 172.21.15.66:0/2104094628 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d800f6b0 con 0x7f38e019d010 2024-04-21T09:08:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.197+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e01a55e0 con 0x7f38e019d010 2024-04-21T09:08:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.198+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e00691c0 con 0x7f38e019d010 2024-04-21T09:08:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.199+0000 7f38dcd99640 1 -- 172.21.15.66:0/2104094628 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f38d80051d0 con 0x7f38e019d010 2024-04-21T09:08:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.199+0000 7f38dcd99640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f38cc0a3db0 0x7f38cc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.200+0000 7f38ded9d640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f38cc0a3db0 0x7f38cc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.200+0000 7f38dcd99640 1 -- 172.21.15.66:0/2104094628 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f38d80d0de0 con 0x7f38e019d010 2024-04-21T09:08:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.200+0000 7f38ded9d640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f38cc0a3db0 0x7f38cc0a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38d0000c00 tx=0x7f38d0004040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.204+0000 7f38dcd99640 1 -- 172.21.15.66:0/2104094628 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38d8014030 con 0x7f38e019d010 2024-04-21T09:08:22.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1831340429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1357741933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3228000166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.392+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f38e0101a50 con 0x7f38e019d010 2024-04-21T09:08:22.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.393+0000 7f38dcd99640 1 -- 172.21.15.66:0/2104094628 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f38d809cf60 con 0x7f38e019d010 2024-04-21T09:08:22.393 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f38cc0a3db0 msgr2=0x7f38cc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f38cc0a3db0 0x7f38cc0a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38d0000c00 tx=0x7f38d0004040 comp rx=0 tx=0).stop 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e019d010 msgr2=0x7f38e019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e019d010 0x7f38e019d4c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f38d800cad0 tx=0x7f38d800cf90 comp rx=0 tx=0).stop 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 shutdown_connections 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f38cc0a3db0 0x7f38cc0a6270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38e019d010 0x7f38e019d4c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e0106170 0x7f38e0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 --2- 172.21.15.66:0/2104094628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e01010a0 0x7f38e0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 >> 172.21.15.66:0/2104094628 conn(0x7f38e0077eb0 msgr2=0x7f38e006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 shutdown_connections 2024-04-21T09:08:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.395+0000 7f38e5081640 1 -- 172.21.15.66:0/2104094628 wait complete. 2024-04-21T09:08:22.406 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-21T09:08:22.406 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:08:22.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 -- 172.21.15.66:0/3637822655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c241031d0 msgr2=0x7f4c241035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 --2- 172.21.15.66:0/3637822655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c241031d0 0x7f4c241035b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009900 tx=0x7f4c2002bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 -- 172.21.15.66:0/3637822655 shutdown_connections 2024-04-21T09:08:22.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 --2- 172.21.15.66:0/3637822655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c2410c5a0 0x7f4c2410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 --2- 172.21.15.66:0/3637822655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c24103af0 0x7f4c2410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 --2- 172.21.15.66:0/3637822655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c241031d0 0x7f4c241035b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 -- 172.21.15.66:0/3637822655 >> 172.21.15.66:0/3637822655 conn(0x7f4c240771d0 msgr2=0x7f4c240775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 -- 172.21.15.66:0/3637822655 shutdown_connections 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.507+0000 7f4c2b923640 1 -- 172.21.15.66:0/3637822655 wait complete. 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 Processor -- start 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 -- start start 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c241031d0 0x7f4c241986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c24103af0 0x7f4c24198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c2410c5a0 0x7f4c2419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241121d0 con 0x7f4c2410c5a0 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24112050 con 0x7f4c24103af0 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c2b923640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24112350 con 0x7f4c241031d0 2024-04-21T09:08:22.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c29ec0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c2410c5a0 0x7f4c2419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c29ec0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c2410c5a0 0x7f4c2419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33224/0 (socket says 172.21.15.66:33224) 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c29ec0640 1 -- 172.21.15.66:0/1939498442 learned_addr learned my addr 172.21.15.66:0/1939498442 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c28ebe640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c24103af0 0x7f4c24198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c296bf640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c241031d0 0x7f4c241986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c29ec0640 1 -- 172.21.15.66:0/1939498442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c241031d0 msgr2=0x7f4c241986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c29ec0640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c241031d0 0x7f4c241986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c29ec0640 1 -- 172.21.15.66:0/1939498442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c24103af0 msgr2=0x7f4c24198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.508+0000 7f4c29ec0640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c24103af0 0x7f4c24198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.509+0000 7f4c29ec0640 1 -- 172.21.15.66:0/1939498442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2419d6c0 con 0x7f4c2410c5a0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.509+0000 7f4c29ec0640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c2410c5a0 0x7f4c2419cf90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ccb0 tx=0x7f4c1c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.509+0000 7f4c1a7fc640 1 -- 172.21.15.66:0/1939498442 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00f450 con 0x7f4c2410c5a0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.509+0000 7f4c1a7fc640 1 -- 172.21.15.66:0/1939498442 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c1c00faa0 con 0x7f4c2410c5a0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.509+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2419d9b0 con 0x7f4c2410c5a0 2024-04-21T09:08:22.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.509+0000 7f4c1a7fc640 1 -- 172.21.15.66:0/1939498442 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c018b00 con 0x7f4c2410c5a0 2024-04-21T09:08:22.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.509+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241a5240 con 0x7f4c2410c5a0 2024-04-21T09:08:22.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.510+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c24069230 con 0x7f4c2410c5a0 2024-04-21T09:08:22.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.511+0000 7f4c1a7fc640 1 -- 172.21.15.66:0/1939498442 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f4c1c00fc50 con 0x7f4c2410c5a0 2024-04-21T09:08:22.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.512+0000 7f4c1a7fc640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c100a3e30 0x7f4c100a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.512+0000 7f4c296bf640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c100a3e30 0x7f4c100a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.512+0000 7f4c1a7fc640 1 -- 172.21.15.66:0/1939498442 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4c1c0d1410 con 0x7f4c2410c5a0 2024-04-21T09:08:22.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.512+0000 7f4c296bf640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c100a3e30 0x7f4c100a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c20000c00 tx=0x7f4c20004040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:22.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.516+0000 7f4c1a7fc640 1 -- 172.21.15.66:0/1939498442 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c1c014030 con 0x7f4c2410c5a0 2024-04-21T09:08:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:22 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1831340429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:22 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1357741933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:22 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3228000166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:22 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1831340429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:22 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1357741933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:22 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3228000166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:22.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.701+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c24062390 con 0x7f4c100a3e30 2024-04-21T09:08:22.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.711+0000 7f4c1a7fc640 1 -- 172.21.15.66:0/1939498442 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339717 (secure 0 0 0) 0x7f4c24062390 con 0x7f4c100a3e30 2024-04-21T09:08:22.715 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:22.717 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c100a3e30 msgr2=0x7f4c100a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c100a3e30 0x7f4c100a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c20000c00 tx=0x7f4c20004040 comp rx=0 tx=0).stop 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c2410c5a0 msgr2=0x7f4c2419cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c2410c5a0 0x7f4c2419cf90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ccb0 tx=0x7f4c1c00a620 comp rx=0 tx=0).stop 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 shutdown_connections 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4c100a3e30 0x7f4c100a62f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c2410c5a0 0x7f4c2419cf90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c24103af0 0x7f4c24198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 --2- 172.21.15.66:0/1939498442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c241031d0 0x7f4c241986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 >> 172.21.15.66:0/1939498442 conn(0x7f4c240771d0 msgr2=0x7f4c240fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:22.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 shutdown_connections 2024-04-21T09:08:22.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.715+0000 7f4c2b923640 1 -- 172.21.15.66:0/1939498442 wait complete. 2024-04-21T09:08:22.726 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T09:08:22.049328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384178,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1213,"num_read_kb":975,"num_write":554,"num_write_kb":5030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":919,"ondisk_log_size":919,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357992,"kb_used_data":12968,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583832,"statfs":{"total":1151940427776,"available":1151573843968,"internally_reserved":0,"allocated":13279232,"data_stored":10041428,"data_compressed":577260,"data_compressed_allocated":786432,"data_compressed_original":4423680,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":246,"num_read_kb":331,"num_write":84,"num_write_kb":652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991412"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764225+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:08:12.764225+0000","last_peered":"2024-04-21T09:08:12.764225+0000","last_clean":"2024-04-21T09:08:12.764225+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:08:12.764225+0000","last_undegraded":"2024-04-21T09:08:12.764225+0000","last_fullsized":"2024-04-21T09:08:12.764225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.634069+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:08:13.634069+0000","last_peered":"2024-04-21T09:08:13.634069+0000","last_clean":"2024-04-21T09:08:13.634069+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:08:13.634069+0000","last_undegraded":"2024-04-21T09:08:13.634069+0000","last_fullsized":"2024-04-21T09:08:13.634069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633957+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:08:13.633957+0000","last_peered":"2024-04-21T09:08:13.633957+0000","last_clean":"2024-04-21T09:08:13.633957+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:08:13.633957+0000","last_undegraded":"2024-04-21T09:08:13.633957+0000","last_fullsized":"2024-04-21T09:08:13.633957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281097+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:08:12.281097+0000","last_peered":"2024-04-21T09:08:12.281097+0000","last_clean":"2024-04-21T09:08:12.281097+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:08:12.281097+0000","last_undegraded":"2024-04-21T09:08:12.281097+0000","last_fullsized":"2024-04-21T09:08:12.281097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142249+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:08:12.142249+0000","last_peered":"2024-04-21T09:08:12.142249+0000","last_clean":"2024-04-21T09:08:12.142249+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:08:12.142249+0000","last_undegraded":"2024-04-21T09:08:12.142249+0000","last_fullsized":"2024-04-21T09:08:12.142249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034661+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:08:12.034661+0000","last_peered":"2024-04-21T09:08:12.034661+0000","last_clean":"2024-04-21T09:08:12.034661+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:08:12.034661+0000","last_undegraded":"2024-04-21T09:08:12.034661+0000","last_fullsized":"2024-04-21T09:08:12.034661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324873+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:08:12.324873+0000","last_peered":"2024-04-21T09:08:12.324873+0000","last_clean":"2024-04-21T09:08:12.324873+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:08:12.324873+0000","last_undegraded":"2024-04-21T09:08:12.324873+0000","last_fullsized":"2024-04-21T09:08:12.324873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142696+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:08:12.142696+0000","last_peered":"2024-04-21T09:08:12.142696+0000","last_clean":"2024-04-21T09:08:12.142696+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:08:12.142696+0000","last_undegraded":"2024-04-21T09:08:12.142696+0000","last_fullsized":"2024-04-21T09:08:12.142696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325536+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:08:12.325536+0000","last_peered":"2024-04-21T09:08:12.325536+0000","last_clean":"2024-04-21T09:08:12.325536+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:08:12.325536+0000","last_undegraded":"2024-04-21T09:08:12.325536+0000","last_fullsized":"2024-04-21T09:08:12.325536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035076+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:08:12.035076+0000","last_peered":"2024-04-21T09:08:12.035076+0000","last_clean":"2024-04-21T09:08:12.035076+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:08:12.035076+0000","last_undegraded":"2024-04-21T09:08:12.035076+0000","last_fullsized":"2024-04-21T09:08:12.035076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325727+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:08:12.325727+0000","last_peered":"2024-04-21T09:08:12.325727+0000","last_clean":"2024-04-21T09:08:12.325727+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:08:12.325727+0000","last_undegraded":"2024-04-21T09:08:12.325727+0000","last_fullsized":"2024-04-21T09:08:12.325727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281072+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:08:12.281072+0000","last_peered":"2024-04-21T09:08:12.281072+0000","last_clean":"2024-04-21T09:08:12.281072+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:08:12.281072+0000","last_undegraded":"2024-04-21T09:08:12.281072+0000","last_fullsized":"2024-04-21T09:08:12.281072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033565+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:08:12.033565+0000","last_peered":"2024-04-21T09:08:12.033565+0000","last_clean":"2024-04-21T09:08:12.033565+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:08:12.033565+0000","last_undegraded":"2024-04-21T09:08:12.033565+0000","last_fullsized":"2024-04-21T09:08:12.033565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633566+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:08:13.633566+0000","last_peered":"2024-04-21T09:08:13.633566+0000","last_clean":"2024-04-21T09:08:13.633566+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:08:13.633566+0000","last_undegraded":"2024-04-21T09:08:13.633566+0000","last_fullsized":"2024-04-21T09:08:13.633566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633612+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:08:13.633612+0000","last_peered":"2024-04-21T09:08:13.633612+0000","last_clean":"2024-04-21T09:08:13.633612+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:08:13.633612+0000","last_undegraded":"2024-04-21T09:08:13.633612+0000","last_fullsized":"2024-04-21T09:08:13.633612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323842+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:08:12.323842+0000","last_peered":"2024-04-21T09:08:12.323842+0000","last_clean":"2024-04-21T09:08:12.323842+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:08:12.323842+0000","last_undegraded":"2024-04-21T09:08:12.323842+0000","last_fullsized":"2024-04-21T09:08:12.323842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764134+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:08:12.764134+0000","last_peered":"2024-04-21T09:08:12.764134+0000","last_clean":"2024-04-21T09:08:12.764134+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:08:12.764134+0000","last_undegraded":"2024-04-21T09:08:12.764134+0000","last_fullsized":"2024-04-21T09:08:12.764134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140490+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:08:12.140490+0000","last_peered":"2024-04-21T09:08:12.140490+0000","last_clean":"2024-04-21T09:08:12.140490+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:08:12.140490+0000","last_undegraded":"2024-04-21T09:08:12.140490+0000","last_fullsized":"2024-04-21T09:08:12.140490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035166+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:08:12.035166+0000","last_peered":"2024-04-21T09:08:12.035166+0000","last_clean":"2024-04-21T09:08:12.035166+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:08:12.035166+0000","last_undegraded":"2024-04-21T09:08:12.035166+0000","last_fullsized":"2024-04-21T09:08:12.035166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034868+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:08:12.034868+0000","last_peered":"2024-04-21T09:08:12.034868+0000","last_clean":"2024-04-21T09:08:12.034868+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:08:12.034868+0000","last_undegraded":"2024-04-21T09:08:12.034868+0000","last_fullsized":"2024-04-21T09:08:12.034868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140333+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:08:12.140333+0000","last_peered":"2024-04-21T09:08:12.140333+0000","last_clean":"2024-04-21T09:08:12.140333+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:08:12.140333+0000","last_undegraded":"2024-04-21T09:08:12.140333+0000","last_fullsized":"2024-04-21T09:08:12.140333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143771+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:08:12.143771+0000","last_peered":"2024-04-21T09:08:12.143771+0000","last_clean":"2024-04-21T09:08:12.143771+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:08:12.143771+0000","last_undegraded":"2024-04-21T09:08:12.143771+0000","last_fullsized":"2024-04-21T09:08:12.143771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280764+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:08:12.280764+0000","last_peered":"2024-04-21T09:08:12.280764+0000","last_clean":"2024-04-21T09:08:12.280764+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:08:12.280764+0000","last_undegraded":"2024-04-21T09:08:12.280764+0000","last_fullsized":"2024-04-21T09:08:12.280764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.144097+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:08:12.144097+0000","last_peered":"2024-04-21T09:08:12.144097+0000","last_clean":"2024-04-21T09:08:12.144097+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:08:12.144097+0000","last_undegraded":"2024-04-21T09:08:12.144097+0000","last_fullsized":"2024-04-21T09:08:12.144097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633901+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:08:13.633901+0000","last_peered":"2024-04-21T09:08:13.633901+0000","last_clean":"2024-04-21T09:08:13.633901+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:08:13.633901+0000","last_undegraded":"2024-04-21T09:08:13.633901+0000","last_fullsized":"2024-04-21T09:08:13.633901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280878+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:08:12.280878+0000","last_peered":"2024-04-21T09:08:12.280878+0000","last_clean":"2024-04-21T09:08:12.280878+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:08:12.280878+0000","last_undegraded":"2024-04-21T09:08:12.280878+0000","last_fullsized":"2024-04-21T09:08:12.280878+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142503+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:08:12.142503+0000","last_peered":"2024-04-21T09:08:12.142503+0000","last_clean":"2024-04-21T09:08:12.142503+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:08:12.142503+0000","last_undegraded":"2024-04-21T09:08:12.142503+0000","last_fullsized":"2024-04-21T09:08:12.142503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325104+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:08:12.325104+0000","last_peered":"2024-04-21T09:08:12.325104+0000","last_clean":"2024-04-21T09:08:12.325104+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:08:12.325104+0000","last_undegraded":"2024-04-21T09:08:12.325104+0000","last_fullsized":"2024-04-21T09:08:12.325104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280773+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:08:12.280773+0000","last_peered":"2024-04-21T09:08:12.280773+0000","last_clean":"2024-04-21T09:08:12.280773+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:08:12.280773+0000","last_undegraded":"2024-04-21T09:08:12.280773+0000","last_fullsized":"2024-04-21T09:08:12.280773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142157+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:08:12.142157+0000","last_peered":"2024-04-21T09:08:12.142157+0000","last_clean":"2024-04-21T09:08:12.142157+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:08:12.142157+0000","last_undegraded":"2024-04-21T09:08:12.142157+0000","last_fullsized":"2024-04-21T09:08:12.142157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150927+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:08:12.150927+0000","last_peered":"2024-04-21T09:08:12.150927+0000","last_clean":"2024-04-21T09:08:12.150927+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:08:12.150927+0000","last_undegraded":"2024-04-21T09:08:12.150927+0000","last_fullsized":"2024-04-21T09:08:12.150927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034947+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:08:12.034947+0000","last_peered":"2024-04-21T09:08:12.034947+0000","last_clean":"2024-04-21T09:08:12.034947+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:08:12.034947+0000","last_undegraded":"2024-04-21T09:08:12.034947+0000","last_fullsized":"2024-04-21T09:08:12.034947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'6","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.321068+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:08:17.321068+0000","last_peered":"2024-04-21T09:08:17.321068+0000","last_clean":"2024-04-21T09:08:17.321068+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:08:17.321068+0000","last_undegraded":"2024-04-21T09:08:17.321068+0000","last_fullsized":"2024-04-21T09:08:17.321068+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142233+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:08:12.142233+0000","last_peered":"2024-04-21T09:08:12.142233+0000","last_clean":"2024-04-21T09:08:12.142233+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:08:12.142233+0000","last_undegraded":"2024-04-21T09:08:12.142233+0000","last_fullsized":"2024-04-21T09:08:12.142233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035266+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:08:12.035266+0000","last_peered":"2024-04-21T09:08:12.035266+0000","last_clean":"2024-04-21T09:08:12.035266+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:08:12.035266+0000","last_undegraded":"2024-04-21T09:08:12.035266+0000","last_fullsized":"2024-04-21T09:08:12.035266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034693+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:08:12.034693+0000","last_peered":"2024-04-21T09:08:12.034693+0000","last_clean":"2024-04-21T09:08:12.034693+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:08:12.034693+0000","last_undegraded":"2024-04-21T09:08:12.034693+0000","last_fullsized":"2024-04-21T09:08:12.034693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150778+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:08:12.150778+0000","last_peered":"2024-04-21T09:08:12.150778+0000","last_clean":"2024-04-21T09:08:12.150778+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:08:12.150778+0000","last_undegraded":"2024-04-21T09:08:12.150778+0000","last_fullsized":"2024-04-21T09:08:12.150778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763967+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:08:12.763967+0000","last_peered":"2024-04-21T09:08:12.763967+0000","last_clean":"2024-04-21T09:08:12.763967+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:08:12.763967+0000","last_undegraded":"2024-04-21T09:08:12.763967+0000","last_fullsized":"2024-04-21T09:08:12.763967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033440+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:08:12.033440+0000","last_peered":"2024-04-21T09:08:12.033440+0000","last_clean":"2024-04-21T09:08:12.033440+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:08:12.033440+0000","last_undegraded":"2024-04-21T09:08:12.033440+0000","last_fullsized":"2024-04-21T09:08:12.033440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142822+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:08:12.142822+0000","last_peered":"2024-04-21T09:08:12.142822+0000","last_clean":"2024-04-21T09:08:12.142822+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:08:12.142822+0000","last_undegraded":"2024-04-21T09:08:12.142822+0000","last_fullsized":"2024-04-21T09:08:12.142822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633942+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:08:13.633942+0000","last_peered":"2024-04-21T09:08:13.633942+0000","last_clean":"2024-04-21T09:08:13.633942+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:08:13.633942+0000","last_undegraded":"2024-04-21T09:08:13.633942+0000","last_fullsized":"2024-04-21T09:08:13.633942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150624+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:08:12.150624+0000","last_peered":"2024-04-21T09:08:12.150624+0000","last_clean":"2024-04-21T09:08:12.150624+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:08:12.150624+0000","last_undegraded":"2024-04-21T09:08:12.150624+0000","last_fullsized":"2024-04-21T09:08:12.150624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139637+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:08:12.139637+0000","last_peered":"2024-04-21T09:08:12.139637+0000","last_clean":"2024-04-21T09:08:12.139637+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:08:12.139637+0000","last_undegraded":"2024-04-21T09:08:12.139637+0000","last_fullsized":"2024-04-21T09:08:12.139637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142906+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:08:12.142906+0000","last_peered":"2024-04-21T09:08:12.142906+0000","last_clean":"2024-04-21T09:08:12.142906+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:08:12.142906+0000","last_undegraded":"2024-04-21T09:08:12.142906+0000","last_fullsized":"2024-04-21T09:08:12.142906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":165,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.319293+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:08:16.319293+0000","last_peered":"2024-04-21T09:08:16.319293+0000","last_clean":"2024-04-21T09:08:16.319293+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:08:16.319293+0000","last_undegraded":"2024-04-21T09:08:16.319293+0000","last_fullsized":"2024-04-21T09:08:16.319293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325164+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:08:12.325164+0000","last_peered":"2024-04-21T09:08:12.325164+0000","last_clean":"2024-04-21T09:08:12.325164+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:08:12.325164+0000","last_undegraded":"2024-04-21T09:08:12.325164+0000","last_fullsized":"2024-04-21T09:08:12.325164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139403+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:08:12.139403+0000","last_peered":"2024-04-21T09:08:12.139403+0000","last_clean":"2024-04-21T09:08:12.139403+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:08:12.139403+0000","last_undegraded":"2024-04-21T09:08:12.139403+0000","last_fullsized":"2024-04-21T09:08:12.139403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325630+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:08:12.325630+0000","last_peered":"2024-04-21T09:08:12.325630+0000","last_clean":"2024-04-21T09:08:12.325630+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:08:12.325630+0000","last_undegraded":"2024-04-21T09:08:12.325630+0000","last_fullsized":"2024-04-21T09:08:12.325630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.634168+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:08:13.634168+0000","last_peered":"2024-04-21T09:08:13.634168+0000","last_clean":"2024-04-21T09:08:13.634168+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:08:13.634168+0000","last_undegraded":"2024-04-21T09:08:13.634168+0000","last_fullsized":"2024-04-21T09:08:13.634168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150830+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:08:12.150830+0000","last_peered":"2024-04-21T09:08:12.150830+0000","last_clean":"2024-04-21T09:08:12.150830+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:08:12.150830+0000","last_undegraded":"2024-04-21T09:08:12.150830+0000","last_fullsized":"2024-04-21T09:08:12.150830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325080+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:08:12.325080+0000","last_peered":"2024-04-21T09:08:12.325080+0000","last_clean":"2024-04-21T09:08:12.325080+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:08:12.325080+0000","last_undegraded":"2024-04-21T09:08:12.325080+0000","last_fullsized":"2024-04-21T09:08:12.325080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139822+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:08:12.139822+0000","last_peered":"2024-04-21T09:08:12.139822+0000","last_clean":"2024-04-21T09:08:12.139822+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:08:12.139822+0000","last_undegraded":"2024-04-21T09:08:12.139822+0000","last_fullsized":"2024-04-21T09:08:12.139822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":337,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.320818+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:08:17.320818+0000","last_peered":"2024-04-21T09:08:17.320818+0000","last_clean":"2024-04-21T09:08:17.320818+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:08:17.320818+0000","last_undegraded":"2024-04-21T09:08:17.320818+0000","last_fullsized":"2024-04-21T09:08:17.320818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":262,"num_read_kb":198,"num_write":40,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142319+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:08:12.142319+0000","last_peered":"2024-04-21T09:08:12.142319+0000","last_clean":"2024-04-21T09:08:12.142319+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:08:12.142319+0000","last_undegraded":"2024-04-21T09:08:12.142319+0000","last_fullsized":"2024-04-21T09:08:12.142319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.151042+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:08:12.151042+0000","last_peered":"2024-04-21T09:08:12.151042+0000","last_clean":"2024-04-21T09:08:12.151042+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:08:12.151042+0000","last_undegraded":"2024-04-21T09:08:12.151042+0000","last_fullsized":"2024-04-21T09:08:12.151042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035051+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:08:12.035051+0000","last_peered":"2024-04-21T09:08:12.035051+0000","last_clean":"2024-04-21T09:08:12.035051+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:08:12.035051+0000","last_undegraded":"2024-04-21T09:08:12.035051+0000","last_fullsized":"2024-04-21T09:08:12.035051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033721+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:08:12.033721+0000","last_peered":"2024-04-21T09:08:12.033721+0000","last_clean":"2024-04-21T09:08:12.033721+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:08:12.033721+0000","last_undegraded":"2024-04-21T09:08:12.033721+0000","last_fullsized":"2024-04-21T09:08:12.033721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150756+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:08:12.150756+0000","last_peered":"2024-04-21T09:08:12.150756+0000","last_clean":"2024-04-21T09:08:12.150756+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:08:12.150756+0000","last_undegraded":"2024-04-21T09:08:12.150756+0000","last_fullsized":"2024-04-21T09:08:12.150756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325172+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:08:12.325172+0000","last_peered":"2024-04-21T09:08:12.325172+0000","last_clean":"2024-04-21T09:08:12.325172+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:08:12.325172+0000","last_undegraded":"2024-04-21T09:08:12.325172+0000","last_fullsized":"2024-04-21T09:08:12.325172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140230+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:08:12.140230+0000","last_peered":"2024-04-21T09:08:12.140230+0000","last_clean":"2024-04-21T09:08:12.140230+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:08:12.140230+0000","last_undegraded":"2024-04-21T09:08:12.140230+0000","last_fullsized":"2024-04-21T09:08:12.140230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'5","reported_seq":170,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.286167+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:08:16.286167+0000","last_peered":"2024-04-21T09:08:16.286167+0000","last_clean":"2024-04-21T09:08:16.286167+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:08:16.286167+0000","last_undegraded":"2024-04-21T09:08:16.286167+0000","last_fullsized":"2024-04-21T09:08:16.286167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324692+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:08:12.324692+0000","last_peered":"2024-04-21T09:08:12.324692+0000","last_clean":"2024-04-21T09:08:12.324692+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:08:12.324692+0000","last_undegraded":"2024-04-21T09:08:12.324692+0000","last_fullsized":"2024-04-21T09:08:12.324692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033898+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:08:12.033898+0000","last_peered":"2024-04-21T09:08:12.033898+0000","last_clean":"2024-04-21T09:08:12.033898+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:08:12.033898+0000","last_undegraded":"2024-04-21T09:08:12.033898+0000","last_fullsized":"2024-04-21T09:08:12.033898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142080+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:08:12.142080+0000","last_peered":"2024-04-21T09:08:12.142080+0000","last_clean":"2024-04-21T09:08:12.142080+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:08:12.142080+0000","last_undegraded":"2024-04-21T09:08:12.142080+0000","last_fullsized":"2024-04-21T09:08:12.142080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150555+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:08:12.150555+0000","last_peered":"2024-04-21T09:08:12.150555+0000","last_clean":"2024-04-21T09:08:12.150555+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:08:12.150555+0000","last_undegraded":"2024-04-21T09:08:12.150555+0000","last_fullsized":"2024-04-21T09:08:12.150555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034523+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:08:12.034523+0000","last_peered":"2024-04-21T09:08:12.034523+0000","last_clean":"2024-04-21T09:08:12.034523+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:08:12.034523+0000","last_undegraded":"2024-04-21T09:08:12.034523+0000","last_fullsized":"2024-04-21T09:08:12.034523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763981+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:08:12.763981+0000","last_peered":"2024-04-21T09:08:12.763981+0000","last_clean":"2024-04-21T09:08:12.763981+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:08:12.763981+0000","last_undegraded":"2024-04-21T09:08:12.763981+0000","last_fullsized":"2024-04-21T09:08:12.763981+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142163+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:08:12.142163+0000","last_peered":"2024-04-21T09:08:12.142163+0000","last_clean":"2024-04-21T09:08:12.142163+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:08:12.142163+0000","last_undegraded":"2024-04-21T09:08:12.142163+0000","last_fullsized":"2024-04-21T09:08:12.142163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139908+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:08:12.139908+0000","last_peered":"2024-04-21T09:08:12.139908+0000","last_clean":"2024-04-21T09:08:12.139908+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:08:12.139908+0000","last_undegraded":"2024-04-21T09:08:12.139908+0000","last_fullsized":"2024-04-21T09:08:12.139908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034862+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:08:12.034862+0000","last_peered":"2024-04-21T09:08:12.034862+0000","last_clean":"2024-04-21T09:08:12.034862+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:08:12.034862+0000","last_undegraded":"2024-04-21T09:08:12.034862+0000","last_fullsized":"2024-04-21T09:08:12.034862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033827+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:08:12.033827+0000","last_peered":"2024-04-21T09:08:12.033827+0000","last_clean":"2024-04-21T09:08:12.033827+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:08:12.033827+0000","last_undegraded":"2024-04-21T09:08:12.033827+0000","last_fullsized":"2024-04-21T09:08:12.033827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325076+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:08:12.325076+0000","last_peered":"2024-04-21T09:08:12.325076+0000","last_clean":"2024-04-21T09:08:12.325076+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:08:12.325076+0000","last_undegraded":"2024-04-21T09:08:12.325076+0000","last_fullsized":"2024-04-21T09:08:12.325076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034855+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:08:12.034855+0000","last_peered":"2024-04-21T09:08:12.034855+0000","last_clean":"2024-04-21T09:08:12.034855+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:08:12.034855+0000","last_undegraded":"2024-04-21T09:08:12.034855+0000","last_fullsized":"2024-04-21T09:08:12.034855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033559+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:08:12.033559+0000","last_peered":"2024-04-21T09:08:12.033559+0000","last_clean":"2024-04-21T09:08:12.033559+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:08:12.033559+0000","last_undegraded":"2024-04-21T09:08:12.033559+0000","last_fullsized":"2024-04-21T09:08:12.033559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150754+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:08:12.150754+0000","last_peered":"2024-04-21T09:08:12.150754+0000","last_clean":"2024-04-21T09:08:12.150754+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:08:12.150754+0000","last_undegraded":"2024-04-21T09:08:12.150754+0000","last_fullsized":"2024-04-21T09:08:12.150754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.285785+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:08:16.285785+0000","last_peered":"2024-04-21T09:08:16.285785+0000","last_clean":"2024-04-21T09:08:16.285785+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:08:16.285785+0000","last_undegraded":"2024-04-21T09:08:16.285785+0000","last_fullsized":"2024-04-21T09:08:16.285785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325490+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:08:12.325490+0000","last_peered":"2024-04-21T09:08:12.325490+0000","last_clean":"2024-04-21T09:08:12.325490+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:08:12.325490+0000","last_undegraded":"2024-04-21T09:08:12.325490+0000","last_fullsized":"2024-04-21T09:08:12.325490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142918+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:08:12.142918+0000","last_peered":"2024-04-21T09:08:12.142918+0000","last_clean":"2024-04-21T09:08:12.142918+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:08:12.142918+0000","last_undegraded":"2024-04-21T09:08:12.142918+0000","last_fullsized":"2024-04-21T09:08:12.142918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280892+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:08:12.280892+0000","last_peered":"2024-04-21T09:08:12.280892+0000","last_clean":"2024-04-21T09:08:12.280892+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:08:12.280892+0000","last_undegraded":"2024-04-21T09:08:12.280892+0000","last_fullsized":"2024-04-21T09:08:12.280892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"96'148","reported_seq":413,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:14.175860+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:08:14.175860+0000","last_peered":"2024-04-21T09:08:14.175860+0000","last_clean":"2024-04-21T09:08:14.175860+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:08:14.175860+0000","last_undegraded":"2024-04-21T09:08:14.175860+0000","last_fullsized":"2024-04-21T09:08:14.175860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150742+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:08:12.150742+0000","last_peered":"2024-04-21T09:08:12.150742+0000","last_clean":"2024-04-21T09:08:12.150742+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:08:12.150742+0000","last_undegraded":"2024-04-21T09:08:12.150742+0000","last_fullsized":"2024-04-21T09:08:12.150742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033204+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:08:12.033204+0000","last_peered":"2024-04-21T09:08:12.033204+0000","last_clean":"2024-04-21T09:08:12.033204+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:08:12.033204+0000","last_undegraded":"2024-04-21T09:08:12.033204+0000","last_fullsized":"2024-04-21T09:08:12.033204+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324532+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:08:12.324532+0000","last_peered":"2024-04-21T09:08:12.324532+0000","last_clean":"2024-04-21T09:08:12.324532+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:08:12.324532+0000","last_undegraded":"2024-04-21T09:08:12.324532+0000","last_fullsized":"2024-04-21T09:08:12.324532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763722+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:08:12.763722+0000","last_peered":"2024-04-21T09:08:12.763722+0000","last_clean":"2024-04-21T09:08:12.763722+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:08:12.763722+0000","last_undegraded":"2024-04-21T09:08:12.763722+0000","last_fullsized":"2024-04-21T09:08:12.763722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142352+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:08:12.142352+0000","last_peered":"2024-04-21T09:08:12.142352+0000","last_clean":"2024-04-21T09:08:12.142352+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:08:12.142352+0000","last_undegraded":"2024-04-21T09:08:12.142352+0000","last_fullsized":"2024-04-21T09:08:12.142352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033592+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:08:12.033592+0000","last_peered":"2024-04-21T09:08:12.033592+0000","last_clean":"2024-04-21T09:08:12.033592+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:08:12.033592+0000","last_undegraded":"2024-04-21T09:08:12.033592+0000","last_fullsized":"2024-04-21T09:08:12.033592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142660+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:08:12.142660+0000","last_peered":"2024-04-21T09:08:12.142660+0000","last_clean":"2024-04-21T09:08:12.142660+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:08:12.142660+0000","last_undegraded":"2024-04-21T09:08:12.142660+0000","last_fullsized":"2024-04-21T09:08:12.142660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'70","reported_seq":159,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.998429+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:08:17.998429+0000","last_peered":"2024-04-21T09:08:17.998429+0000","last_clean":"2024-04-21T09:08:17.998429+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:08:17.998429+0000","last_undegraded":"2024-04-21T09:08:17.998429+0000","last_fullsized":"2024-04-21T09:08:17.998429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":125,"num_write_kb":1157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143688+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:08:12.143688+0000","last_peered":"2024-04-21T09:08:12.143688+0000","last_clean":"2024-04-21T09:08:12.143688+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:08:12.143688+0000","last_undegraded":"2024-04-21T09:08:12.143688+0000","last_fullsized":"2024-04-21T09:08:12.143688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034657+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:08:12.034657+0000","last_peered":"2024-04-21T09:08:12.034657+0000","last_clean":"2024-04-21T09:08:12.034657+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:08:12.034657+0000","last_undegraded":"2024-04-21T09:08:12.034657+0000","last_fullsized":"2024-04-21T09:08:12.034657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281282+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:08:12.281282+0000","last_peered":"2024-04-21T09:08:12.281282+0000","last_clean":"2024-04-21T09:08:12.281282+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:08:12.281282+0000","last_undegraded":"2024-04-21T09:08:12.281282+0000","last_fullsized":"2024-04-21T09:08:12.281282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633978+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:08:13.633978+0000","last_peered":"2024-04-21T09:08:13.633978+0000","last_clean":"2024-04-21T09:08:13.633978+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:08:13.633978+0000","last_undegraded":"2024-04-21T09:08:13.633978+0000","last_fullsized":"2024-04-21T09:08:13.633978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281130+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:08:12.281130+0000","last_peered":"2024-04-21T09:08:12.281130+0000","last_clean":"2024-04-21T09:08:12.281130+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:08:12.281130+0000","last_undegraded":"2024-04-21T09:08:12.281130+0000","last_fullsized":"2024-04-21T09:08:12.281130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323960+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:08:12.323960+0000","last_peered":"2024-04-21T09:08:12.323960+0000","last_clean":"2024-04-21T09:08:12.323960+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:08:12.323960+0000","last_undegraded":"2024-04-21T09:08:12.323960+0000","last_fullsized":"2024-04-21T09:08:12.323960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.141989+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:08:12.141989+0000","last_peered":"2024-04-21T09:08:12.141989+0000","last_clean":"2024-04-21T09:08:12.141989+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:08:12.141989+0000","last_undegraded":"2024-04-21T09:08:12.141989+0000","last_fullsized":"2024-04-21T09:08:12.141989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140305+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:08:12.140305+0000","last_peered":"2024-04-21T09:08:12.140305+0000","last_clean":"2024-04-21T09:08:12.140305+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:08:12.140305+0000","last_undegraded":"2024-04-21T09:08:12.140305+0000","last_fullsized":"2024-04-21T09:08:12.140305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143035+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:08:12.143035+0000","last_peered":"2024-04-21T09:08:12.143035+0000","last_clean":"2024-04-21T09:08:12.143035+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:08:12.143035+0000","last_undegraded":"2024-04-21T09:08:12.143035+0000","last_fullsized":"2024-04-21T09:08:12.143035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763571+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:08:12.763571+0000","last_peered":"2024-04-21T09:08:12.763571+0000","last_clean":"2024-04-21T09:08:12.763571+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:08:12.763571+0000","last_undegraded":"2024-04-21T09:08:12.763571+0000","last_fullsized":"2024-04-21T09:08:12.763571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281011+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:08:12.281011+0000","last_peered":"2024-04-21T09:08:12.281011+0000","last_clean":"2024-04-21T09:08:12.281011+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:08:12.281011+0000","last_undegraded":"2024-04-21T09:08:12.281011+0000","last_fullsized":"2024-04-21T09:08:12.281011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034866+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:08:12.034866+0000","last_peered":"2024-04-21T09:08:12.034866+0000","last_clean":"2024-04-21T09:08:12.034866+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:08:12.034866+0000","last_undegraded":"2024-04-21T09:08:12.034866+0000","last_fullsized":"2024-04-21T09:08:12.034866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143614+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:08:12.143614+0000","last_peered":"2024-04-21T09:08:12.143614+0000","last_clean":"2024-04-21T09:08:12.143614+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:08:12.143614+0000","last_undegraded":"2024-04-21T09:08:12.143614+0000","last_fullsized":"2024-04-21T09:08:12.143614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763696+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:08:12.763696+0000","last_peered":"2024-04-21T09:08:12.763696+0000","last_clean":"2024-04-21T09:08:12.763696+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:08:12.763696+0000","last_undegraded":"2024-04-21T09:08:12.763696+0000","last_fullsized":"2024-04-21T09:08:12.763696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325194+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:08:12.325194+0000","last_peered":"2024-04-21T09:08:12.325194+0000","last_clean":"2024-04-21T09:08:12.325194+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:08:12.325194+0000","last_undegraded":"2024-04-21T09:08:12.325194+0000","last_fullsized":"2024-04-21T09:08:12.325194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324153+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:08:12.324153+0000","last_peered":"2024-04-21T09:08:12.324153+0000","last_clean":"2024-04-21T09:08:12.324153+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:08:12.324153+0000","last_undegraded":"2024-04-21T09:08:12.324153+0000","last_fullsized":"2024-04-21T09:08:12.324153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324834+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:08:12.324834+0000","last_peered":"2024-04-21T09:08:12.324834+0000","last_clean":"2024-04-21T09:08:12.324834+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:08:12.324834+0000","last_undegraded":"2024-04-21T09:08:12.324834+0000","last_fullsized":"2024-04-21T09:08:12.324834+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325898+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:08:12.325898+0000","last_peered":"2024-04-21T09:08:12.325898+0000","last_clean":"2024-04-21T09:08:12.325898+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:08:12.325898+0000","last_undegraded":"2024-04-21T09:08:12.325898+0000","last_fullsized":"2024-04-21T09:08:12.325898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633710+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:08:13.633710+0000","last_peered":"2024-04-21T09:08:13.633710+0000","last_clean":"2024-04-21T09:08:13.633710+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:08:13.633710+0000","last_undegraded":"2024-04-21T09:08:13.633710+0000","last_fullsized":"2024-04-21T09:08:13.633710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"96'57","reported_seq":111,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.993708+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:08:17.993708+0000","last_peered":"2024-04-21T09:08:17.993708+0000","last_clean":"2024-04-21T09:08:17.993708+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:08:17.993708+0000","last_undegraded":"2024-04-21T09:08:17.993708+0000","last_fullsized":"2024-04-21T09:08:17.993708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324272+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:08:12.324272+0000","last_peered":"2024-04-21T09:08:12.324272+0000","last_clean":"2024-04-21T09:08:12.324272+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:08:12.324272+0000","last_undegraded":"2024-04-21T09:08:12.324272+0000","last_fullsized":"2024-04-21T09:08:12.324272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150326+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:08:12.150326+0000","last_peered":"2024-04-21T09:08:12.150326+0000","last_clean":"2024-04-21T09:08:12.150326+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:08:12.150326+0000","last_undegraded":"2024-04-21T09:08:12.150326+0000","last_fullsized":"2024-04-21T09:08:12.150326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033814+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:08:12.033814+0000","last_peered":"2024-04-21T09:08:12.033814+0000","last_clean":"2024-04-21T09:08:12.033814+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:08:12.033814+0000","last_undegraded":"2024-04-21T09:08:12.033814+0000","last_fullsized":"2024-04-21T09:08:12.033814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325958+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:08:12.325958+0000","last_peered":"2024-04-21T09:08:12.325958+0000","last_clean":"2024-04-21T09:08:12.325958+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:08:12.325958+0000","last_undegraded":"2024-04-21T09:08:12.325958+0000","last_fullsized":"2024-04-21T09:08:12.325958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763645+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:08:12.763645+0000","last_peered":"2024-04-21T09:08:12.763645+0000","last_clean":"2024-04-21T09:08:12.763645+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:08:12.763645+0000","last_undegraded":"2024-04-21T09:08:12.763645+0000","last_fullsized":"2024-04-21T09:08:12.763645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034801+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:08:12.034801+0000","last_peered":"2024-04-21T09:08:12.034801+0000","last_clean":"2024-04-21T09:08:12.034801+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:08:12.034801+0000","last_undegraded":"2024-04-21T09:08:12.034801+0000","last_fullsized":"2024-04-21T09:08:12.034801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034715+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:08:12.034715+0000","last_peered":"2024-04-21T09:08:12.034715+0000","last_clean":"2024-04-21T09:08:12.034715+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:08:12.034715+0000","last_undegraded":"2024-04-21T09:08:12.034715+0000","last_fullsized":"2024-04-21T09:08:12.034715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633881+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:08:13.633881+0000","last_peered":"2024-04-21T09:08:13.633881+0000","last_clean":"2024-04-21T09:08:13.633881+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:08:13.633881+0000","last_undegraded":"2024-04-21T09:08:13.633881+0000","last_fullsized":"2024-04-21T09:08:13.633881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033916+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:08:12.033916+0000","last_peered":"2024-04-21T09:08:12.033916+0000","last_clean":"2024-04-21T09:08:12.033916+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:08:12.033916+0000","last_undegraded":"2024-04-21T09:08:12.033916+0000","last_fullsized":"2024-04-21T09:08:12.033916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763506+0000","last_change":"2024-04-21T09:07:40.649825+0000","last_active":"2024-04-21T09:08:12.763506+0000","last_peered":"2024-04-21T09:08:12.763506+0000","last_clean":"2024-04-21T09:08:12.763506+0000","last_became_active":"2024-04-21T09:07:40.649264+0000","last_became_peered":"2024-04-21T09:07:40.649264+0000","last_unstale":"2024-04-21T09:08:12.763506+0000","last_undegraded":"2024-04-21T09:08:12.763506+0000","last_fullsized":"2024-04-21T09:08:12.763506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:38.142948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035249+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:08:12.035249+0000","last_peered":"2024-04-21T09:08:12.035249+0000","last_clean":"2024-04-21T09:08:12.035249+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:08:12.035249+0000","last_undegraded":"2024-04-21T09:08:12.035249+0000","last_fullsized":"2024-04-21T09:08:12.035249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143896+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:08:12.143896+0000","last_peered":"2024-04-21T09:08:12.143896+0000","last_clean":"2024-04-21T09:08:12.143896+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:08:12.143896+0000","last_undegraded":"2024-04-21T09:08:12.143896+0000","last_fullsized":"2024-04-21T09:08:12.143896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150636+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:08:12.150636+0000","last_peered":"2024-04-21T09:08:12.150636+0000","last_clean":"2024-04-21T09:08:12.150636+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:08:12.150636+0000","last_undegraded":"2024-04-21T09:08:12.150636+0000","last_fullsized":"2024-04-21T09:08:12.150636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143204+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:08:12.143204+0000","last_peered":"2024-04-21T09:08:12.143204+0000","last_clean":"2024-04-21T09:08:12.143204+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:08:12.143204+0000","last_undegraded":"2024-04-21T09:08:12.143204+0000","last_fullsized":"2024-04-21T09:08:12.143204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324412+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:08:12.324412+0000","last_peered":"2024-04-21T09:08:12.324412+0000","last_clean":"2024-04-21T09:08:12.324412+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:08:12.324412+0000","last_undegraded":"2024-04-21T09:08:12.324412+0000","last_fullsized":"2024-04-21T09:08:12.324412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033628+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:08:12.033628+0000","last_peered":"2024-04-21T09:08:12.033628+0000","last_clean":"2024-04-21T09:08:12.033628+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:08:12.033628+0000","last_undegraded":"2024-04-21T09:08:12.033628+0000","last_fullsized":"2024-04-21T09:08:12.033628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763507+0000","last_change":"2024-04-21T09:07:40.650222+0000","last_active":"2024-04-21T09:08:12.763507+0000","last_peered":"2024-04-21T09:08:12.763507+0000","last_clean":"2024-04-21T09:08:12.763507+0000","last_became_active":"2024-04-21T09:07:40.649595+0000","last_became_peered":"2024-04-21T09:07:40.649595+0000","last_unstale":"2024-04-21T09:08:12.763507+0000","last_undegraded":"2024-04-21T09:08:12.763507+0000","last_fullsized":"2024-04-21T09:08:12.763507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:25.006017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107013,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793576,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":976,"num_read_kb":645,"num_write":351,"num_write_kb":2090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":638976,"data_stored":2380728,"data_compressed":107238,"data_compressed_allocated":147456,"data_compressed_original":2125824,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":541927,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32368,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712784,"statfs":{"total":95995035648,"available":95961890816,"internally_reserved":0,"allocated":1220608,"data_stored":1328900,"data_compressed":75912,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":939453,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":935255,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1138669,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":537317,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":605237,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":538705,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":604801,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1062327,"data_compressed":57756,"data_compressed_allocated":77824,"data_compressed_original":606208,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888143,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":543679,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":1265158,"data_compressed":74281,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68010,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67574,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34115,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330099,"data_compressed":14840,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":397937,"data_compressed":16471,"data_compressed_allocated":24576,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:08:22.731 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:08:22.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.867+0000 7f72583a4640 1 -- 172.21.15.66:0/1552982331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250103af0 msgr2=0x7f725010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.867+0000 7f72583a4640 1 --2- 172.21.15.66:0/1552982331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250103af0 0x7f725010bfd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7240009900 tx=0x7f724002bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.867+0000 7f72583a4640 1 -- 172.21.15.66:0/1552982331 shutdown_connections 2024-04-21T09:08:22.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 --2- 172.21.15.66:0/1552982331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f725010c5a0 0x7f725010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 --2- 172.21.15.66:0/1552982331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250103af0 0x7f725010bfd0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 --2- 172.21.15.66:0/1552982331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72501031d0 0x7f72501035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 -- 172.21.15.66:0/1552982331 >> 172.21.15.66:0/1552982331 conn(0x7f72500771d0 msgr2=0x7f72500775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 -- 172.21.15.66:0/1552982331 shutdown_connections 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 -- 172.21.15.66:0/1552982331 wait complete. 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 Processor -- start 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 -- start start 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72501031d0 0x7f725019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7250103af0 0x7f725019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f725010c5a0 0x7f7250198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f72501121e0 con 0x7f725010c5a0 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7250112060 con 0x7f72501031d0 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f72583a4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7250112360 con 0x7f7250103af0 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f7256941640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f725010c5a0 0x7f7250198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.868+0000 7f7256941640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f725010c5a0 0x7f7250198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33242/0 (socket says 172.21.15.66:33242) 2024-04-21T09:08:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256941640 1 -- 172.21.15.66:0/2147428302 learned_addr learned my addr 172.21.15.66:0/2147428302 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256140640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72501031d0 0x7f725019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f725593f640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7250103af0 0x7f725019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256941640 1 -- 172.21.15.66:0/2147428302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7250103af0 msgr2=0x7f725019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256941640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7250103af0 0x7f725019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256941640 1 -- 172.21.15.66:0/2147428302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72501031d0 msgr2=0x7f725019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256941640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72501031d0 0x7f725019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256941640 1 -- 172.21.15.66:0/2147428302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7250198da0 con 0x7f725010c5a0 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256140640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72501031d0 0x7f725019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f7256941640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f725010c5a0 0x7f7250198620 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f724800cab0 tx=0x7f724800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f72477fe640 1 -- 172.21.15.66:0/2147428302 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724800ae20 con 0x7f725010c5a0 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f72477fe640 1 -- 172.21.15.66:0/2147428302 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724800f900 con 0x7f725010c5a0 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7250199090 con 0x7f725010c5a0 2024-04-21T09:08:22.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f72477fe640 1 -- 172.21.15.66:0/2147428302 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248018a50 con 0x7f725010c5a0 2024-04-21T09:08:22.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.869+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72501a5330 con 0x7f725010c5a0 2024-04-21T09:08:22.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.870+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7250069230 con 0x7f725010c5a0 2024-04-21T09:08:22.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.871+0000 7f72477fe640 1 -- 172.21.15.66:0/2147428302 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f724800f450 con 0x7f725010c5a0 2024-04-21T09:08:22.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.872+0000 7f72477fe640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f723c0a3db0 0x7f723c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:22.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.872+0000 7f7256140640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f723c0a3db0 0x7f723c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:22.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.872+0000 7f72477fe640 1 -- 172.21.15.66:0/2147428302 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f72480d1780 con 0x7f725010c5a0 2024-04-21T09:08:22.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.872+0000 7f7256140640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f723c0a3db0 0x7f723c0a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f724c0058c0 tx=0x7f724c005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:22.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:22.876+0000 7f72477fe640 1 -- 172.21.15.66:0/2147428302 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7248014030 con 0x7f725010c5a0 2024-04-21T09:08:23.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.061+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7250062390 con 0x7f723c0a3db0 2024-04-21T09:08:23.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.070+0000 7f72477fe640 1 -- 172.21.15.66:0/2147428302 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339717 (secure 0 0 0) 0x7f7250062390 con 0x7f723c0a3db0 2024-04-21T09:08:23.074 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:23.076 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:08:23.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f723c0a3db0 msgr2=0x7f723c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f723c0a3db0 0x7f723c0a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f724c0058c0 tx=0x7f724c005850 comp rx=0 tx=0).stop 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f725010c5a0 msgr2=0x7f7250198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f725010c5a0 0x7f7250198620 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f724800cab0 tx=0x7f724800cf70 comp rx=0 tx=0).stop 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 shutdown_connections 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f723c0a3db0 0x7f723c0a6270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f725010c5a0 0x7f7250198620 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7250103af0 0x7f725019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 --2- 172.21.15.66:0/2147428302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72501031d0 0x7f725019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 >> 172.21.15.66:0/2147428302 conn(0x7f72500771d0 msgr2=0x7f72500ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 shutdown_connections 2024-04-21T09:08:23.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.074+0000 7f72583a4640 1 -- 172.21.15.66:0/2147428302 wait complete. 2024-04-21T09:08:23.085 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T09:08:22.049328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384178,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1213,"num_read_kb":975,"num_write":554,"num_write_kb":5030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":919,"ondisk_log_size":919,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357992,"kb_used_data":12968,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583832,"statfs":{"total":1151940427776,"available":1151573843968,"internally_reserved":0,"allocated":13279232,"data_stored":10041428,"data_compressed":577260,"data_compressed_allocated":786432,"data_compressed_original":4423680,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":246,"num_read_kb":331,"num_write":84,"num_write_kb":652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991412"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764225+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:08:12.764225+0000","last_peered":"2024-04-21T09:08:12.764225+0000","last_clean":"2024-04-21T09:08:12.764225+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:08:12.764225+0000","last_undegraded":"2024-04-21T09:08:12.764225+0000","last_fullsized":"2024-04-21T09:08:12.764225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.634069+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:08:13.634069+0000","last_peered":"2024-04-21T09:08:13.634069+0000","last_clean":"2024-04-21T09:08:13.634069+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:08:13.634069+0000","last_undegraded":"2024-04-21T09:08:13.634069+0000","last_fullsized":"2024-04-21T09:08:13.634069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633957+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:08:13.633957+0000","last_peered":"2024-04-21T09:08:13.633957+0000","last_clean":"2024-04-21T09:08:13.633957+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:08:13.633957+0000","last_undegraded":"2024-04-21T09:08:13.633957+0000","last_fullsized":"2024-04-21T09:08:13.633957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281097+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:08:12.281097+0000","last_peered":"2024-04-21T09:08:12.281097+0000","last_clean":"2024-04-21T09:08:12.281097+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:08:12.281097+0000","last_undegraded":"2024-04-21T09:08:12.281097+0000","last_fullsized":"2024-04-21T09:08:12.281097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142249+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:08:12.142249+0000","last_peered":"2024-04-21T09:08:12.142249+0000","last_clean":"2024-04-21T09:08:12.142249+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:08:12.142249+0000","last_undegraded":"2024-04-21T09:08:12.142249+0000","last_fullsized":"2024-04-21T09:08:12.142249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034661+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:08:12.034661+0000","last_peered":"2024-04-21T09:08:12.034661+0000","last_clean":"2024-04-21T09:08:12.034661+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:08:12.034661+0000","last_undegraded":"2024-04-21T09:08:12.034661+0000","last_fullsized":"2024-04-21T09:08:12.034661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324873+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:08:12.324873+0000","last_peered":"2024-04-21T09:08:12.324873+0000","last_clean":"2024-04-21T09:08:12.324873+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:08:12.324873+0000","last_undegraded":"2024-04-21T09:08:12.324873+0000","last_fullsized":"2024-04-21T09:08:12.324873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142696+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:08:12.142696+0000","last_peered":"2024-04-21T09:08:12.142696+0000","last_clean":"2024-04-21T09:08:12.142696+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:08:12.142696+0000","last_undegraded":"2024-04-21T09:08:12.142696+0000","last_fullsized":"2024-04-21T09:08:12.142696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325536+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:08:12.325536+0000","last_peered":"2024-04-21T09:08:12.325536+0000","last_clean":"2024-04-21T09:08:12.325536+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:08:12.325536+0000","last_undegraded":"2024-04-21T09:08:12.325536+0000","last_fullsized":"2024-04-21T09:08:12.325536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035076+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:08:12.035076+0000","last_peered":"2024-04-21T09:08:12.035076+0000","last_clean":"2024-04-21T09:08:12.035076+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:08:12.035076+0000","last_undegraded":"2024-04-21T09:08:12.035076+0000","last_fullsized":"2024-04-21T09:08:12.035076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325727+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:08:12.325727+0000","last_peered":"2024-04-21T09:08:12.325727+0000","last_clean":"2024-04-21T09:08:12.325727+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:08:12.325727+0000","last_undegraded":"2024-04-21T09:08:12.325727+0000","last_fullsized":"2024-04-21T09:08:12.325727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281072+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:08:12.281072+0000","last_peered":"2024-04-21T09:08:12.281072+0000","last_clean":"2024-04-21T09:08:12.281072+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:08:12.281072+0000","last_undegraded":"2024-04-21T09:08:12.281072+0000","last_fullsized":"2024-04-21T09:08:12.281072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033565+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:08:12.033565+0000","last_peered":"2024-04-21T09:08:12.033565+0000","last_clean":"2024-04-21T09:08:12.033565+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:08:12.033565+0000","last_undegraded":"2024-04-21T09:08:12.033565+0000","last_fullsized":"2024-04-21T09:08:12.033565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633566+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:08:13.633566+0000","last_peered":"2024-04-21T09:08:13.633566+0000","last_clean":"2024-04-21T09:08:13.633566+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:08:13.633566+0000","last_undegraded":"2024-04-21T09:08:13.633566+0000","last_fullsized":"2024-04-21T09:08:13.633566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633612+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:08:13.633612+0000","last_peered":"2024-04-21T09:08:13.633612+0000","last_clean":"2024-04-21T09:08:13.633612+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:08:13.633612+0000","last_undegraded":"2024-04-21T09:08:13.633612+0000","last_fullsized":"2024-04-21T09:08:13.633612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323842+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:08:12.323842+0000","last_peered":"2024-04-21T09:08:12.323842+0000","last_clean":"2024-04-21T09:08:12.323842+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:08:12.323842+0000","last_undegraded":"2024-04-21T09:08:12.323842+0000","last_fullsized":"2024-04-21T09:08:12.323842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764134+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:08:12.764134+0000","last_peered":"2024-04-21T09:08:12.764134+0000","last_clean":"2024-04-21T09:08:12.764134+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:08:12.764134+0000","last_undegraded":"2024-04-21T09:08:12.764134+0000","last_fullsized":"2024-04-21T09:08:12.764134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140490+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:08:12.140490+0000","last_peered":"2024-04-21T09:08:12.140490+0000","last_clean":"2024-04-21T09:08:12.140490+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:08:12.140490+0000","last_undegraded":"2024-04-21T09:08:12.140490+0000","last_fullsized":"2024-04-21T09:08:12.140490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035166+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:08:12.035166+0000","last_peered":"2024-04-21T09:08:12.035166+0000","last_clean":"2024-04-21T09:08:12.035166+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:08:12.035166+0000","last_undegraded":"2024-04-21T09:08:12.035166+0000","last_fullsized":"2024-04-21T09:08:12.035166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034868+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:08:12.034868+0000","last_peered":"2024-04-21T09:08:12.034868+0000","last_clean":"2024-04-21T09:08:12.034868+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:08:12.034868+0000","last_undegraded":"2024-04-21T09:08:12.034868+0000","last_fullsized":"2024-04-21T09:08:12.034868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140333+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:08:12.140333+0000","last_peered":"2024-04-21T09:08:12.140333+0000","last_clean":"2024-04-21T09:08:12.140333+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:08:12.140333+0000","last_undegraded":"2024-04-21T09:08:12.140333+0000","last_fullsized":"2024-04-21T09:08:12.140333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143771+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:08:12.143771+0000","last_peered":"2024-04-21T09:08:12.143771+0000","last_clean":"2024-04-21T09:08:12.143771+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:08:12.143771+0000","last_undegraded":"2024-04-21T09:08:12.143771+0000","last_fullsized":"2024-04-21T09:08:12.143771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280764+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:08:12.280764+0000","last_peered":"2024-04-21T09:08:12.280764+0000","last_clean":"2024-04-21T09:08:12.280764+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:08:12.280764+0000","last_undegraded":"2024-04-21T09:08:12.280764+0000","last_fullsized":"2024-04-21T09:08:12.280764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.144097+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:08:12.144097+0000","last_peered":"2024-04-21T09:08:12.144097+0000","last_clean":"2024-04-21T09:08:12.144097+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:08:12.144097+0000","last_undegraded":"2024-04-21T09:08:12.144097+0000","last_fullsized":"2024-04-21T09:08:12.144097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633901+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:08:13.633901+0000","last_peered":"2024-04-21T09:08:13.633901+0000","last_clean":"2024-04-21T09:08:13.633901+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:08:13.633901+0000","last_undegraded":"2024-04-21T09:08:13.633901+0000","last_fullsized":"2024-04-21T09:08:13.633901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280878+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:08:12.280878+0000","last_peered":"2024-04-21T09:08:12.280878+0000","last_clean":"2024-04-21T09:08:12.280878+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:08:12.280878+0000","last_undegraded":"2024-04-21T09:08:12.280878+0000","last_fullsized":"2024-04-21T09:08:12.280878+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142503+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:08:12.142503+0000","last_peered":"2024-04-21T09:08:12.142503+0000","last_clean":"2024-04-21T09:08:12.142503+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:08:12.142503+0000","last_undegraded":"2024-04-21T09:08:12.142503+0000","last_fullsized":"2024-04-21T09:08:12.142503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325104+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:08:12.325104+0000","last_peered":"2024-04-21T09:08:12.325104+0000","last_clean":"2024-04-21T09:08:12.325104+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:08:12.325104+0000","last_undegraded":"2024-04-21T09:08:12.325104+0000","last_fullsized":"2024-04-21T09:08:12.325104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280773+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:08:12.280773+0000","last_peered":"2024-04-21T09:08:12.280773+0000","last_clean":"2024-04-21T09:08:12.280773+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:08:12.280773+0000","last_undegraded":"2024-04-21T09:08:12.280773+0000","last_fullsized":"2024-04-21T09:08:12.280773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142157+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:08:12.142157+0000","last_peered":"2024-04-21T09:08:12.142157+0000","last_clean":"2024-04-21T09:08:12.142157+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:08:12.142157+0000","last_undegraded":"2024-04-21T09:08:12.142157+0000","last_fullsized":"2024-04-21T09:08:12.142157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150927+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:08:12.150927+0000","last_peered":"2024-04-21T09:08:12.150927+0000","last_clean":"2024-04-21T09:08:12.150927+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:08:12.150927+0000","last_undegraded":"2024-04-21T09:08:12.150927+0000","last_fullsized":"2024-04-21T09:08:12.150927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034947+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:08:12.034947+0000","last_peered":"2024-04-21T09:08:12.034947+0000","last_clean":"2024-04-21T09:08:12.034947+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:08:12.034947+0000","last_undegraded":"2024-04-21T09:08:12.034947+0000","last_fullsized":"2024-04-21T09:08:12.034947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'6","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.321068+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:08:17.321068+0000","last_peered":"2024-04-21T09:08:17.321068+0000","last_clean":"2024-04-21T09:08:17.321068+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:08:17.321068+0000","last_undegraded":"2024-04-21T09:08:17.321068+0000","last_fullsized":"2024-04-21T09:08:17.321068+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142233+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:08:12.142233+0000","last_peered":"2024-04-21T09:08:12.142233+0000","last_clean":"2024-04-21T09:08:12.142233+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:08:12.142233+0000","last_undegraded":"2024-04-21T09:08:12.142233+0000","last_fullsized":"2024-04-21T09:08:12.142233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035266+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:08:12.035266+0000","last_peered":"2024-04-21T09:08:12.035266+0000","last_clean":"2024-04-21T09:08:12.035266+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:08:12.035266+0000","last_undegraded":"2024-04-21T09:08:12.035266+0000","last_fullsized":"2024-04-21T09:08:12.035266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034693+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:08:12.034693+0000","last_peered":"2024-04-21T09:08:12.034693+0000","last_clean":"2024-04-21T09:08:12.034693+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:08:12.034693+0000","last_undegraded":"2024-04-21T09:08:12.034693+0000","last_fullsized":"2024-04-21T09:08:12.034693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150778+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:08:12.150778+0000","last_peered":"2024-04-21T09:08:12.150778+0000","last_clean":"2024-04-21T09:08:12.150778+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:08:12.150778+0000","last_undegraded":"2024-04-21T09:08:12.150778+0000","last_fullsized":"2024-04-21T09:08:12.150778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763967+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:08:12.763967+0000","last_peered":"2024-04-21T09:08:12.763967+0000","last_clean":"2024-04-21T09:08:12.763967+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:08:12.763967+0000","last_undegraded":"2024-04-21T09:08:12.763967+0000","last_fullsized":"2024-04-21T09:08:12.763967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033440+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:08:12.033440+0000","last_peered":"2024-04-21T09:08:12.033440+0000","last_clean":"2024-04-21T09:08:12.033440+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:08:12.033440+0000","last_undegraded":"2024-04-21T09:08:12.033440+0000","last_fullsized":"2024-04-21T09:08:12.033440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142822+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:08:12.142822+0000","last_peered":"2024-04-21T09:08:12.142822+0000","last_clean":"2024-04-21T09:08:12.142822+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:08:12.142822+0000","last_undegraded":"2024-04-21T09:08:12.142822+0000","last_fullsized":"2024-04-21T09:08:12.142822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633942+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:08:13.633942+0000","last_peered":"2024-04-21T09:08:13.633942+0000","last_clean":"2024-04-21T09:08:13.633942+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:08:13.633942+0000","last_undegraded":"2024-04-21T09:08:13.633942+0000","last_fullsized":"2024-04-21T09:08:13.633942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150624+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:08:12.150624+0000","last_peered":"2024-04-21T09:08:12.150624+0000","last_clean":"2024-04-21T09:08:12.150624+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:08:12.150624+0000","last_undegraded":"2024-04-21T09:08:12.150624+0000","last_fullsized":"2024-04-21T09:08:12.150624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139637+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:08:12.139637+0000","last_peered":"2024-04-21T09:08:12.139637+0000","last_clean":"2024-04-21T09:08:12.139637+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:08:12.139637+0000","last_undegraded":"2024-04-21T09:08:12.139637+0000","last_fullsized":"2024-04-21T09:08:12.139637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142906+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:08:12.142906+0000","last_peered":"2024-04-21T09:08:12.142906+0000","last_clean":"2024-04-21T09:08:12.142906+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:08:12.142906+0000","last_undegraded":"2024-04-21T09:08:12.142906+0000","last_fullsized":"2024-04-21T09:08:12.142906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":165,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.319293+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:08:16.319293+0000","last_peered":"2024-04-21T09:08:16.319293+0000","last_clean":"2024-04-21T09:08:16.319293+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:08:16.319293+0000","last_undegraded":"2024-04-21T09:08:16.319293+0000","last_fullsized":"2024-04-21T09:08:16.319293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325164+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:08:12.325164+0000","last_peered":"2024-04-21T09:08:12.325164+0000","last_clean":"2024-04-21T09:08:12.325164+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:08:12.325164+0000","last_undegraded":"2024-04-21T09:08:12.325164+0000","last_fullsized":"2024-04-21T09:08:12.325164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139403+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:08:12.139403+0000","last_peered":"2024-04-21T09:08:12.139403+0000","last_clean":"2024-04-21T09:08:12.139403+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:08:12.139403+0000","last_undegraded":"2024-04-21T09:08:12.139403+0000","last_fullsized":"2024-04-21T09:08:12.139403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325630+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:08:12.325630+0000","last_peered":"2024-04-21T09:08:12.325630+0000","last_clean":"2024-04-21T09:08:12.325630+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:08:12.325630+0000","last_undegraded":"2024-04-21T09:08:12.325630+0000","last_fullsized":"2024-04-21T09:08:12.325630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.634168+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:08:13.634168+0000","last_peered":"2024-04-21T09:08:13.634168+0000","last_clean":"2024-04-21T09:08:13.634168+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:08:13.634168+0000","last_undegraded":"2024-04-21T09:08:13.634168+0000","last_fullsized":"2024-04-21T09:08:13.634168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150830+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:08:12.150830+0000","last_peered":"2024-04-21T09:08:12.150830+0000","last_clean":"2024-04-21T09:08:12.150830+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:08:12.150830+0000","last_undegraded":"2024-04-21T09:08:12.150830+0000","last_fullsized":"2024-04-21T09:08:12.150830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325080+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:08:12.325080+0000","last_peered":"2024-04-21T09:08:12.325080+0000","last_clean":"2024-04-21T09:08:12.325080+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:08:12.325080+0000","last_undegraded":"2024-04-21T09:08:12.325080+0000","last_fullsized":"2024-04-21T09:08:12.325080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139822+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:08:12.139822+0000","last_peered":"2024-04-21T09:08:12.139822+0000","last_clean":"2024-04-21T09:08:12.139822+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:08:12.139822+0000","last_undegraded":"2024-04-21T09:08:12.139822+0000","last_fullsized":"2024-04-21T09:08:12.139822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":337,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.320818+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:08:17.320818+0000","last_peered":"2024-04-21T09:08:17.320818+0000","last_clean":"2024-04-21T09:08:17.320818+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:08:17.320818+0000","last_undegraded":"2024-04-21T09:08:17.320818+0000","last_fullsized":"2024-04-21T09:08:17.320818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":262,"num_read_kb":198,"num_write":40,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142319+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:08:12.142319+0000","last_peered":"2024-04-21T09:08:12.142319+0000","last_clean":"2024-04-21T09:08:12.142319+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:08:12.142319+0000","last_undegraded":"2024-04-21T09:08:12.142319+0000","last_fullsized":"2024-04-21T09:08:12.142319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.151042+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:08:12.151042+0000","last_peered":"2024-04-21T09:08:12.151042+0000","last_clean":"2024-04-21T09:08:12.151042+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:08:12.151042+0000","last_undegraded":"2024-04-21T09:08:12.151042+0000","last_fullsized":"2024-04-21T09:08:12.151042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035051+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:08:12.035051+0000","last_peered":"2024-04-21T09:08:12.035051+0000","last_clean":"2024-04-21T09:08:12.035051+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:08:12.035051+0000","last_undegraded":"2024-04-21T09:08:12.035051+0000","last_fullsized":"2024-04-21T09:08:12.035051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033721+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:08:12.033721+0000","last_peered":"2024-04-21T09:08:12.033721+0000","last_clean":"2024-04-21T09:08:12.033721+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:08:12.033721+0000","last_undegraded":"2024-04-21T09:08:12.033721+0000","last_fullsized":"2024-04-21T09:08:12.033721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150756+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:08:12.150756+0000","last_peered":"2024-04-21T09:08:12.150756+0000","last_clean":"2024-04-21T09:08:12.150756+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:08:12.150756+0000","last_undegraded":"2024-04-21T09:08:12.150756+0000","last_fullsized":"2024-04-21T09:08:12.150756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325172+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:08:12.325172+0000","last_peered":"2024-04-21T09:08:12.325172+0000","last_clean":"2024-04-21T09:08:12.325172+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:08:12.325172+0000","last_undegraded":"2024-04-21T09:08:12.325172+0000","last_fullsized":"2024-04-21T09:08:12.325172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140230+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:08:12.140230+0000","last_peered":"2024-04-21T09:08:12.140230+0000","last_clean":"2024-04-21T09:08:12.140230+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:08:12.140230+0000","last_undegraded":"2024-04-21T09:08:12.140230+0000","last_fullsized":"2024-04-21T09:08:12.140230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'5","reported_seq":170,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.286167+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:08:16.286167+0000","last_peered":"2024-04-21T09:08:16.286167+0000","last_clean":"2024-04-21T09:08:16.286167+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:08:16.286167+0000","last_undegraded":"2024-04-21T09:08:16.286167+0000","last_fullsized":"2024-04-21T09:08:16.286167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324692+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:08:12.324692+0000","last_peered":"2024-04-21T09:08:12.324692+0000","last_clean":"2024-04-21T09:08:12.324692+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:08:12.324692+0000","last_undegraded":"2024-04-21T09:08:12.324692+0000","last_fullsized":"2024-04-21T09:08:12.324692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033898+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:08:12.033898+0000","last_peered":"2024-04-21T09:08:12.033898+0000","last_clean":"2024-04-21T09:08:12.033898+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:08:12.033898+0000","last_undegraded":"2024-04-21T09:08:12.033898+0000","last_fullsized":"2024-04-21T09:08:12.033898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142080+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:08:12.142080+0000","last_peered":"2024-04-21T09:08:12.142080+0000","last_clean":"2024-04-21T09:08:12.142080+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:08:12.142080+0000","last_undegraded":"2024-04-21T09:08:12.142080+0000","last_fullsized":"2024-04-21T09:08:12.142080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150555+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:08:12.150555+0000","last_peered":"2024-04-21T09:08:12.150555+0000","last_clean":"2024-04-21T09:08:12.150555+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:08:12.150555+0000","last_undegraded":"2024-04-21T09:08:12.150555+0000","last_fullsized":"2024-04-21T09:08:12.150555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034523+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:08:12.034523+0000","last_peered":"2024-04-21T09:08:12.034523+0000","last_clean":"2024-04-21T09:08:12.034523+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:08:12.034523+0000","last_undegraded":"2024-04-21T09:08:12.034523+0000","last_fullsized":"2024-04-21T09:08:12.034523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763981+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:08:12.763981+0000","last_peered":"2024-04-21T09:08:12.763981+0000","last_clean":"2024-04-21T09:08:12.763981+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:08:12.763981+0000","last_undegraded":"2024-04-21T09:08:12.763981+0000","last_fullsized":"2024-04-21T09:08:12.763981+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142163+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:08:12.142163+0000","last_peered":"2024-04-21T09:08:12.142163+0000","last_clean":"2024-04-21T09:08:12.142163+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:08:12.142163+0000","last_undegraded":"2024-04-21T09:08:12.142163+0000","last_fullsized":"2024-04-21T09:08:12.142163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139908+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:08:12.139908+0000","last_peered":"2024-04-21T09:08:12.139908+0000","last_clean":"2024-04-21T09:08:12.139908+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:08:12.139908+0000","last_undegraded":"2024-04-21T09:08:12.139908+0000","last_fullsized":"2024-04-21T09:08:12.139908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034862+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:08:12.034862+0000","last_peered":"2024-04-21T09:08:12.034862+0000","last_clean":"2024-04-21T09:08:12.034862+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:08:12.034862+0000","last_undegraded":"2024-04-21T09:08:12.034862+0000","last_fullsized":"2024-04-21T09:08:12.034862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033827+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:08:12.033827+0000","last_peered":"2024-04-21T09:08:12.033827+0000","last_clean":"2024-04-21T09:08:12.033827+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:08:12.033827+0000","last_undegraded":"2024-04-21T09:08:12.033827+0000","last_fullsized":"2024-04-21T09:08:12.033827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325076+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:08:12.325076+0000","last_peered":"2024-04-21T09:08:12.325076+0000","last_clean":"2024-04-21T09:08:12.325076+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:08:12.325076+0000","last_undegraded":"2024-04-21T09:08:12.325076+0000","last_fullsized":"2024-04-21T09:08:12.325076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034855+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:08:12.034855+0000","last_peered":"2024-04-21T09:08:12.034855+0000","last_clean":"2024-04-21T09:08:12.034855+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:08:12.034855+0000","last_undegraded":"2024-04-21T09:08:12.034855+0000","last_fullsized":"2024-04-21T09:08:12.034855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033559+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:08:12.033559+0000","last_peered":"2024-04-21T09:08:12.033559+0000","last_clean":"2024-04-21T09:08:12.033559+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:08:12.033559+0000","last_undegraded":"2024-04-21T09:08:12.033559+0000","last_fullsized":"2024-04-21T09:08:12.033559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150754+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:08:12.150754+0000","last_peered":"2024-04-21T09:08:12.150754+0000","last_clean":"2024-04-21T09:08:12.150754+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:08:12.150754+0000","last_undegraded":"2024-04-21T09:08:12.150754+0000","last_fullsized":"2024-04-21T09:08:12.150754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.285785+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:08:16.285785+0000","last_peered":"2024-04-21T09:08:16.285785+0000","last_clean":"2024-04-21T09:08:16.285785+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:08:16.285785+0000","last_undegraded":"2024-04-21T09:08:16.285785+0000","last_fullsized":"2024-04-21T09:08:16.285785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325490+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:08:12.325490+0000","last_peered":"2024-04-21T09:08:12.325490+0000","last_clean":"2024-04-21T09:08:12.325490+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:08:12.325490+0000","last_undegraded":"2024-04-21T09:08:12.325490+0000","last_fullsized":"2024-04-21T09:08:12.325490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142918+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:08:12.142918+0000","last_peered":"2024-04-21T09:08:12.142918+0000","last_clean":"2024-04-21T09:08:12.142918+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:08:12.142918+0000","last_undegraded":"2024-04-21T09:08:12.142918+0000","last_fullsized":"2024-04-21T09:08:12.142918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280892+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:08:12.280892+0000","last_peered":"2024-04-21T09:08:12.280892+0000","last_clean":"2024-04-21T09:08:12.280892+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:08:12.280892+0000","last_undegraded":"2024-04-21T09:08:12.280892+0000","last_fullsized":"2024-04-21T09:08:12.280892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"96'148","reported_seq":413,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:14.175860+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:08:14.175860+0000","last_peered":"2024-04-21T09:08:14.175860+0000","last_clean":"2024-04-21T09:08:14.175860+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:08:14.175860+0000","last_undegraded":"2024-04-21T09:08:14.175860+0000","last_fullsized":"2024-04-21T09:08:14.175860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150742+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:08:12.150742+0000","last_peered":"2024-04-21T09:08:12.150742+0000","last_clean":"2024-04-21T09:08:12.150742+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:08:12.150742+0000","last_undegraded":"2024-04-21T09:08:12.150742+0000","last_fullsized":"2024-04-21T09:08:12.150742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033204+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:08:12.033204+0000","last_peered":"2024-04-21T09:08:12.033204+0000","last_clean":"2024-04-21T09:08:12.033204+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:08:12.033204+0000","last_undegraded":"2024-04-21T09:08:12.033204+0000","last_fullsized":"2024-04-21T09:08:12.033204+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324532+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:08:12.324532+0000","last_peered":"2024-04-21T09:08:12.324532+0000","last_clean":"2024-04-21T09:08:12.324532+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:08:12.324532+0000","last_undegraded":"2024-04-21T09:08:12.324532+0000","last_fullsized":"2024-04-21T09:08:12.324532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763722+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:08:12.763722+0000","last_peered":"2024-04-21T09:08:12.763722+0000","last_clean":"2024-04-21T09:08:12.763722+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:08:12.763722+0000","last_undegraded":"2024-04-21T09:08:12.763722+0000","last_fullsized":"2024-04-21T09:08:12.763722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142352+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:08:12.142352+0000","last_peered":"2024-04-21T09:08:12.142352+0000","last_clean":"2024-04-21T09:08:12.142352+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:08:12.142352+0000","last_undegraded":"2024-04-21T09:08:12.142352+0000","last_fullsized":"2024-04-21T09:08:12.142352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033592+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:08:12.033592+0000","last_peered":"2024-04-21T09:08:12.033592+0000","last_clean":"2024-04-21T09:08:12.033592+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:08:12.033592+0000","last_undegraded":"2024-04-21T09:08:12.033592+0000","last_fullsized":"2024-04-21T09:08:12.033592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142660+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:08:12.142660+0000","last_peered":"2024-04-21T09:08:12.142660+0000","last_clean":"2024-04-21T09:08:12.142660+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:08:12.142660+0000","last_undegraded":"2024-04-21T09:08:12.142660+0000","last_fullsized":"2024-04-21T09:08:12.142660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'70","reported_seq":159,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.998429+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:08:17.998429+0000","last_peered":"2024-04-21T09:08:17.998429+0000","last_clean":"2024-04-21T09:08:17.998429+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:08:17.998429+0000","last_undegraded":"2024-04-21T09:08:17.998429+0000","last_fullsized":"2024-04-21T09:08:17.998429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":125,"num_write_kb":1157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143688+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:08:12.143688+0000","last_peered":"2024-04-21T09:08:12.143688+0000","last_clean":"2024-04-21T09:08:12.143688+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:08:12.143688+0000","last_undegraded":"2024-04-21T09:08:12.143688+0000","last_fullsized":"2024-04-21T09:08:12.143688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034657+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:08:12.034657+0000","last_peered":"2024-04-21T09:08:12.034657+0000","last_clean":"2024-04-21T09:08:12.034657+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:08:12.034657+0000","last_undegraded":"2024-04-21T09:08:12.034657+0000","last_fullsized":"2024-04-21T09:08:12.034657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281282+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:08:12.281282+0000","last_peered":"2024-04-21T09:08:12.281282+0000","last_clean":"2024-04-21T09:08:12.281282+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:08:12.281282+0000","last_undegraded":"2024-04-21T09:08:12.281282+0000","last_fullsized":"2024-04-21T09:08:12.281282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633978+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:08:13.633978+0000","last_peered":"2024-04-21T09:08:13.633978+0000","last_clean":"2024-04-21T09:08:13.633978+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:08:13.633978+0000","last_undegraded":"2024-04-21T09:08:13.633978+0000","last_fullsized":"2024-04-21T09:08:13.633978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281130+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:08:12.281130+0000","last_peered":"2024-04-21T09:08:12.281130+0000","last_clean":"2024-04-21T09:08:12.281130+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:08:12.281130+0000","last_undegraded":"2024-04-21T09:08:12.281130+0000","last_fullsized":"2024-04-21T09:08:12.281130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323960+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:08:12.323960+0000","last_peered":"2024-04-21T09:08:12.323960+0000","last_clean":"2024-04-21T09:08:12.323960+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:08:12.323960+0000","last_undegraded":"2024-04-21T09:08:12.323960+0000","last_fullsized":"2024-04-21T09:08:12.323960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.141989+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:08:12.141989+0000","last_peered":"2024-04-21T09:08:12.141989+0000","last_clean":"2024-04-21T09:08:12.141989+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:08:12.141989+0000","last_undegraded":"2024-04-21T09:08:12.141989+0000","last_fullsized":"2024-04-21T09:08:12.141989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140305+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:08:12.140305+0000","last_peered":"2024-04-21T09:08:12.140305+0000","last_clean":"2024-04-21T09:08:12.140305+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:08:12.140305+0000","last_undegraded":"2024-04-21T09:08:12.140305+0000","last_fullsized":"2024-04-21T09:08:12.140305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143035+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:08:12.143035+0000","last_peered":"2024-04-21T09:08:12.143035+0000","last_clean":"2024-04-21T09:08:12.143035+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:08:12.143035+0000","last_undegraded":"2024-04-21T09:08:12.143035+0000","last_fullsized":"2024-04-21T09:08:12.143035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763571+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:08:12.763571+0000","last_peered":"2024-04-21T09:08:12.763571+0000","last_clean":"2024-04-21T09:08:12.763571+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:08:12.763571+0000","last_undegraded":"2024-04-21T09:08:12.763571+0000","last_fullsized":"2024-04-21T09:08:12.763571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281011+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:08:12.281011+0000","last_peered":"2024-04-21T09:08:12.281011+0000","last_clean":"2024-04-21T09:08:12.281011+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:08:12.281011+0000","last_undegraded":"2024-04-21T09:08:12.281011+0000","last_fullsized":"2024-04-21T09:08:12.281011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034866+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:08:12.034866+0000","last_peered":"2024-04-21T09:08:12.034866+0000","last_clean":"2024-04-21T09:08:12.034866+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:08:12.034866+0000","last_undegraded":"2024-04-21T09:08:12.034866+0000","last_fullsized":"2024-04-21T09:08:12.034866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143614+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:08:12.143614+0000","last_peered":"2024-04-21T09:08:12.143614+0000","last_clean":"2024-04-21T09:08:12.143614+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:08:12.143614+0000","last_undegraded":"2024-04-21T09:08:12.143614+0000","last_fullsized":"2024-04-21T09:08:12.143614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763696+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:08:12.763696+0000","last_peered":"2024-04-21T09:08:12.763696+0000","last_clean":"2024-04-21T09:08:12.763696+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:08:12.763696+0000","last_undegraded":"2024-04-21T09:08:12.763696+0000","last_fullsized":"2024-04-21T09:08:12.763696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325194+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:08:12.325194+0000","last_peered":"2024-04-21T09:08:12.325194+0000","last_clean":"2024-04-21T09:08:12.325194+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:08:12.325194+0000","last_undegraded":"2024-04-21T09:08:12.325194+0000","last_fullsized":"2024-04-21T09:08:12.325194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324153+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:08:12.324153+0000","last_peered":"2024-04-21T09:08:12.324153+0000","last_clean":"2024-04-21T09:08:12.324153+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:08:12.324153+0000","last_undegraded":"2024-04-21T09:08:12.324153+0000","last_fullsized":"2024-04-21T09:08:12.324153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324834+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:08:12.324834+0000","last_peered":"2024-04-21T09:08:12.324834+0000","last_clean":"2024-04-21T09:08:12.324834+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:08:12.324834+0000","last_undegraded":"2024-04-21T09:08:12.324834+0000","last_fullsized":"2024-04-21T09:08:12.324834+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325898+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:08:12.325898+0000","last_peered":"2024-04-21T09:08:12.325898+0000","last_clean":"2024-04-21T09:08:12.325898+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:08:12.325898+0000","last_undegraded":"2024-04-21T09:08:12.325898+0000","last_fullsized":"2024-04-21T09:08:12.325898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633710+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:08:13.633710+0000","last_peered":"2024-04-21T09:08:13.633710+0000","last_clean":"2024-04-21T09:08:13.633710+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:08:13.633710+0000","last_undegraded":"2024-04-21T09:08:13.633710+0000","last_fullsized":"2024-04-21T09:08:13.633710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"96'57","reported_seq":111,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.993708+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:08:17.993708+0000","last_peered":"2024-04-21T09:08:17.993708+0000","last_clean":"2024-04-21T09:08:17.993708+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:08:17.993708+0000","last_undegraded":"2024-04-21T09:08:17.993708+0000","last_fullsized":"2024-04-21T09:08:17.993708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324272+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:08:12.324272+0000","last_peered":"2024-04-21T09:08:12.324272+0000","last_clean":"2024-04-21T09:08:12.324272+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:08:12.324272+0000","last_undegraded":"2024-04-21T09:08:12.324272+0000","last_fullsized":"2024-04-21T09:08:12.324272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150326+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:08:12.150326+0000","last_peered":"2024-04-21T09:08:12.150326+0000","last_clean":"2024-04-21T09:08:12.150326+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:08:12.150326+0000","last_undegraded":"2024-04-21T09:08:12.150326+0000","last_fullsized":"2024-04-21T09:08:12.150326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033814+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:08:12.033814+0000","last_peered":"2024-04-21T09:08:12.033814+0000","last_clean":"2024-04-21T09:08:12.033814+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:08:12.033814+0000","last_undegraded":"2024-04-21T09:08:12.033814+0000","last_fullsized":"2024-04-21T09:08:12.033814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325958+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:08:12.325958+0000","last_peered":"2024-04-21T09:08:12.325958+0000","last_clean":"2024-04-21T09:08:12.325958+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:08:12.325958+0000","last_undegraded":"2024-04-21T09:08:12.325958+0000","last_fullsized":"2024-04-21T09:08:12.325958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763645+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:08:12.763645+0000","last_peered":"2024-04-21T09:08:12.763645+0000","last_clean":"2024-04-21T09:08:12.763645+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:08:12.763645+0000","last_undegraded":"2024-04-21T09:08:12.763645+0000","last_fullsized":"2024-04-21T09:08:12.763645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034801+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:08:12.034801+0000","last_peered":"2024-04-21T09:08:12.034801+0000","last_clean":"2024-04-21T09:08:12.034801+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:08:12.034801+0000","last_undegraded":"2024-04-21T09:08:12.034801+0000","last_fullsized":"2024-04-21T09:08:12.034801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034715+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:08:12.034715+0000","last_peered":"2024-04-21T09:08:12.034715+0000","last_clean":"2024-04-21T09:08:12.034715+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:08:12.034715+0000","last_undegraded":"2024-04-21T09:08:12.034715+0000","last_fullsized":"2024-04-21T09:08:12.034715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633881+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:08:13.633881+0000","last_peered":"2024-04-21T09:08:13.633881+0000","last_clean":"2024-04-21T09:08:13.633881+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:08:13.633881+0000","last_undegraded":"2024-04-21T09:08:13.633881+0000","last_fullsized":"2024-04-21T09:08:13.633881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033916+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:08:12.033916+0000","last_peered":"2024-04-21T09:08:12.033916+0000","last_clean":"2024-04-21T09:08:12.033916+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:08:12.033916+0000","last_undegraded":"2024-04-21T09:08:12.033916+0000","last_fullsized":"2024-04-21T09:08:12.033916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763506+0000","last_change":"2024-04-21T09:07:40.649825+0000","last_active":"2024-04-21T09:08:12.763506+0000","last_peered":"2024-04-21T09:08:12.763506+0000","last_clean":"2024-04-21T09:08:12.763506+0000","last_became_active":"2024-04-21T09:07:40.649264+0000","last_became_peered":"2024-04-21T09:07:40.649264+0000","last_unstale":"2024-04-21T09:08:12.763506+0000","last_undegraded":"2024-04-21T09:08:12.763506+0000","last_fullsized":"2024-04-21T09:08:12.763506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:38.142948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035249+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:08:12.035249+0000","last_peered":"2024-04-21T09:08:12.035249+0000","last_clean":"2024-04-21T09:08:12.035249+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:08:12.035249+0000","last_undegraded":"2024-04-21T09:08:12.035249+0000","last_fullsized":"2024-04-21T09:08:12.035249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143896+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:08:12.143896+0000","last_peered":"2024-04-21T09:08:12.143896+0000","last_clean":"2024-04-21T09:08:12.143896+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:08:12.143896+0000","last_undegraded":"2024-04-21T09:08:12.143896+0000","last_fullsized":"2024-04-21T09:08:12.143896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150636+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:08:12.150636+0000","last_peered":"2024-04-21T09:08:12.150636+0000","last_clean":"2024-04-21T09:08:12.150636+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:08:12.150636+0000","last_undegraded":"2024-04-21T09:08:12.150636+0000","last_fullsized":"2024-04-21T09:08:12.150636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143204+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:08:12.143204+0000","last_peered":"2024-04-21T09:08:12.143204+0000","last_clean":"2024-04-21T09:08:12.143204+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:08:12.143204+0000","last_undegraded":"2024-04-21T09:08:12.143204+0000","last_fullsized":"2024-04-21T09:08:12.143204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324412+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:08:12.324412+0000","last_peered":"2024-04-21T09:08:12.324412+0000","last_clean":"2024-04-21T09:08:12.324412+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:08:12.324412+0000","last_undegraded":"2024-04-21T09:08:12.324412+0000","last_fullsized":"2024-04-21T09:08:12.324412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033628+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:08:12.033628+0000","last_peered":"2024-04-21T09:08:12.033628+0000","last_clean":"2024-04-21T09:08:12.033628+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:08:12.033628+0000","last_undegraded":"2024-04-21T09:08:12.033628+0000","last_fullsized":"2024-04-21T09:08:12.033628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763507+0000","last_change":"2024-04-21T09:07:40.650222+0000","last_active":"2024-04-21T09:08:12.763507+0000","last_peered":"2024-04-21T09:08:12.763507+0000","last_clean":"2024-04-21T09:08:12.763507+0000","last_became_active":"2024-04-21T09:07:40.649595+0000","last_became_peered":"2024-04-21T09:07:40.649595+0000","last_unstale":"2024-04-21T09:08:12.763507+0000","last_undegraded":"2024-04-21T09:08:12.763507+0000","last_fullsized":"2024-04-21T09:08:12.763507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:25.006017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107013,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793576,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":976,"num_read_kb":645,"num_write":351,"num_write_kb":2090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":638976,"data_stored":2380728,"data_compressed":107238,"data_compressed_allocated":147456,"data_compressed_original":2125824,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":541927,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32368,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712784,"statfs":{"total":95995035648,"available":95961890816,"internally_reserved":0,"allocated":1220608,"data_stored":1328900,"data_compressed":75912,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":939453,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":935255,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1138669,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":537317,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":605237,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":538705,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":604801,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1062327,"data_compressed":57756,"data_compressed_allocated":77824,"data_compressed_original":606208,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888143,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":543679,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":1265158,"data_compressed":74281,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68010,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67574,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34115,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330099,"data_compressed":14840,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":397937,"data_compressed":16471,"data_compressed_allocated":24576,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:08:23.090 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-21T09:08:23.091 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T09:08:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 -- 172.21.15.66:0/2616794666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 msgr2=0x7f9bc410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/2616794666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc410dc90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009cd0 tx=0x7f9bbc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 -- 172.21.15.66:0/2616794666 shutdown_connections 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/2616794666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc410dc90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/2616794666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bc41011a0 0x7f9bc4101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/2616794666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc4105e30 0x7f9bc4106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 -- 172.21.15.66:0/2616794666 >> 172.21.15.66:0/2616794666 conn(0x7f9bc4077e30 msgr2=0x7f9bc4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 -- 172.21.15.66:0/2616794666 shutdown_connections 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.227+0000 7f9bcb53d640 1 -- 172.21.15.66:0/2616794666 wait complete. 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 Processor -- start 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 -- start start 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc41011a0 0x7f9bc4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bc4105e30 0x7f9bc419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc41110c0 con 0x7f9bc4101b60 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4110f40 con 0x7f9bc4105e30 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bcb53d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4111240 con 0x7f9bc41011a0 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33266/0 (socket says 172.21.15.66:33266) 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 -- 172.21.15.66:0/879428525 learned_addr learned my addr 172.21.15.66:0/879428525 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc92d9640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc41011a0 0x7f9bc4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc9ada640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bc4105e30 0x7f9bc419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 -- 172.21.15.66:0/879428525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc41011a0 msgr2=0x7f9bc4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc41011a0 0x7f9bc4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 -- 172.21.15.66:0/879428525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bc4105e30 msgr2=0x7f9bc419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bc4105e30 0x7f9bc419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 -- 172.21.15.66:0/879428525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc419d650 con 0x7f9bc4101b60 2024-04-21T09:08:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.228+0000 7f9bc8ad8640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc4198ba0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000d950 tx=0x7f9bc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.229+0000 7f9bba7fc640 1 -- 172.21.15.66:0/879428525 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bc4101b60 2024-04-21T09:08:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.229+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc419d940 con 0x7f9bc4101b60 2024-04-21T09:08:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.229+0000 7f9bba7fc640 1 -- 172.21.15.66:0/879428525 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc0009c30 con 0x7f9bc4101b60 2024-04-21T09:08:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.229+0000 7f9bba7fc640 1 -- 172.21.15.66:0/879428525 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000f620 con 0x7f9bc4101b60 2024-04-21T09:08:23.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.229+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc41a51d0 con 0x7f9bc4101b60 2024-04-21T09:08:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.230+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc4068de0 con 0x7f9bc4101b60 2024-04-21T09:08:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.231+0000 7f9bba7fc640 1 -- 172.21.15.66:0/879428525 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f9bc0010460 con 0x7f9bc4101b60 2024-04-21T09:08:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.232+0000 7f9bba7fc640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9bb00a3db0 0x7f9bb00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.232+0000 7f9bc92d9640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9bb00a3db0 0x7f9bb00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.232+0000 7f9bba7fc640 1 -- 172.21.15.66:0/879428525 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f9bc00d08c0 con 0x7f9bc4101b60 2024-04-21T09:08:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.232+0000 7f9bc92d9640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9bb00a3db0 0x7f9bb00a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400bbe0 tx=0x7f9bb400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:23.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.237+0000 7f9bba7fc640 1 -- 172.21.15.66:0/879428525 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9bc0009de0 con 0x7f9bc4101b60 2024-04-21T09:08:23.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:23 smithi066 ceph-mon[27953]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-04-21T09:08:23.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:23 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2104094628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:23.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.424+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bc410a3c0 con 0x7f9bb00a3db0 2024-04-21T09:08:23.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.433+0000 7f9bba7fc640 1 -- 172.21.15.66:0/879428525 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339717 (secure 0 0 0) 0x7f9bc410a3c0 con 0x7f9bb00a3db0 2024-04-21T09:08:23.437 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:23.439 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-04-21T09:08:23.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9bb00a3db0 msgr2=0x7f9bb00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9bb00a3db0 0x7f9bb00a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400bbe0 tx=0x7f9bb400b390 comp rx=0 tx=0).stop 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 msgr2=0x7f9bc4198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc4198ba0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000d950 tx=0x7f9bc000de10 comp rx=0 tx=0).stop 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 shutdown_connections 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9bb00a3db0 0x7f9bb00a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bc4105e30 0x7f9bc419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bc4101b60 0x7f9bc4198ba0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 --2- 172.21.15.66:0/879428525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc41011a0 0x7f9bc4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 >> 172.21.15.66:0/879428525 conn(0x7f9bc4077e30 msgr2=0x7f9bc4069360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 shutdown_connections 2024-04-21T09:08:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.437+0000 7f9bcb53d640 1 -- 172.21.15.66:0/879428525 wait complete. 2024-04-21T09:08:23.448 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T09:08:22.049328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384178,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1213,"num_read_kb":975,"num_write":554,"num_write_kb":5030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":919,"ondisk_log_size":919,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357992,"kb_used_data":12968,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583832,"statfs":{"total":1151940427776,"available":1151573843968,"internally_reserved":0,"allocated":13279232,"data_stored":10041428,"data_compressed":577260,"data_compressed_allocated":786432,"data_compressed_original":4423680,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":246,"num_read_kb":331,"num_write":84,"num_write_kb":652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991412"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764225+0000","last_change":"2024-04-21T09:05:46.968644+0000","last_active":"2024-04-21T09:08:12.764225+0000","last_peered":"2024-04-21T09:08:12.764225+0000","last_clean":"2024-04-21T09:08:12.764225+0000","last_became_active":"2024-04-21T09:05:46.967752+0000","last_became_peered":"2024-04-21T09:05:46.967752+0000","last_unstale":"2024-04-21T09:08:12.764225+0000","last_undegraded":"2024-04-21T09:08:12.764225+0000","last_fullsized":"2024-04-21T09:08:12.764225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:15:42.383670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.634069+0000","last_change":"2024-04-21T09:05:46.968000+0000","last_active":"2024-04-21T09:08:13.634069+0000","last_peered":"2024-04-21T09:08:13.634069+0000","last_clean":"2024-04-21T09:08:13.634069+0000","last_became_active":"2024-04-21T09:05:46.967657+0000","last_became_peered":"2024-04-21T09:05:46.967657+0000","last_unstale":"2024-04-21T09:08:13.634069+0000","last_undegraded":"2024-04-21T09:08:13.634069+0000","last_fullsized":"2024-04-21T09:08:13.634069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:25.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633957+0000","last_change":"2024-04-21T09:05:46.972187+0000","last_active":"2024-04-21T09:08:13.633957+0000","last_peered":"2024-04-21T09:08:13.633957+0000","last_clean":"2024-04-21T09:08:13.633957+0000","last_became_active":"2024-04-21T09:05:46.971816+0000","last_became_peered":"2024-04-21T09:05:46.971816+0000","last_unstale":"2024-04-21T09:08:13.633957+0000","last_undegraded":"2024-04-21T09:08:13.633957+0000","last_fullsized":"2024-04-21T09:08:13.633957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:10.342218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281097+0000","last_change":"2024-04-21T09:05:46.976434+0000","last_active":"2024-04-21T09:08:12.281097+0000","last_peered":"2024-04-21T09:08:12.281097+0000","last_clean":"2024-04-21T09:08:12.281097+0000","last_became_active":"2024-04-21T09:05:46.976006+0000","last_became_peered":"2024-04-21T09:05:46.976006+0000","last_unstale":"2024-04-21T09:08:12.281097+0000","last_undegraded":"2024-04-21T09:08:12.281097+0000","last_fullsized":"2024-04-21T09:08:12.281097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:47.265077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142249+0000","last_change":"2024-04-21T09:05:46.975614+0000","last_active":"2024-04-21T09:08:12.142249+0000","last_peered":"2024-04-21T09:08:12.142249+0000","last_clean":"2024-04-21T09:08:12.142249+0000","last_became_active":"2024-04-21T09:05:46.975175+0000","last_became_peered":"2024-04-21T09:05:46.975175+0000","last_unstale":"2024-04-21T09:08:12.142249+0000","last_undegraded":"2024-04-21T09:08:12.142249+0000","last_fullsized":"2024-04-21T09:08:12.142249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:37.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034661+0000","last_change":"2024-04-21T09:05:46.973773+0000","last_active":"2024-04-21T09:08:12.034661+0000","last_peered":"2024-04-21T09:08:12.034661+0000","last_clean":"2024-04-21T09:08:12.034661+0000","last_became_active":"2024-04-21T09:05:46.972789+0000","last_became_peered":"2024-04-21T09:05:46.972789+0000","last_unstale":"2024-04-21T09:08:12.034661+0000","last_undegraded":"2024-04-21T09:08:12.034661+0000","last_fullsized":"2024-04-21T09:08:12.034661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:42.487540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324873+0000","last_change":"2024-04-21T09:05:46.981239+0000","last_active":"2024-04-21T09:08:12.324873+0000","last_peered":"2024-04-21T09:08:12.324873+0000","last_clean":"2024-04-21T09:08:12.324873+0000","last_became_active":"2024-04-21T09:05:46.980744+0000","last_became_peered":"2024-04-21T09:05:46.980744+0000","last_unstale":"2024-04-21T09:08:12.324873+0000","last_undegraded":"2024-04-21T09:08:12.324873+0000","last_fullsized":"2024-04-21T09:08:12.324873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:17.560226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142696+0000","last_change":"2024-04-21T09:05:46.976447+0000","last_active":"2024-04-21T09:08:12.142696+0000","last_peered":"2024-04-21T09:08:12.142696+0000","last_clean":"2024-04-21T09:08:12.142696+0000","last_became_active":"2024-04-21T09:05:46.976135+0000","last_became_peered":"2024-04-21T09:05:46.976135+0000","last_unstale":"2024-04-21T09:08:12.142696+0000","last_undegraded":"2024-04-21T09:08:12.142696+0000","last_fullsized":"2024-04-21T09:08:12.142696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:19.022357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325536+0000","last_change":"2024-04-21T09:05:46.968656+0000","last_active":"2024-04-21T09:08:12.325536+0000","last_peered":"2024-04-21T09:08:12.325536+0000","last_clean":"2024-04-21T09:08:12.325536+0000","last_became_active":"2024-04-21T09:05:46.968343+0000","last_became_peered":"2024-04-21T09:05:46.968343+0000","last_unstale":"2024-04-21T09:08:12.325536+0000","last_undegraded":"2024-04-21T09:08:12.325536+0000","last_fullsized":"2024-04-21T09:08:12.325536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:47.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035076+0000","last_change":"2024-04-21T09:05:46.970825+0000","last_active":"2024-04-21T09:08:12.035076+0000","last_peered":"2024-04-21T09:08:12.035076+0000","last_clean":"2024-04-21T09:08:12.035076+0000","last_became_active":"2024-04-21T09:05:46.970485+0000","last_became_peered":"2024-04-21T09:05:46.970485+0000","last_unstale":"2024-04-21T09:08:12.035076+0000","last_undegraded":"2024-04-21T09:08:12.035076+0000","last_fullsized":"2024-04-21T09:08:12.035076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:26.073481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325727+0000","last_change":"2024-04-21T09:05:46.982117+0000","last_active":"2024-04-21T09:08:12.325727+0000","last_peered":"2024-04-21T09:08:12.325727+0000","last_clean":"2024-04-21T09:08:12.325727+0000","last_became_active":"2024-04-21T09:05:46.978016+0000","last_became_peered":"2024-04-21T09:05:46.978016+0000","last_unstale":"2024-04-21T09:08:12.325727+0000","last_undegraded":"2024-04-21T09:08:12.325727+0000","last_fullsized":"2024-04-21T09:08:12.325727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:27:53.755934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281072+0000","last_change":"2024-04-21T09:05:46.980385+0000","last_active":"2024-04-21T09:08:12.281072+0000","last_peered":"2024-04-21T09:08:12.281072+0000","last_clean":"2024-04-21T09:08:12.281072+0000","last_became_active":"2024-04-21T09:05:46.979804+0000","last_became_peered":"2024-04-21T09:05:46.979804+0000","last_unstale":"2024-04-21T09:08:12.281072+0000","last_undegraded":"2024-04-21T09:08:12.281072+0000","last_fullsized":"2024-04-21T09:08:12.281072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:14:45.112741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033565+0000","last_change":"2024-04-21T09:05:46.967699+0000","last_active":"2024-04-21T09:08:12.033565+0000","last_peered":"2024-04-21T09:08:12.033565+0000","last_clean":"2024-04-21T09:08:12.033565+0000","last_became_active":"2024-04-21T09:05:46.967334+0000","last_became_peered":"2024-04-21T09:05:46.967334+0000","last_unstale":"2024-04-21T09:08:12.033565+0000","last_undegraded":"2024-04-21T09:08:12.033565+0000","last_fullsized":"2024-04-21T09:08:12.033565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:06.986491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633566+0000","last_change":"2024-04-21T09:05:46.965971+0000","last_active":"2024-04-21T09:08:13.633566+0000","last_peered":"2024-04-21T09:08:13.633566+0000","last_clean":"2024-04-21T09:08:13.633566+0000","last_became_active":"2024-04-21T09:05:46.965616+0000","last_became_peered":"2024-04-21T09:05:46.965616+0000","last_unstale":"2024-04-21T09:08:13.633566+0000","last_undegraded":"2024-04-21T09:08:13.633566+0000","last_fullsized":"2024-04-21T09:08:13.633566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:01:02.763064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633612+0000","last_change":"2024-04-21T09:05:46.987193+0000","last_active":"2024-04-21T09:08:13.633612+0000","last_peered":"2024-04-21T09:08:13.633612+0000","last_clean":"2024-04-21T09:08:13.633612+0000","last_became_active":"2024-04-21T09:05:46.986701+0000","last_became_peered":"2024-04-21T09:05:46.986701+0000","last_unstale":"2024-04-21T09:08:13.633612+0000","last_undegraded":"2024-04-21T09:08:13.633612+0000","last_fullsized":"2024-04-21T09:08:13.633612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:42:22.507757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323842+0000","last_change":"2024-04-21T09:05:46.975473+0000","last_active":"2024-04-21T09:08:12.323842+0000","last_peered":"2024-04-21T09:08:12.323842+0000","last_clean":"2024-04-21T09:08:12.323842+0000","last_became_active":"2024-04-21T09:05:46.975091+0000","last_became_peered":"2024-04-21T09:05:46.975091+0000","last_unstale":"2024-04-21T09:08:12.323842+0000","last_undegraded":"2024-04-21T09:08:12.323842+0000","last_fullsized":"2024-04-21T09:08:12.323842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:38:07.843566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.764134+0000","last_change":"2024-04-21T09:05:46.969337+0000","last_active":"2024-04-21T09:08:12.764134+0000","last_peered":"2024-04-21T09:08:12.764134+0000","last_clean":"2024-04-21T09:08:12.764134+0000","last_became_active":"2024-04-21T09:05:46.968523+0000","last_became_peered":"2024-04-21T09:05:46.968523+0000","last_unstale":"2024-04-21T09:08:12.764134+0000","last_undegraded":"2024-04-21T09:08:12.764134+0000","last_fullsized":"2024-04-21T09:08:12.764134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:58.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140490+0000","last_change":"2024-04-21T09:05:46.966047+0000","last_active":"2024-04-21T09:08:12.140490+0000","last_peered":"2024-04-21T09:08:12.140490+0000","last_clean":"2024-04-21T09:08:12.140490+0000","last_became_active":"2024-04-21T09:05:46.965705+0000","last_became_peered":"2024-04-21T09:05:46.965705+0000","last_unstale":"2024-04-21T09:08:12.140490+0000","last_undegraded":"2024-04-21T09:08:12.140490+0000","last_fullsized":"2024-04-21T09:08:12.140490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:52:39.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035166+0000","last_change":"2024-04-21T09:05:46.968735+0000","last_active":"2024-04-21T09:08:12.035166+0000","last_peered":"2024-04-21T09:08:12.035166+0000","last_clean":"2024-04-21T09:08:12.035166+0000","last_became_active":"2024-04-21T09:05:46.968406+0000","last_became_peered":"2024-04-21T09:05:46.968406+0000","last_unstale":"2024-04-21T09:08:12.035166+0000","last_undegraded":"2024-04-21T09:08:12.035166+0000","last_fullsized":"2024-04-21T09:08:12.035166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:25.735221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034868+0000","last_change":"2024-04-21T09:05:51.014759+0000","last_active":"2024-04-21T09:08:12.034868+0000","last_peered":"2024-04-21T09:08:12.034868+0000","last_clean":"2024-04-21T09:08:12.034868+0000","last_became_active":"2024-04-21T09:05:51.014420+0000","last_became_peered":"2024-04-21T09:05:51.014420+0000","last_unstale":"2024-04-21T09:08:12.034868+0000","last_undegraded":"2024-04-21T09:08:12.034868+0000","last_fullsized":"2024-04-21T09:08:12.034868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:43.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140333+0000","last_change":"2024-04-21T09:05:45.964877+0000","last_active":"2024-04-21T09:08:12.140333+0000","last_peered":"2024-04-21T09:08:12.140333+0000","last_clean":"2024-04-21T09:08:12.140333+0000","last_became_active":"2024-04-21T09:05:45.964515+0000","last_became_peered":"2024-04-21T09:05:45.964515+0000","last_unstale":"2024-04-21T09:08:12.140333+0000","last_undegraded":"2024-04-21T09:08:12.140333+0000","last_fullsized":"2024-04-21T09:08:12.140333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:32:58.533209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143771+0000","last_change":"2024-04-21T09:05:46.978940+0000","last_active":"2024-04-21T09:08:12.143771+0000","last_peered":"2024-04-21T09:08:12.143771+0000","last_clean":"2024-04-21T09:08:12.143771+0000","last_became_active":"2024-04-21T09:05:46.978618+0000","last_became_peered":"2024-04-21T09:05:46.978618+0000","last_unstale":"2024-04-21T09:08:12.143771+0000","last_undegraded":"2024-04-21T09:08:12.143771+0000","last_fullsized":"2024-04-21T09:08:12.143771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:22.741220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280764+0000","last_change":"2024-04-21T09:05:51.021902+0000","last_active":"2024-04-21T09:08:12.280764+0000","last_peered":"2024-04-21T09:08:12.280764+0000","last_clean":"2024-04-21T09:08:12.280764+0000","last_became_active":"2024-04-21T09:05:51.021407+0000","last_became_peered":"2024-04-21T09:05:51.021407+0000","last_unstale":"2024-04-21T09:08:12.280764+0000","last_undegraded":"2024-04-21T09:08:12.280764+0000","last_fullsized":"2024-04-21T09:08:12.280764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:57:16.742015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.144097+0000","last_change":"2024-04-21T09:05:45.967792+0000","last_active":"2024-04-21T09:08:12.144097+0000","last_peered":"2024-04-21T09:08:12.144097+0000","last_clean":"2024-04-21T09:08:12.144097+0000","last_became_active":"2024-04-21T09:05:45.967446+0000","last_became_peered":"2024-04-21T09:05:45.967446+0000","last_unstale":"2024-04-21T09:08:12.144097+0000","last_undegraded":"2024-04-21T09:08:12.144097+0000","last_fullsized":"2024-04-21T09:08:12.144097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:43:35.875281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633901+0000","last_change":"2024-04-21T09:05:46.968823+0000","last_active":"2024-04-21T09:08:13.633901+0000","last_peered":"2024-04-21T09:08:13.633901+0000","last_clean":"2024-04-21T09:08:13.633901+0000","last_became_active":"2024-04-21T09:05:46.968473+0000","last_became_peered":"2024-04-21T09:05:46.968473+0000","last_unstale":"2024-04-21T09:08:13.633901+0000","last_undegraded":"2024-04-21T09:08:13.633901+0000","last_fullsized":"2024-04-21T09:08:13.633901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:38.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280878+0000","last_change":"2024-04-21T09:06:40.717725+0000","last_active":"2024-04-21T09:08:12.280878+0000","last_peered":"2024-04-21T09:08:12.280878+0000","last_clean":"2024-04-21T09:08:12.280878+0000","last_became_active":"2024-04-21T09:06:40.717292+0000","last_became_peered":"2024-04-21T09:06:40.717292+0000","last_unstale":"2024-04-21T09:08:12.280878+0000","last_undegraded":"2024-04-21T09:08:12.280878+0000","last_fullsized":"2024-04-21T09:08:12.280878+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:31.114619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142503+0000","last_change":"2024-04-21T09:05:45.967838+0000","last_active":"2024-04-21T09:08:12.142503+0000","last_peered":"2024-04-21T09:08:12.142503+0000","last_clean":"2024-04-21T09:08:12.142503+0000","last_became_active":"2024-04-21T09:05:45.967416+0000","last_became_peered":"2024-04-21T09:05:45.967416+0000","last_unstale":"2024-04-21T09:08:12.142503+0000","last_undegraded":"2024-04-21T09:08:12.142503+0000","last_fullsized":"2024-04-21T09:08:12.142503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:36.797412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325104+0000","last_change":"2024-04-21T09:05:46.966310+0000","last_active":"2024-04-21T09:08:12.325104+0000","last_peered":"2024-04-21T09:08:12.325104+0000","last_clean":"2024-04-21T09:08:12.325104+0000","last_became_active":"2024-04-21T09:05:46.965961+0000","last_became_peered":"2024-04-21T09:05:46.965961+0000","last_unstale":"2024-04-21T09:08:12.325104+0000","last_undegraded":"2024-04-21T09:08:12.325104+0000","last_fullsized":"2024-04-21T09:08:12.325104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:20:52.180332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280773+0000","last_change":"2024-04-21T09:05:51.022071+0000","last_active":"2024-04-21T09:08:12.280773+0000","last_peered":"2024-04-21T09:08:12.280773+0000","last_clean":"2024-04-21T09:08:12.280773+0000","last_became_active":"2024-04-21T09:05:51.021526+0000","last_became_peered":"2024-04-21T09:05:51.021526+0000","last_unstale":"2024-04-21T09:08:12.280773+0000","last_undegraded":"2024-04-21T09:08:12.280773+0000","last_fullsized":"2024-04-21T09:08:12.280773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:15.259080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142157+0000","last_change":"2024-04-21T09:05:45.960677+0000","last_active":"2024-04-21T09:08:12.142157+0000","last_peered":"2024-04-21T09:08:12.142157+0000","last_clean":"2024-04-21T09:08:12.142157+0000","last_became_active":"2024-04-21T09:05:45.960356+0000","last_became_peered":"2024-04-21T09:05:45.960356+0000","last_unstale":"2024-04-21T09:08:12.142157+0000","last_undegraded":"2024-04-21T09:08:12.142157+0000","last_fullsized":"2024-04-21T09:08:12.142157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:53:31.562096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150927+0000","last_change":"2024-04-21T09:05:46.979888+0000","last_active":"2024-04-21T09:08:12.150927+0000","last_peered":"2024-04-21T09:08:12.150927+0000","last_clean":"2024-04-21T09:08:12.150927+0000","last_became_active":"2024-04-21T09:05:46.979564+0000","last_became_peered":"2024-04-21T09:05:46.979564+0000","last_unstale":"2024-04-21T09:08:12.150927+0000","last_undegraded":"2024-04-21T09:08:12.150927+0000","last_fullsized":"2024-04-21T09:08:12.150927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.809520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034947+0000","last_change":"2024-04-21T09:05:51.010068+0000","last_active":"2024-04-21T09:08:12.034947+0000","last_peered":"2024-04-21T09:08:12.034947+0000","last_clean":"2024-04-21T09:08:12.034947+0000","last_became_active":"2024-04-21T09:05:51.009685+0000","last_became_peered":"2024-04-21T09:05:51.009685+0000","last_unstale":"2024-04-21T09:08:12.034947+0000","last_undegraded":"2024-04-21T09:08:12.034947+0000","last_fullsized":"2024-04-21T09:08:12.034947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:21.446260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'6","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.321068+0000","last_change":"2024-04-21T09:06:40.179732+0000","last_active":"2024-04-21T09:08:17.321068+0000","last_peered":"2024-04-21T09:08:17.321068+0000","last_clean":"2024-04-21T09:08:17.321068+0000","last_became_active":"2024-04-21T09:06:40.173361+0000","last_became_peered":"2024-04-21T09:06:40.173361+0000","last_unstale":"2024-04-21T09:08:17.321068+0000","last_undegraded":"2024-04-21T09:08:17.321068+0000","last_fullsized":"2024-04-21T09:08:17.321068+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:49.984341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142233+0000","last_change":"2024-04-21T09:05:46.974090+0000","last_active":"2024-04-21T09:08:12.142233+0000","last_peered":"2024-04-21T09:08:12.142233+0000","last_clean":"2024-04-21T09:08:12.142233+0000","last_became_active":"2024-04-21T09:05:46.973520+0000","last_became_peered":"2024-04-21T09:05:46.973520+0000","last_unstale":"2024-04-21T09:08:12.142233+0000","last_undegraded":"2024-04-21T09:08:12.142233+0000","last_fullsized":"2024-04-21T09:08:12.142233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:15.932982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035266+0000","last_change":"2024-04-21T09:05:51.020482+0000","last_active":"2024-04-21T09:08:12.035266+0000","last_peered":"2024-04-21T09:08:12.035266+0000","last_clean":"2024-04-21T09:08:12.035266+0000","last_became_active":"2024-04-21T09:05:51.020081+0000","last_became_peered":"2024-04-21T09:05:51.020081+0000","last_unstale":"2024-04-21T09:08:12.035266+0000","last_undegraded":"2024-04-21T09:08:12.035266+0000","last_fullsized":"2024-04-21T09:08:12.035266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:03.480043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034693+0000","last_change":"2024-04-21T09:05:45.970231+0000","last_active":"2024-04-21T09:08:12.034693+0000","last_peered":"2024-04-21T09:08:12.034693+0000","last_clean":"2024-04-21T09:08:12.034693+0000","last_became_active":"2024-04-21T09:05:45.969916+0000","last_became_peered":"2024-04-21T09:05:45.969916+0000","last_unstale":"2024-04-21T09:08:12.034693+0000","last_undegraded":"2024-04-21T09:08:12.034693+0000","last_fullsized":"2024-04-21T09:08:12.034693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:13:08.371101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150778+0000","last_change":"2024-04-21T09:05:46.977961+0000","last_active":"2024-04-21T09:08:12.150778+0000","last_peered":"2024-04-21T09:08:12.150778+0000","last_clean":"2024-04-21T09:08:12.150778+0000","last_became_active":"2024-04-21T09:05:46.977623+0000","last_became_peered":"2024-04-21T09:05:46.977623+0000","last_unstale":"2024-04-21T09:08:12.150778+0000","last_undegraded":"2024-04-21T09:08:12.150778+0000","last_fullsized":"2024-04-21T09:08:12.150778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:08:54.872793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763967+0000","last_change":"2024-04-21T09:05:51.016197+0000","last_active":"2024-04-21T09:08:12.763967+0000","last_peered":"2024-04-21T09:08:12.763967+0000","last_clean":"2024-04-21T09:08:12.763967+0000","last_became_active":"2024-04-21T09:05:51.015865+0000","last_became_peered":"2024-04-21T09:05:51.015865+0000","last_unstale":"2024-04-21T09:08:12.763967+0000","last_undegraded":"2024-04-21T09:08:12.763967+0000","last_fullsized":"2024-04-21T09:08:12.763967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:35:30.687882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033440+0000","last_change":"2024-04-21T09:05:45.963366+0000","last_active":"2024-04-21T09:08:12.033440+0000","last_peered":"2024-04-21T09:08:12.033440+0000","last_clean":"2024-04-21T09:08:12.033440+0000","last_became_active":"2024-04-21T09:05:45.963029+0000","last_became_peered":"2024-04-21T09:05:45.963029+0000","last_unstale":"2024-04-21T09:08:12.033440+0000","last_undegraded":"2024-04-21T09:08:12.033440+0000","last_fullsized":"2024-04-21T09:08:12.033440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:09:51.061302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142822+0000","last_change":"2024-04-21T09:05:46.970336+0000","last_active":"2024-04-21T09:08:12.142822+0000","last_peered":"2024-04-21T09:08:12.142822+0000","last_clean":"2024-04-21T09:08:12.142822+0000","last_became_active":"2024-04-21T09:05:46.969921+0000","last_became_peered":"2024-04-21T09:05:46.969921+0000","last_unstale":"2024-04-21T09:08:12.142822+0000","last_undegraded":"2024-04-21T09:08:12.142822+0000","last_fullsized":"2024-04-21T09:08:12.142822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:56.776962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633942+0000","last_change":"2024-04-21T09:05:51.024776+0000","last_active":"2024-04-21T09:08:13.633942+0000","last_peered":"2024-04-21T09:08:13.633942+0000","last_clean":"2024-04-21T09:08:13.633942+0000","last_became_active":"2024-04-21T09:05:51.024275+0000","last_became_peered":"2024-04-21T09:05:51.024275+0000","last_unstale":"2024-04-21T09:08:13.633942+0000","last_undegraded":"2024-04-21T09:08:13.633942+0000","last_fullsized":"2024-04-21T09:08:13.633942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:42:33.681705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150624+0000","last_change":"2024-04-21T09:05:45.964715+0000","last_active":"2024-04-21T09:08:12.150624+0000","last_peered":"2024-04-21T09:08:12.150624+0000","last_clean":"2024-04-21T09:08:12.150624+0000","last_became_active":"2024-04-21T09:05:45.964398+0000","last_became_peered":"2024-04-21T09:05:45.964398+0000","last_unstale":"2024-04-21T09:08:12.150624+0000","last_undegraded":"2024-04-21T09:08:12.150624+0000","last_fullsized":"2024-04-21T09:08:12.150624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:24:36.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139637+0000","last_change":"2024-04-21T09:05:46.975976+0000","last_active":"2024-04-21T09:08:12.139637+0000","last_peered":"2024-04-21T09:08:12.139637+0000","last_clean":"2024-04-21T09:08:12.139637+0000","last_became_active":"2024-04-21T09:05:46.975634+0000","last_became_peered":"2024-04-21T09:05:46.975634+0000","last_unstale":"2024-04-21T09:08:12.139637+0000","last_undegraded":"2024-04-21T09:08:12.139637+0000","last_fullsized":"2024-04-21T09:08:12.139637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:08:41.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142906+0000","last_change":"2024-04-21T09:05:51.017157+0000","last_active":"2024-04-21T09:08:12.142906+0000","last_peered":"2024-04-21T09:08:12.142906+0000","last_clean":"2024-04-21T09:08:12.142906+0000","last_became_active":"2024-04-21T09:05:51.016676+0000","last_became_peered":"2024-04-21T09:05:51.016676+0000","last_unstale":"2024-04-21T09:08:12.142906+0000","last_undegraded":"2024-04-21T09:08:12.142906+0000","last_fullsized":"2024-04-21T09:08:12.142906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:52:04.280930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":165,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.319293+0000","last_change":"2024-04-21T09:05:45.964010+0000","last_active":"2024-04-21T09:08:16.319293+0000","last_peered":"2024-04-21T09:08:16.319293+0000","last_clean":"2024-04-21T09:08:16.319293+0000","last_became_active":"2024-04-21T09:05:45.962802+0000","last_became_peered":"2024-04-21T09:05:45.962802+0000","last_unstale":"2024-04-21T09:08:16.319293+0000","last_undegraded":"2024-04-21T09:08:16.319293+0000","last_fullsized":"2024-04-21T09:08:16.319293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:15.681666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325164+0000","last_change":"2024-04-21T09:05:46.966427+0000","last_active":"2024-04-21T09:08:12.325164+0000","last_peered":"2024-04-21T09:08:12.325164+0000","last_clean":"2024-04-21T09:08:12.325164+0000","last_became_active":"2024-04-21T09:05:46.966012+0000","last_became_peered":"2024-04-21T09:05:46.966012+0000","last_unstale":"2024-04-21T09:08:12.325164+0000","last_undegraded":"2024-04-21T09:08:12.325164+0000","last_fullsized":"2024-04-21T09:08:12.325164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:00:29.124116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139403+0000","last_change":"2024-04-21T09:05:51.010759+0000","last_active":"2024-04-21T09:08:12.139403+0000","last_peered":"2024-04-21T09:08:12.139403+0000","last_clean":"2024-04-21T09:08:12.139403+0000","last_became_active":"2024-04-21T09:05:51.010328+0000","last_became_peered":"2024-04-21T09:05:51.010328+0000","last_unstale":"2024-04-21T09:08:12.139403+0000","last_undegraded":"2024-04-21T09:08:12.139403+0000","last_fullsized":"2024-04-21T09:08:12.139403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:30:58.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325630+0000","last_change":"2024-04-21T09:05:45.967635+0000","last_active":"2024-04-21T09:08:12.325630+0000","last_peered":"2024-04-21T09:08:12.325630+0000","last_clean":"2024-04-21T09:08:12.325630+0000","last_became_active":"2024-04-21T09:05:45.967332+0000","last_became_peered":"2024-04-21T09:05:45.967332+0000","last_unstale":"2024-04-21T09:08:12.325630+0000","last_undegraded":"2024-04-21T09:08:12.325630+0000","last_fullsized":"2024-04-21T09:08:12.325630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:51.752774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.634168+0000","last_change":"2024-04-21T09:05:46.987489+0000","last_active":"2024-04-21T09:08:13.634168+0000","last_peered":"2024-04-21T09:08:13.634168+0000","last_clean":"2024-04-21T09:08:13.634168+0000","last_became_active":"2024-04-21T09:05:46.986797+0000","last_became_peered":"2024-04-21T09:05:46.986797+0000","last_unstale":"2024-04-21T09:08:13.634168+0000","last_undegraded":"2024-04-21T09:08:13.634168+0000","last_fullsized":"2024-04-21T09:08:13.634168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:54:53.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:06:40.177313+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:06:40.176790+0000","last_became_peered":"2024-04-21T09:06:40.176790+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:04:50.337907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150830+0000","last_change":"2024-04-21T09:05:45.963682+0000","last_active":"2024-04-21T09:08:12.150830+0000","last_peered":"2024-04-21T09:08:12.150830+0000","last_clean":"2024-04-21T09:08:12.150830+0000","last_became_active":"2024-04-21T09:05:45.963392+0000","last_became_peered":"2024-04-21T09:05:45.963392+0000","last_unstale":"2024-04-21T09:08:12.150830+0000","last_undegraded":"2024-04-21T09:08:12.150830+0000","last_fullsized":"2024-04-21T09:08:12.150830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:09:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325080+0000","last_change":"2024-04-21T09:06:40.175441+0000","last_active":"2024-04-21T09:08:12.325080+0000","last_peered":"2024-04-21T09:08:12.325080+0000","last_clean":"2024-04-21T09:08:12.325080+0000","last_became_active":"2024-04-21T09:06:40.175048+0000","last_became_peered":"2024-04-21T09:06:40.175048+0000","last_unstale":"2024-04-21T09:08:12.325080+0000","last_undegraded":"2024-04-21T09:08:12.325080+0000","last_fullsized":"2024-04-21T09:08:12.325080+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:29:42.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139822+0000","last_change":"2024-04-21T09:05:51.016473+0000","last_active":"2024-04-21T09:08:12.139822+0000","last_peered":"2024-04-21T09:08:12.139822+0000","last_clean":"2024-04-21T09:08:12.139822+0000","last_became_active":"2024-04-21T09:05:51.015934+0000","last_became_peered":"2024-04-21T09:05:51.015934+0000","last_unstale":"2024-04-21T09:08:12.139822+0000","last_undegraded":"2024-04-21T09:08:12.139822+0000","last_fullsized":"2024-04-21T09:08:12.139822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:44.046457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":337,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.320818+0000","last_change":"2024-04-21T09:05:45.966935+0000","last_active":"2024-04-21T09:08:17.320818+0000","last_peered":"2024-04-21T09:08:17.320818+0000","last_clean":"2024-04-21T09:08:17.320818+0000","last_became_active":"2024-04-21T09:05:45.966591+0000","last_became_peered":"2024-04-21T09:05:45.966591+0000","last_unstale":"2024-04-21T09:08:17.320818+0000","last_undegraded":"2024-04-21T09:08:17.320818+0000","last_fullsized":"2024-04-21T09:08:17.320818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:24.329581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":262,"num_read_kb":198,"num_write":40,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142319+0000","last_change":"2024-04-21T09:05:46.982108+0000","last_active":"2024-04-21T09:08:12.142319+0000","last_peered":"2024-04-21T09:08:12.142319+0000","last_clean":"2024-04-21T09:08:12.142319+0000","last_became_active":"2024-04-21T09:05:46.981779+0000","last_became_peered":"2024-04-21T09:05:46.981779+0000","last_unstale":"2024-04-21T09:08:12.142319+0000","last_undegraded":"2024-04-21T09:08:12.142319+0000","last_fullsized":"2024-04-21T09:08:12.142319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:58:42.813430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.151042+0000","last_change":"2024-04-21T09:05:51.024959+0000","last_active":"2024-04-21T09:08:12.151042+0000","last_peered":"2024-04-21T09:08:12.151042+0000","last_clean":"2024-04-21T09:08:12.151042+0000","last_became_active":"2024-04-21T09:05:51.024248+0000","last_became_peered":"2024-04-21T09:05:51.024248+0000","last_unstale":"2024-04-21T09:08:12.151042+0000","last_undegraded":"2024-04-21T09:08:12.151042+0000","last_fullsized":"2024-04-21T09:08:12.151042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:20.282197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035051+0000","last_change":"2024-04-21T09:05:45.973577+0000","last_active":"2024-04-21T09:08:12.035051+0000","last_peered":"2024-04-21T09:08:12.035051+0000","last_clean":"2024-04-21T09:08:12.035051+0000","last_became_active":"2024-04-21T09:05:45.973270+0000","last_became_peered":"2024-04-21T09:05:45.973270+0000","last_unstale":"2024-04-21T09:08:12.035051+0000","last_undegraded":"2024-04-21T09:08:12.035051+0000","last_fullsized":"2024-04-21T09:08:12.035051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:50:49.298489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033721+0000","last_change":"2024-04-21T09:05:46.983152+0000","last_active":"2024-04-21T09:08:12.033721+0000","last_peered":"2024-04-21T09:08:12.033721+0000","last_clean":"2024-04-21T09:08:12.033721+0000","last_became_active":"2024-04-21T09:05:46.982606+0000","last_became_peered":"2024-04-21T09:05:46.982606+0000","last_unstale":"2024-04-21T09:08:12.033721+0000","last_undegraded":"2024-04-21T09:08:12.033721+0000","last_fullsized":"2024-04-21T09:08:12.033721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:10:28.448489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:51.023220+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:51.022879+0000","last_became_peered":"2024-04-21T09:05:51.022879+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:09:11.349643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150756+0000","last_change":"2024-04-21T09:05:45.968975+0000","last_active":"2024-04-21T09:08:12.150756+0000","last_peered":"2024-04-21T09:08:12.150756+0000","last_clean":"2024-04-21T09:08:12.150756+0000","last_became_active":"2024-04-21T09:05:45.968609+0000","last_became_peered":"2024-04-21T09:05:45.968609+0000","last_unstale":"2024-04-21T09:08:12.150756+0000","last_undegraded":"2024-04-21T09:08:12.150756+0000","last_fullsized":"2024-04-21T09:08:12.150756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:06:23.634148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325172+0000","last_change":"2024-04-21T09:05:46.988552+0000","last_active":"2024-04-21T09:08:12.325172+0000","last_peered":"2024-04-21T09:08:12.325172+0000","last_clean":"2024-04-21T09:08:12.325172+0000","last_became_active":"2024-04-21T09:05:46.988238+0000","last_became_peered":"2024-04-21T09:05:46.988238+0000","last_unstale":"2024-04-21T09:08:12.325172+0000","last_undegraded":"2024-04-21T09:08:12.325172+0000","last_fullsized":"2024-04-21T09:08:12.325172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:23.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140230+0000","last_change":"2024-04-21T09:05:51.018240+0000","last_active":"2024-04-21T09:08:12.140230+0000","last_peered":"2024-04-21T09:08:12.140230+0000","last_clean":"2024-04-21T09:08:12.140230+0000","last_became_active":"2024-04-21T09:05:51.017841+0000","last_became_peered":"2024-04-21T09:05:51.017841+0000","last_unstale":"2024-04-21T09:08:12.140230+0000","last_undegraded":"2024-04-21T09:08:12.140230+0000","last_fullsized":"2024-04-21T09:08:12.140230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:23.161196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'5","reported_seq":170,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.286167+0000","last_change":"2024-04-21T09:05:45.968442+0000","last_active":"2024-04-21T09:08:16.286167+0000","last_peered":"2024-04-21T09:08:16.286167+0000","last_clean":"2024-04-21T09:08:16.286167+0000","last_became_active":"2024-04-21T09:05:45.968044+0000","last_became_peered":"2024-04-21T09:05:45.968044+0000","last_unstale":"2024-04-21T09:08:16.286167+0000","last_undegraded":"2024-04-21T09:08:16.286167+0000","last_fullsized":"2024-04-21T09:08:16.286167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:22.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324692+0000","last_change":"2024-04-21T09:05:46.977975+0000","last_active":"2024-04-21T09:08:12.324692+0000","last_peered":"2024-04-21T09:08:12.324692+0000","last_clean":"2024-04-21T09:08:12.324692+0000","last_became_active":"2024-04-21T09:05:46.977604+0000","last_became_peered":"2024-04-21T09:05:46.977604+0000","last_unstale":"2024-04-21T09:08:12.324692+0000","last_undegraded":"2024-04-21T09:08:12.324692+0000","last_fullsized":"2024-04-21T09:08:12.324692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:47.822848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033898+0000","last_change":"2024-04-21T09:05:51.026224+0000","last_active":"2024-04-21T09:08:12.033898+0000","last_peered":"2024-04-21T09:08:12.033898+0000","last_clean":"2024-04-21T09:08:12.033898+0000","last_became_active":"2024-04-21T09:05:51.025791+0000","last_became_peered":"2024-04-21T09:05:51.025791+0000","last_unstale":"2024-04-21T09:08:12.033898+0000","last_undegraded":"2024-04-21T09:08:12.033898+0000","last_fullsized":"2024-04-21T09:08:12.033898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:59:43.792158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142080+0000","last_change":"2024-04-21T09:05:45.967320+0000","last_active":"2024-04-21T09:08:12.142080+0000","last_peered":"2024-04-21T09:08:12.142080+0000","last_clean":"2024-04-21T09:08:12.142080+0000","last_became_active":"2024-04-21T09:05:45.966661+0000","last_became_peered":"2024-04-21T09:05:45.966661+0000","last_unstale":"2024-04-21T09:08:12.142080+0000","last_undegraded":"2024-04-21T09:08:12.142080+0000","last_fullsized":"2024-04-21T09:08:12.142080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:41.002066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150555+0000","last_change":"2024-04-21T09:05:46.980454+0000","last_active":"2024-04-21T09:08:12.150555+0000","last_peered":"2024-04-21T09:08:12.150555+0000","last_clean":"2024-04-21T09:08:12.150555+0000","last_became_active":"2024-04-21T09:05:46.980121+0000","last_became_peered":"2024-04-21T09:05:46.980121+0000","last_unstale":"2024-04-21T09:08:12.150555+0000","last_undegraded":"2024-04-21T09:08:12.150555+0000","last_fullsized":"2024-04-21T09:08:12.150555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:59:57.699370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034523+0000","last_change":"2024-04-21T09:06:40.716698+0000","last_active":"2024-04-21T09:08:12.034523+0000","last_peered":"2024-04-21T09:08:12.034523+0000","last_clean":"2024-04-21T09:08:12.034523+0000","last_became_active":"2024-04-21T09:06:40.716236+0000","last_became_peered":"2024-04-21T09:06:40.716236+0000","last_unstale":"2024-04-21T09:08:12.034523+0000","last_undegraded":"2024-04-21T09:08:12.034523+0000","last_fullsized":"2024-04-21T09:08:12.034523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:01:29.272111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763981+0000","last_change":"2024-04-21T09:06:40.186339+0000","last_active":"2024-04-21T09:08:12.763981+0000","last_peered":"2024-04-21T09:08:12.763981+0000","last_clean":"2024-04-21T09:08:12.763981+0000","last_became_active":"2024-04-21T09:06:40.175783+0000","last_became_peered":"2024-04-21T09:06:40.175783+0000","last_unstale":"2024-04-21T09:08:12.763981+0000","last_undegraded":"2024-04-21T09:08:12.763981+0000","last_fullsized":"2024-04-21T09:08:12.763981+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:39:42.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763964+0000","last_change":"2024-04-21T09:05:46.988602+0000","last_active":"2024-04-21T09:08:12.763964+0000","last_peered":"2024-04-21T09:08:12.763964+0000","last_clean":"2024-04-21T09:08:12.763964+0000","last_became_active":"2024-04-21T09:05:46.988227+0000","last_became_peered":"2024-04-21T09:05:46.988227+0000","last_unstale":"2024-04-21T09:08:12.763964+0000","last_undegraded":"2024-04-21T09:08:12.763964+0000","last_fullsized":"2024-04-21T09:08:12.763964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:42:24.130926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142163+0000","last_change":"2024-04-21T09:05:51.414771+0000","last_active":"2024-04-21T09:08:12.142163+0000","last_peered":"2024-04-21T09:08:12.142163+0000","last_clean":"2024-04-21T09:08:12.142163+0000","last_became_active":"2024-04-21T09:05:51.414338+0000","last_became_peered":"2024-04-21T09:05:51.414338+0000","last_unstale":"2024-04-21T09:08:12.142163+0000","last_undegraded":"2024-04-21T09:08:12.142163+0000","last_fullsized":"2024-04-21T09:08:12.142163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:20:19.178992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.139908+0000","last_change":"2024-04-21T09:05:45.968671+0000","last_active":"2024-04-21T09:08:12.139908+0000","last_peered":"2024-04-21T09:08:12.139908+0000","last_clean":"2024-04-21T09:08:12.139908+0000","last_became_active":"2024-04-21T09:05:45.968349+0000","last_became_peered":"2024-04-21T09:05:45.968349+0000","last_unstale":"2024-04-21T09:08:12.139908+0000","last_undegraded":"2024-04-21T09:08:12.139908+0000","last_fullsized":"2024-04-21T09:08:12.139908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:27:06.648143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034862+0000","last_change":"2024-04-21T09:05:46.970721+0000","last_active":"2024-04-21T09:08:12.034862+0000","last_peered":"2024-04-21T09:08:12.034862+0000","last_clean":"2024-04-21T09:08:12.034862+0000","last_became_active":"2024-04-21T09:05:46.970234+0000","last_became_peered":"2024-04-21T09:05:46.970234+0000","last_unstale":"2024-04-21T09:08:12.034862+0000","last_undegraded":"2024-04-21T09:08:12.034862+0000","last_fullsized":"2024-04-21T09:08:12.034862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:19.371647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033827+0000","last_change":"2024-04-21T09:05:51.022413+0000","last_active":"2024-04-21T09:08:12.033827+0000","last_peered":"2024-04-21T09:08:12.033827+0000","last_clean":"2024-04-21T09:08:12.033827+0000","last_became_active":"2024-04-21T09:05:51.021941+0000","last_became_peered":"2024-04-21T09:05:51.021941+0000","last_unstale":"2024-04-21T09:08:12.033827+0000","last_undegraded":"2024-04-21T09:08:12.033827+0000","last_fullsized":"2024-04-21T09:08:12.033827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:25.215442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325076+0000","last_change":"2024-04-21T09:05:45.972968+0000","last_active":"2024-04-21T09:08:12.325076+0000","last_peered":"2024-04-21T09:08:12.325076+0000","last_clean":"2024-04-21T09:08:12.325076+0000","last_became_active":"2024-04-21T09:05:45.972412+0000","last_became_peered":"2024-04-21T09:05:45.972412+0000","last_unstale":"2024-04-21T09:08:12.325076+0000","last_undegraded":"2024-04-21T09:08:12.325076+0000","last_fullsized":"2024-04-21T09:08:12.325076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:30:18.924578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034855+0000","last_change":"2024-04-21T09:05:46.987030+0000","last_active":"2024-04-21T09:08:12.034855+0000","last_peered":"2024-04-21T09:08:12.034855+0000","last_clean":"2024-04-21T09:08:12.034855+0000","last_became_active":"2024-04-21T09:05:46.986681+0000","last_became_peered":"2024-04-21T09:05:46.986681+0000","last_unstale":"2024-04-21T09:08:12.034855+0000","last_undegraded":"2024-04-21T09:08:12.034855+0000","last_fullsized":"2024-04-21T09:08:12.034855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:36.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033559+0000","last_change":"2024-04-21T09:05:51.415493+0000","last_active":"2024-04-21T09:08:12.033559+0000","last_peered":"2024-04-21T09:08:12.033559+0000","last_clean":"2024-04-21T09:08:12.033559+0000","last_became_active":"2024-04-21T09:05:51.414390+0000","last_became_peered":"2024-04-21T09:05:51.414390+0000","last_unstale":"2024-04-21T09:08:12.033559+0000","last_undegraded":"2024-04-21T09:08:12.033559+0000","last_fullsized":"2024-04-21T09:08:12.033559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:42.450683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150754+0000","last_change":"2024-04-21T09:05:46.980874+0000","last_active":"2024-04-21T09:08:12.150754+0000","last_peered":"2024-04-21T09:08:12.150754+0000","last_clean":"2024-04-21T09:08:12.150754+0000","last_became_active":"2024-04-21T09:05:46.978517+0000","last_became_peered":"2024-04-21T09:05:46.978517+0000","last_unstale":"2024-04-21T09:08:12.150754+0000","last_undegraded":"2024-04-21T09:08:12.150754+0000","last_fullsized":"2024-04-21T09:08:12.150754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:26.641999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:16.285785+0000","last_change":"2024-04-21T09:05:45.965338+0000","last_active":"2024-04-21T09:08:16.285785+0000","last_peered":"2024-04-21T09:08:16.285785+0000","last_clean":"2024-04-21T09:08:16.285785+0000","last_became_active":"2024-04-21T09:05:45.965027+0000","last_became_peered":"2024-04-21T09:05:45.965027+0000","last_unstale":"2024-04-21T09:08:16.285785+0000","last_undegraded":"2024-04-21T09:08:16.285785+0000","last_fullsized":"2024-04-21T09:08:16.285785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:03.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325490+0000","last_change":"2024-04-21T09:05:51.011078+0000","last_active":"2024-04-21T09:08:12.325490+0000","last_peered":"2024-04-21T09:08:12.325490+0000","last_clean":"2024-04-21T09:08:12.325490+0000","last_became_active":"2024-04-21T09:05:51.010599+0000","last_became_peered":"2024-04-21T09:05:51.010599+0000","last_unstale":"2024-04-21T09:08:12.325490+0000","last_undegraded":"2024-04-21T09:08:12.325490+0000","last_fullsized":"2024-04-21T09:08:12.325490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:46:31.575288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142918+0000","last_change":"2024-04-21T09:05:46.978279+0000","last_active":"2024-04-21T09:08:12.142918+0000","last_peered":"2024-04-21T09:08:12.142918+0000","last_clean":"2024-04-21T09:08:12.142918+0000","last_became_active":"2024-04-21T09:05:46.975864+0000","last_became_peered":"2024-04-21T09:05:46.975864+0000","last_unstale":"2024-04-21T09:08:12.142918+0000","last_undegraded":"2024-04-21T09:08:12.142918+0000","last_fullsized":"2024-04-21T09:08:12.142918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:50:04.007431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.280892+0000","last_change":"2024-04-21T09:05:45.966990+0000","last_active":"2024-04-21T09:08:12.280892+0000","last_peered":"2024-04-21T09:08:12.280892+0000","last_clean":"2024-04-21T09:08:12.280892+0000","last_became_active":"2024-04-21T09:05:45.966650+0000","last_became_peered":"2024-04-21T09:05:45.966650+0000","last_unstale":"2024-04-21T09:08:12.280892+0000","last_undegraded":"2024-04-21T09:08:12.280892+0000","last_fullsized":"2024-04-21T09:08:12.280892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:56:19.365231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"96'148","reported_seq":413,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:14.175860+0000","last_change":"2024-04-21T09:04:33.636846+0000","last_active":"2024-04-21T09:08:14.175860+0000","last_peered":"2024-04-21T09:08:14.175860+0000","last_clean":"2024-04-21T09:08:14.175860+0000","last_became_active":"2024-04-21T09:04:33.636400+0000","last_became_peered":"2024-04-21T09:04:33.636400+0000","last_unstale":"2024-04-21T09:08:14.175860+0000","last_undegraded":"2024-04-21T09:08:14.175860+0000","last_fullsized":"2024-04-21T09:08:14.175860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:01:06.526568+0000","last_clean_scrub_stamp":"2024-04-21T09:01:06.526568+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:06.777153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150742+0000","last_change":"2024-04-21T09:06:40.176390+0000","last_active":"2024-04-21T09:08:12.150742+0000","last_peered":"2024-04-21T09:08:12.150742+0000","last_clean":"2024-04-21T09:08:12.150742+0000","last_became_active":"2024-04-21T09:06:40.175858+0000","last_became_peered":"2024-04-21T09:06:40.175858+0000","last_unstale":"2024-04-21T09:08:12.150742+0000","last_undegraded":"2024-04-21T09:08:12.150742+0000","last_fullsized":"2024-04-21T09:08:12.150742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:53:25.571305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033204+0000","last_change":"2024-04-21T09:06:40.718126+0000","last_active":"2024-04-21T09:08:12.033204+0000","last_peered":"2024-04-21T09:08:12.033204+0000","last_clean":"2024-04-21T09:08:12.033204+0000","last_became_active":"2024-04-21T09:06:40.717694+0000","last_became_peered":"2024-04-21T09:06:40.717694+0000","last_unstale":"2024-04-21T09:08:12.033204+0000","last_undegraded":"2024-04-21T09:08:12.033204+0000","last_fullsized":"2024-04-21T09:08:12.033204+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:27:43.766253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324532+0000","last_change":"2024-04-21T09:05:45.969943+0000","last_active":"2024-04-21T09:08:12.324532+0000","last_peered":"2024-04-21T09:08:12.324532+0000","last_clean":"2024-04-21T09:08:12.324532+0000","last_became_active":"2024-04-21T09:05:45.969606+0000","last_became_peered":"2024-04-21T09:05:45.969606+0000","last_unstale":"2024-04-21T09:08:12.324532+0000","last_undegraded":"2024-04-21T09:08:12.324532+0000","last_fullsized":"2024-04-21T09:08:12.324532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:41.999829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763722+0000","last_change":"2024-04-21T09:05:51.024489+0000","last_active":"2024-04-21T09:08:12.763722+0000","last_peered":"2024-04-21T09:08:12.763722+0000","last_clean":"2024-04-21T09:08:12.763722+0000","last_became_active":"2024-04-21T09:05:51.024035+0000","last_became_peered":"2024-04-21T09:05:51.024035+0000","last_unstale":"2024-04-21T09:08:12.763722+0000","last_undegraded":"2024-04-21T09:08:12.763722+0000","last_fullsized":"2024-04-21T09:08:12.763722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:14.556008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142352+0000","last_change":"2024-04-21T09:05:45.967198+0000","last_active":"2024-04-21T09:08:12.142352+0000","last_peered":"2024-04-21T09:08:12.142352+0000","last_clean":"2024-04-21T09:08:12.142352+0000","last_became_active":"2024-04-21T09:05:45.966683+0000","last_became_peered":"2024-04-21T09:05:45.966683+0000","last_unstale":"2024-04-21T09:08:12.142352+0000","last_undegraded":"2024-04-21T09:08:12.142352+0000","last_fullsized":"2024-04-21T09:08:12.142352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:48:23.028601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033592+0000","last_change":"2024-04-21T09:05:46.962169+0000","last_active":"2024-04-21T09:08:12.033592+0000","last_peered":"2024-04-21T09:08:12.033592+0000","last_clean":"2024-04-21T09:08:12.033592+0000","last_became_active":"2024-04-21T09:05:46.961817+0000","last_became_peered":"2024-04-21T09:05:46.961817+0000","last_unstale":"2024-04-21T09:08:12.033592+0000","last_undegraded":"2024-04-21T09:08:12.033592+0000","last_fullsized":"2024-04-21T09:08:12.033592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:10.383333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.142660+0000","last_change":"2024-04-21T09:05:51.414679+0000","last_active":"2024-04-21T09:08:12.142660+0000","last_peered":"2024-04-21T09:08:12.142660+0000","last_clean":"2024-04-21T09:08:12.142660+0000","last_became_active":"2024-04-21T09:05:51.414280+0000","last_became_peered":"2024-04-21T09:05:51.414280+0000","last_unstale":"2024-04-21T09:08:12.142660+0000","last_undegraded":"2024-04-21T09:08:12.142660+0000","last_fullsized":"2024-04-21T09:08:12.142660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:05:46.844578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'70","reported_seq":159,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.998429+0000","last_change":"2024-04-21T09:05:45.974036+0000","last_active":"2024-04-21T09:08:17.998429+0000","last_peered":"2024-04-21T09:08:17.998429+0000","last_clean":"2024-04-21T09:08:17.998429+0000","last_became_active":"2024-04-21T09:05:45.973681+0000","last_became_peered":"2024-04-21T09:05:45.973681+0000","last_unstale":"2024-04-21T09:08:17.998429+0000","last_undegraded":"2024-04-21T09:08:17.998429+0000","last_fullsized":"2024-04-21T09:08:17.998429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:12.061669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":125,"num_write_kb":1157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143688+0000","last_change":"2024-04-21T09:05:46.984443+0000","last_active":"2024-04-21T09:08:12.143688+0000","last_peered":"2024-04-21T09:08:12.143688+0000","last_clean":"2024-04-21T09:08:12.143688+0000","last_became_active":"2024-04-21T09:05:46.980916+0000","last_became_peered":"2024-04-21T09:05:46.980916+0000","last_unstale":"2024-04-21T09:08:12.143688+0000","last_undegraded":"2024-04-21T09:08:12.143688+0000","last_fullsized":"2024-04-21T09:08:12.143688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:46.993070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034657+0000","last_change":"2024-04-21T09:05:51.007837+0000","last_active":"2024-04-21T09:08:12.034657+0000","last_peered":"2024-04-21T09:08:12.034657+0000","last_clean":"2024-04-21T09:08:12.034657+0000","last_became_active":"2024-04-21T09:05:51.007428+0000","last_became_peered":"2024-04-21T09:05:51.007428+0000","last_unstale":"2024-04-21T09:08:12.034657+0000","last_undegraded":"2024-04-21T09:08:12.034657+0000","last_fullsized":"2024-04-21T09:08:12.034657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:14:32.956971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281282+0000","last_change":"2024-04-21T09:05:45.977266+0000","last_active":"2024-04-21T09:08:12.281282+0000","last_peered":"2024-04-21T09:08:12.281282+0000","last_clean":"2024-04-21T09:08:12.281282+0000","last_became_active":"2024-04-21T09:05:45.976867+0000","last_became_peered":"2024-04-21T09:05:45.976867+0000","last_unstale":"2024-04-21T09:08:12.281282+0000","last_undegraded":"2024-04-21T09:08:12.281282+0000","last_fullsized":"2024-04-21T09:08:12.281282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:08:14.390355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633978+0000","last_change":"2024-04-21T09:05:46.987011+0000","last_active":"2024-04-21T09:08:13.633978+0000","last_peered":"2024-04-21T09:08:13.633978+0000","last_clean":"2024-04-21T09:08:13.633978+0000","last_became_active":"2024-04-21T09:05:46.986639+0000","last_became_peered":"2024-04-21T09:05:46.986639+0000","last_unstale":"2024-04-21T09:08:13.633978+0000","last_undegraded":"2024-04-21T09:08:13.633978+0000","last_fullsized":"2024-04-21T09:08:13.633978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:43:34.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281130+0000","last_change":"2024-04-21T09:05:51.414999+0000","last_active":"2024-04-21T09:08:12.281130+0000","last_peered":"2024-04-21T09:08:12.281130+0000","last_clean":"2024-04-21T09:08:12.281130+0000","last_became_active":"2024-04-21T09:05:51.414543+0000","last_became_peered":"2024-04-21T09:05:51.414543+0000","last_unstale":"2024-04-21T09:08:12.281130+0000","last_undegraded":"2024-04-21T09:08:12.281130+0000","last_fullsized":"2024-04-21T09:08:12.281130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:54.503608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.323960+0000","last_change":"2024-04-21T09:05:45.967358+0000","last_active":"2024-04-21T09:08:12.323960+0000","last_peered":"2024-04-21T09:08:12.323960+0000","last_clean":"2024-04-21T09:08:12.323960+0000","last_became_active":"2024-04-21T09:05:45.966941+0000","last_became_peered":"2024-04-21T09:05:45.966941+0000","last_unstale":"2024-04-21T09:08:12.323960+0000","last_undegraded":"2024-04-21T09:08:12.323960+0000","last_fullsized":"2024-04-21T09:08:12.323960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:26.129927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.141989+0000","last_change":"2024-04-21T09:05:46.973284+0000","last_active":"2024-04-21T09:08:12.141989+0000","last_peered":"2024-04-21T09:08:12.141989+0000","last_clean":"2024-04-21T09:08:12.141989+0000","last_became_active":"2024-04-21T09:05:46.972867+0000","last_became_peered":"2024-04-21T09:05:46.972867+0000","last_unstale":"2024-04-21T09:08:12.141989+0000","last_undegraded":"2024-04-21T09:08:12.141989+0000","last_fullsized":"2024-04-21T09:08:12.141989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:50:16.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.140305+0000","last_change":"2024-04-21T09:05:51.018412+0000","last_active":"2024-04-21T09:08:12.140305+0000","last_peered":"2024-04-21T09:08:12.140305+0000","last_clean":"2024-04-21T09:08:12.140305+0000","last_became_active":"2024-04-21T09:05:51.018026+0000","last_became_peered":"2024-04-21T09:05:51.018026+0000","last_unstale":"2024-04-21T09:08:12.140305+0000","last_undegraded":"2024-04-21T09:08:12.140305+0000","last_fullsized":"2024-04-21T09:08:12.140305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:48.319781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143035+0000","last_change":"2024-04-21T09:05:45.972157+0000","last_active":"2024-04-21T09:08:12.143035+0000","last_peered":"2024-04-21T09:08:12.143035+0000","last_clean":"2024-04-21T09:08:12.143035+0000","last_became_active":"2024-04-21T09:05:45.971838+0000","last_became_peered":"2024-04-21T09:05:45.971838+0000","last_unstale":"2024-04-21T09:08:12.143035+0000","last_undegraded":"2024-04-21T09:08:12.143035+0000","last_fullsized":"2024-04-21T09:08:12.143035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:11.949137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763571+0000","last_change":"2024-04-21T09:05:46.968711+0000","last_active":"2024-04-21T09:08:12.763571+0000","last_peered":"2024-04-21T09:08:12.763571+0000","last_clean":"2024-04-21T09:08:12.763571+0000","last_became_active":"2024-04-21T09:05:46.968278+0000","last_became_peered":"2024-04-21T09:05:46.968278+0000","last_unstale":"2024-04-21T09:08:12.763571+0000","last_undegraded":"2024-04-21T09:08:12.763571+0000","last_fullsized":"2024-04-21T09:08:12.763571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:11.970720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281011+0000","last_change":"2024-04-21T09:05:51.012744+0000","last_active":"2024-04-21T09:08:12.281011+0000","last_peered":"2024-04-21T09:08:12.281011+0000","last_clean":"2024-04-21T09:08:12.281011+0000","last_became_active":"2024-04-21T09:05:51.012116+0000","last_became_peered":"2024-04-21T09:05:51.012116+0000","last_unstale":"2024-04-21T09:08:12.281011+0000","last_undegraded":"2024-04-21T09:08:12.281011+0000","last_fullsized":"2024-04-21T09:08:12.281011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:30.243892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034866+0000","last_change":"2024-04-21T09:05:45.973628+0000","last_active":"2024-04-21T09:08:12.034866+0000","last_peered":"2024-04-21T09:08:12.034866+0000","last_clean":"2024-04-21T09:08:12.034866+0000","last_became_active":"2024-04-21T09:05:45.973244+0000","last_became_peered":"2024-04-21T09:05:45.973244+0000","last_unstale":"2024-04-21T09:08:12.034866+0000","last_undegraded":"2024-04-21T09:08:12.034866+0000","last_fullsized":"2024-04-21T09:08:12.034866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:29.082190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143614+0000","last_change":"2024-04-21T09:05:46.977887+0000","last_active":"2024-04-21T09:08:12.143614+0000","last_peered":"2024-04-21T09:08:12.143614+0000","last_clean":"2024-04-21T09:08:12.143614+0000","last_became_active":"2024-04-21T09:05:46.975863+0000","last_became_peered":"2024-04-21T09:05:46.975863+0000","last_unstale":"2024-04-21T09:08:12.143614+0000","last_undegraded":"2024-04-21T09:08:12.143614+0000","last_fullsized":"2024-04-21T09:08:12.143614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:58.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.281038+0000","last_change":"2024-04-21T09:05:51.414929+0000","last_active":"2024-04-21T09:08:12.281038+0000","last_peered":"2024-04-21T09:08:12.281038+0000","last_clean":"2024-04-21T09:08:12.281038+0000","last_became_active":"2024-04-21T09:05:51.414412+0000","last_became_peered":"2024-04-21T09:05:51.414412+0000","last_unstale":"2024-04-21T09:08:12.281038+0000","last_undegraded":"2024-04-21T09:08:12.281038+0000","last_fullsized":"2024-04-21T09:08:12.281038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:01.602493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763696+0000","last_change":"2024-04-21T09:06:40.182917+0000","last_active":"2024-04-21T09:08:12.763696+0000","last_peered":"2024-04-21T09:08:12.763696+0000","last_clean":"2024-04-21T09:08:12.763696+0000","last_became_active":"2024-04-21T09:06:40.175183+0000","last_became_peered":"2024-04-21T09:06:40.175183+0000","last_unstale":"2024-04-21T09:08:12.763696+0000","last_undegraded":"2024-04-21T09:08:12.763696+0000","last_fullsized":"2024-04-21T09:08:12.763696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:43:59.268560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325194+0000","last_change":"2024-04-21T09:05:46.977632+0000","last_active":"2024-04-21T09:08:12.325194+0000","last_peered":"2024-04-21T09:08:12.325194+0000","last_clean":"2024-04-21T09:08:12.325194+0000","last_became_active":"2024-04-21T09:05:46.977283+0000","last_became_peered":"2024-04-21T09:05:46.977283+0000","last_unstale":"2024-04-21T09:08:12.325194+0000","last_undegraded":"2024-04-21T09:08:12.325194+0000","last_fullsized":"2024-04-21T09:08:12.325194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:39.753237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324153+0000","last_change":"2024-04-21T09:05:51.021868+0000","last_active":"2024-04-21T09:08:12.324153+0000","last_peered":"2024-04-21T09:08:12.324153+0000","last_clean":"2024-04-21T09:08:12.324153+0000","last_became_active":"2024-04-21T09:05:51.021421+0000","last_became_peered":"2024-04-21T09:05:51.021421+0000","last_unstale":"2024-04-21T09:08:12.324153+0000","last_undegraded":"2024-04-21T09:08:12.324153+0000","last_fullsized":"2024-04-21T09:08:12.324153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:51.036726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324834+0000","last_change":"2024-04-21T09:06:40.181015+0000","last_active":"2024-04-21T09:08:12.324834+0000","last_peered":"2024-04-21T09:08:12.324834+0000","last_clean":"2024-04-21T09:08:12.324834+0000","last_became_active":"2024-04-21T09:06:40.175517+0000","last_became_peered":"2024-04-21T09:06:40.175517+0000","last_unstale":"2024-04-21T09:08:12.324834+0000","last_undegraded":"2024-04-21T09:08:12.324834+0000","last_fullsized":"2024-04-21T09:08:12.324834+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:21:14.420277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325898+0000","last_change":"2024-04-21T09:05:46.979622+0000","last_active":"2024-04-21T09:08:12.325898+0000","last_peered":"2024-04-21T09:08:12.325898+0000","last_clean":"2024-04-21T09:08:12.325898+0000","last_became_active":"2024-04-21T09:05:46.977601+0000","last_became_peered":"2024-04-21T09:05:46.977601+0000","last_unstale":"2024-04-21T09:08:12.325898+0000","last_undegraded":"2024-04-21T09:08:12.325898+0000","last_fullsized":"2024-04-21T09:08:12.325898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:12:15.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633710+0000","last_change":"2024-04-21T09:05:51.012262+0000","last_active":"2024-04-21T09:08:13.633710+0000","last_peered":"2024-04-21T09:08:13.633710+0000","last_clean":"2024-04-21T09:08:13.633710+0000","last_became_active":"2024-04-21T09:05:51.011814+0000","last_became_peered":"2024-04-21T09:05:51.011814+0000","last_unstale":"2024-04-21T09:08:13.633710+0000","last_undegraded":"2024-04-21T09:08:13.633710+0000","last_fullsized":"2024-04-21T09:08:13.633710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:48.491981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"96'57","reported_seq":111,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:17.993708+0000","last_change":"2024-04-21T09:05:45.962116+0000","last_active":"2024-04-21T09:08:17.993708+0000","last_peered":"2024-04-21T09:08:17.993708+0000","last_clean":"2024-04-21T09:08:17.993708+0000","last_became_active":"2024-04-21T09:05:45.961786+0000","last_became_peered":"2024-04-21T09:05:45.961786+0000","last_unstale":"2024-04-21T09:08:17.993708+0000","last_undegraded":"2024-04-21T09:08:17.993708+0000","last_fullsized":"2024-04-21T09:08:17.993708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:46.993814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324272+0000","last_change":"2024-04-21T09:05:46.981804+0000","last_active":"2024-04-21T09:08:12.324272+0000","last_peered":"2024-04-21T09:08:12.324272+0000","last_clean":"2024-04-21T09:08:12.324272+0000","last_became_active":"2024-04-21T09:05:46.974461+0000","last_became_peered":"2024-04-21T09:05:46.974461+0000","last_unstale":"2024-04-21T09:08:12.324272+0000","last_undegraded":"2024-04-21T09:08:12.324272+0000","last_fullsized":"2024-04-21T09:08:12.324272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:34.579377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150326+0000","last_change":"2024-04-21T09:05:51.010117+0000","last_active":"2024-04-21T09:08:12.150326+0000","last_peered":"2024-04-21T09:08:12.150326+0000","last_clean":"2024-04-21T09:08:12.150326+0000","last_became_active":"2024-04-21T09:05:51.009469+0000","last_became_peered":"2024-04-21T09:05:51.009469+0000","last_unstale":"2024-04-21T09:08:12.150326+0000","last_undegraded":"2024-04-21T09:08:12.150326+0000","last_fullsized":"2024-04-21T09:08:12.150326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:49.984713+0000","last_clean_scrub_stamp":"2024-04-21T09:05:49.984713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:29:22.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033814+0000","last_change":"2024-04-21T09:05:45.966608+0000","last_active":"2024-04-21T09:08:12.033814+0000","last_peered":"2024-04-21T09:08:12.033814+0000","last_clean":"2024-04-21T09:08:12.033814+0000","last_became_active":"2024-04-21T09:05:45.966279+0000","last_became_peered":"2024-04-21T09:05:45.966279+0000","last_unstale":"2024-04-21T09:08:12.033814+0000","last_undegraded":"2024-04-21T09:08:12.033814+0000","last_fullsized":"2024-04-21T09:08:12.033814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:44.936079+0000","last_clean_scrub_stamp":"2024-04-21T09:05:44.936079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:24:41.443424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.325958+0000","last_change":"2024-04-21T09:05:46.977209+0000","last_active":"2024-04-21T09:08:12.325958+0000","last_peered":"2024-04-21T09:08:12.325958+0000","last_clean":"2024-04-21T09:08:12.325958+0000","last_became_active":"2024-04-21T09:05:46.976777+0000","last_became_peered":"2024-04-21T09:05:46.976777+0000","last_unstale":"2024-04-21T09:08:12.325958+0000","last_undegraded":"2024-04-21T09:08:12.325958+0000","last_fullsized":"2024-04-21T09:08:12.325958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:20.039346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763645+0000","last_change":"2024-04-21T09:05:46.968189+0000","last_active":"2024-04-21T09:08:12.763645+0000","last_peered":"2024-04-21T09:08:12.763645+0000","last_clean":"2024-04-21T09:08:12.763645+0000","last_became_active":"2024-04-21T09:05:46.967603+0000","last_became_peered":"2024-04-21T09:05:46.967603+0000","last_unstale":"2024-04-21T09:08:12.763645+0000","last_undegraded":"2024-04-21T09:08:12.763645+0000","last_fullsized":"2024-04-21T09:08:12.763645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:03:30.562790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034801+0000","last_change":"2024-04-21T09:05:46.971615+0000","last_active":"2024-04-21T09:08:12.034801+0000","last_peered":"2024-04-21T09:08:12.034801+0000","last_clean":"2024-04-21T09:08:12.034801+0000","last_became_active":"2024-04-21T09:05:46.971121+0000","last_became_peered":"2024-04-21T09:05:46.971121+0000","last_unstale":"2024-04-21T09:08:12.034801+0000","last_undegraded":"2024-04-21T09:08:12.034801+0000","last_fullsized":"2024-04-21T09:08:12.034801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:44.312368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.034715+0000","last_change":"2024-04-21T09:05:46.967820+0000","last_active":"2024-04-21T09:08:12.034715+0000","last_peered":"2024-04-21T09:08:12.034715+0000","last_clean":"2024-04-21T09:08:12.034715+0000","last_became_active":"2024-04-21T09:05:46.967470+0000","last_became_peered":"2024-04-21T09:05:46.967470+0000","last_unstale":"2024-04-21T09:08:12.034715+0000","last_undegraded":"2024-04-21T09:08:12.034715+0000","last_fullsized":"2024-04-21T09:08:12.034715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:09:36.279081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:13.633881+0000","last_change":"2024-04-21T09:05:46.987062+0000","last_active":"2024-04-21T09:08:13.633881+0000","last_peered":"2024-04-21T09:08:13.633881+0000","last_clean":"2024-04-21T09:08:13.633881+0000","last_became_active":"2024-04-21T09:05:46.986692+0000","last_became_peered":"2024-04-21T09:05:46.986692+0000","last_unstale":"2024-04-21T09:08:13.633881+0000","last_undegraded":"2024-04-21T09:08:13.633881+0000","last_fullsized":"2024-04-21T09:08:13.633881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:20:33.172982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033916+0000","last_change":"2024-04-21T09:05:46.969152+0000","last_active":"2024-04-21T09:08:12.033916+0000","last_peered":"2024-04-21T09:08:12.033916+0000","last_clean":"2024-04-21T09:08:12.033916+0000","last_became_active":"2024-04-21T09:05:46.968776+0000","last_became_peered":"2024-04-21T09:05:46.968776+0000","last_unstale":"2024-04-21T09:08:12.033916+0000","last_undegraded":"2024-04-21T09:08:12.033916+0000","last_fullsized":"2024-04-21T09:08:12.033916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:55:01.024316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763506+0000","last_change":"2024-04-21T09:07:40.649825+0000","last_active":"2024-04-21T09:08:12.763506+0000","last_peered":"2024-04-21T09:08:12.763506+0000","last_clean":"2024-04-21T09:08:12.763506+0000","last_became_active":"2024-04-21T09:07:40.649264+0000","last_became_peered":"2024-04-21T09:07:40.649264+0000","last_unstale":"2024-04-21T09:08:12.763506+0000","last_undegraded":"2024-04-21T09:08:12.763506+0000","last_fullsized":"2024-04-21T09:08:12.763506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:38.142948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.035249+0000","last_change":"2024-04-21T09:05:46.964913+0000","last_active":"2024-04-21T09:08:12.035249+0000","last_peered":"2024-04-21T09:08:12.035249+0000","last_clean":"2024-04-21T09:08:12.035249+0000","last_became_active":"2024-04-21T09:05:46.964567+0000","last_became_peered":"2024-04-21T09:05:46.964567+0000","last_unstale":"2024-04-21T09:08:12.035249+0000","last_undegraded":"2024-04-21T09:08:12.035249+0000","last_fullsized":"2024-04-21T09:08:12.035249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:26:53.448826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143896+0000","last_change":"2024-04-21T09:05:46.985284+0000","last_active":"2024-04-21T09:08:12.143896+0000","last_peered":"2024-04-21T09:08:12.143896+0000","last_clean":"2024-04-21T09:08:12.143896+0000","last_became_active":"2024-04-21T09:05:46.984909+0000","last_became_peered":"2024-04-21T09:05:46.984909+0000","last_unstale":"2024-04-21T09:08:12.143896+0000","last_undegraded":"2024-04-21T09:08:12.143896+0000","last_fullsized":"2024-04-21T09:08:12.143896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:22:25.551773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.150636+0000","last_change":"2024-04-21T09:05:46.971987+0000","last_active":"2024-04-21T09:08:12.150636+0000","last_peered":"2024-04-21T09:08:12.150636+0000","last_clean":"2024-04-21T09:08:12.150636+0000","last_became_active":"2024-04-21T09:05:46.971643+0000","last_became_peered":"2024-04-21T09:05:46.971643+0000","last_unstale":"2024-04-21T09:08:12.150636+0000","last_undegraded":"2024-04-21T09:08:12.150636+0000","last_fullsized":"2024-04-21T09:08:12.150636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:46:04.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.143204+0000","last_change":"2024-04-21T09:05:46.977255+0000","last_active":"2024-04-21T09:08:12.143204+0000","last_peered":"2024-04-21T09:08:12.143204+0000","last_clean":"2024-04-21T09:08:12.143204+0000","last_became_active":"2024-04-21T09:05:46.976914+0000","last_became_peered":"2024-04-21T09:05:46.976914+0000","last_unstale":"2024-04-21T09:08:12.143204+0000","last_undegraded":"2024-04-21T09:08:12.143204+0000","last_fullsized":"2024-04-21T09:08:12.143204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:27.991049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.324412+0000","last_change":"2024-04-21T09:05:46.979504+0000","last_active":"2024-04-21T09:08:12.324412+0000","last_peered":"2024-04-21T09:08:12.324412+0000","last_clean":"2024-04-21T09:08:12.324412+0000","last_became_active":"2024-04-21T09:05:46.978850+0000","last_became_peered":"2024-04-21T09:05:46.978850+0000","last_unstale":"2024-04-21T09:08:12.324412+0000","last_undegraded":"2024-04-21T09:08:12.324412+0000","last_fullsized":"2024-04-21T09:08:12.324412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:42.789754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.033628+0000","last_change":"2024-04-21T09:05:46.981486+0000","last_active":"2024-04-21T09:08:12.033628+0000","last_peered":"2024-04-21T09:08:12.033628+0000","last_clean":"2024-04-21T09:08:12.033628+0000","last_became_active":"2024-04-21T09:05:46.981125+0000","last_became_peered":"2024-04-21T09:05:46.981125+0000","last_unstale":"2024-04-21T09:08:12.033628+0000","last_undegraded":"2024-04-21T09:08:12.033628+0000","last_fullsized":"2024-04-21T09:08:12.033628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:29.860926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-21T09:08:12.763507+0000","last_change":"2024-04-21T09:07:40.650222+0000","last_active":"2024-04-21T09:08:12.763507+0000","last_peered":"2024-04-21T09:08:12.763507+0000","last_clean":"2024-04-21T09:08:12.763507+0000","last_became_active":"2024-04-21T09:07:40.649595+0000","last_became_peered":"2024-04-21T09:07:40.649595+0000","last_unstale":"2024-04-21T09:08:12.763507+0000","last_undegraded":"2024-04-21T09:08:12.763507+0000","last_fullsized":"2024-04-21T09:08:12.763507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T09:05:45.941299+0000","last_clean_scrub_stamp":"2024-04-21T09:05:45.941299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:41:25.006017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107013,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793576,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":976,"num_read_kb":645,"num_write":351,"num_write_kb":2090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":638976,"data_stored":2380728,"data_compressed":107238,"data_compressed_allocated":147456,"data_compressed_original":2125824,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":541927,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32368,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712784,"statfs":{"total":95995035648,"available":95961890816,"internally_reserved":0,"allocated":1220608,"data_stored":1328900,"data_compressed":75912,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":939453,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":935255,"data_compressed":51600,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1138669,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":537317,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":605237,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":538705,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":604801,"data_compressed":34085,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1062327,"data_compressed":57756,"data_compressed_allocated":77824,"data_compressed_original":606208,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888143,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":543679,"data_compressed":32454,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":1265158,"data_compressed":74281,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35671,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68010,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67574,"data_compressed":1631,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34115,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330099,"data_compressed":14840,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":397937,"data_compressed":16471,"data_compressed_allocated":24576,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T09:08:23.453 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:23 smithi186 ceph-mon[31505]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-04-21T09:08:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:23 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2104094628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 -- 172.21.15.66:0/1790895687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 msgr2=0x7f282010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 --2- 172.21.15.66:0/1790895687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f282010ec30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2818009ca0 tx=0x7f281802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 -- 172.21.15.66:0/1790895687 shutdown_connections 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 --2- 172.21.15.66:0/1790895687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f282010ec30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 --2- 172.21.15.66:0/1790895687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2820102940 0x7f2820106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 --2- 172.21.15.66:0/1790895687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2820101f90 0x7f2820102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 -- 172.21.15.66:0/1790895687 >> 172.21.15.66:0/1790895687 conn(0x7f28200fd810 msgr2=0x7f28200ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:23.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 -- 172.21.15.66:0/1790895687 shutdown_connections 2024-04-21T09:08:23.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.590+0000 7f282659f640 1 -- 172.21.15.66:0/1790895687 wait complete. 2024-04-21T09:08:23.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 Processor -- start 2024-04-21T09:08:23.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 -- start start 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2820101f90 0x7f2820074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f2820074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2820074cd0 0x7f28201a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2820111fd0 con 0x7f2820107060 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2820111e50 con 0x7f2820101f90 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282659f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2820112150 con 0x7f2820074cd0 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f2824d9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f2820074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f2824d9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f2820074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33278/0 (socket says 172.21.15.66:33278) 2024-04-21T09:08:23.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f2824d9c640 1 -- 172.21.15.66:0/604885666 learned_addr learned my addr 172.21.15.66:0/604885666 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f282559d640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2820101f90 0x7f2820074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.591+0000 7f2825d9e640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2820074cd0 0x7f28201a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f2824d9c640 1 -- 172.21.15.66:0/604885666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2820074cd0 msgr2=0x7f28201a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f2824d9c640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2820074cd0 0x7f28201a66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f2824d9c640 1 -- 172.21.15.66:0/604885666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2820101f90 msgr2=0x7f2820074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f2824d9c640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2820101f90 0x7f2820074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f2824d9c640 1 -- 172.21.15.66:0/604885666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28201a6de0 con 0x7f2820107060 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f282559d640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2820101f90 0x7f2820074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f2824d9c640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f2820074790 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d740 tx=0x7f281c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f28167fc640 1 -- 172.21.15.66:0/604885666 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c017070 con 0x7f2820107060 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f28167fc640 1 -- 172.21.15.66:0/604885666 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281c00de50 con 0x7f2820107060 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28201a70d0 con 0x7f2820107060 2024-04-21T09:08:23.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f28167fc640 1 -- 172.21.15.66:0/604885666 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c004ce0 con 0x7f2820107060 2024-04-21T09:08:23.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.592+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28201a75d0 con 0x7f2820107060 2024-04-21T09:08:23.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.593+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28200690e0 con 0x7f2820107060 2024-04-21T09:08:23.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.594+0000 7f28167fc640 1 -- 172.21.15.66:0/604885666 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f281c004e80 con 0x7f2820107060 2024-04-21T09:08:23.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.595+0000 7f28167fc640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f280c0a3de0 0x7f280c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.595+0000 7f282559d640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f280c0a3de0 0x7f280c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.595+0000 7f28167fc640 1 -- 172.21.15.66:0/604885666 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f281c0d06f0 con 0x7f2820107060 2024-04-21T09:08:23.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.595+0000 7f282559d640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f280c0a3de0 0x7f280c0a62a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2810005fd0 tx=0x7f281000b5f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:23.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.599+0000 7f28167fc640 1 -- 172.21.15.66:0/604885666 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f281c09c8b0 con 0x7f2820107060 2024-04-21T09:08:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:23 smithi083 ceph-mon[31413]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-04-21T09:08:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:23 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2104094628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.824+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2820075780 con 0x7f2820107060 2024-04-21T09:08:23.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.825+0000 7f28167fc640 1 -- 172.21.15.66:0/604885666 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f281c09c1b0 con 0x7f2820107060 2024-04-21T09:08:23.825 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:23.825 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:08:23.826 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:08:23.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.826+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f280c0a3de0 msgr2=0x7f280c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.826+0000 7f282659f640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f280c0a3de0 0x7f280c0a62a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2810005fd0 tx=0x7f281000b5f0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.826+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 msgr2=0x7f2820074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.826+0000 7f282659f640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f2820074790 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d740 tx=0x7f281c00dc00 comp rx=0 tx=0).stop 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 shutdown_connections 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f280c0a3de0 0x7f280c0a62a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2820074cd0 0x7f28201a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820107060 0x7f2820074790 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 --2- 172.21.15.66:0/604885666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2820101f90 0x7f2820074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 >> 172.21.15.66:0/604885666 conn(0x7f28200fd810 msgr2=0x7f28200feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 shutdown_connections 2024-04-21T09:08:23.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.827+0000 7f282659f640 1 -- 172.21.15.66:0/604885666 wait complete. 2024-04-21T09:08:23.839 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:23.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1229521484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac107010 msgr2=0x7fd4ac1073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1229521484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac107010 0x7fd4ac1073f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009930 tx=0x7fd4a002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:23.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1229521484 shutdown_connections 2024-04-21T09:08:23.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1229521484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4ac101ab0 0x7fd4ac10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1229521484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4ac101110 0x7fd4ac101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1229521484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac107010 0x7fd4ac1073f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1229521484 >> 172.21.15.66:0/1229521484 conn(0x7fd4ac077e30 msgr2=0x7fd4ac078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1229521484 shutdown_connections 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.940+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1229521484 wait complete. 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 Processor -- start 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 -- start start 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4ac101110 0x7fd4ac198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac101ab0 0x7fd4ac198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4ac107010 0x7fd4ac19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac112250 con 0x7fd4ac101ab0 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1120d0 con 0x7fd4ac107010 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4b28a8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1123d0 con 0x7fd4ac101110 2024-04-21T09:08:23.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac101ab0 0x7fd4ac198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac101ab0 0x7fd4ac198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33284/0 (socket says 172.21.15.66:33284) 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.941+0000 7fd4ab7fe640 1 -- 172.21.15.66:0/1089809669 learned_addr learned my addr 172.21.15.66:0/1089809669 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4abfff640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4ac101110 0x7fd4ac198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4b0e45640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4ac107010 0x7fd4ac19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4ab7fe640 1 -- 172.21.15.66:0/1089809669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4ac101110 msgr2=0x7fd4ac198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4ab7fe640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4ac101110 0x7fd4ac198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4ab7fe640 1 -- 172.21.15.66:0/1089809669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4ac107010 msgr2=0x7fd4ac19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4ab7fe640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4ac107010 0x7fd4ac19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4ab7fe640 1 -- 172.21.15.66:0/1089809669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac19d720 con 0x7fd4ac101ab0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4b0e45640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4ac107010 0x7fd4ac19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4ab7fe640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac101ab0 0x7fd4ac198c70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c890 tx=0x7fd49c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4a97fa640 1 -- 172.21.15.66:0/1089809669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c004220 con 0x7fd4ac101ab0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4a97fa640 1 -- 172.21.15.66:0/1089809669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd49c0043c0 con 0x7fd4ac101ab0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ac19da10 con 0x7fd4ac101ab0 2024-04-21T09:08:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4a97fa640 1 -- 172.21.15.66:0/1089809669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c004a00 con 0x7fd4ac101ab0 2024-04-21T09:08:23.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.942+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ac1a52a0 con 0x7fd4ac101ab0 2024-04-21T09:08:23.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.943+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ac069230 con 0x7fd4ac101ab0 2024-04-21T09:08:23.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.944+0000 7fd4a97fa640 1 -- 172.21.15.66:0/1089809669 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fd49c0079b0 con 0x7fd4ac101ab0 2024-04-21T09:08:23.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.945+0000 7fd4a97fa640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd4940a3db0 0x7fd4940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:23.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.945+0000 7fd4abfff640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd4940a3db0 0x7fd4940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:23.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.945+0000 7fd4a97fa640 1 -- 172.21.15.66:0/1089809669 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd49c0d0b10 con 0x7fd4ac101ab0 2024-04-21T09:08:23.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.946+0000 7fd4abfff640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd4940a3db0 0x7fd4940a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0006010 tx=0x7fd4a0004120 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:23.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:23.950+0000 7fd4a97fa640 1 -- 172.21.15.66:0/1089809669 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd49c014030 con 0x7fd4ac101ab0 2024-04-21T09:08:24.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.139+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4ac109350 con 0x7fd4ac101ab0 2024-04-21T09:08:24.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.140+0000 7fd4a97fa640 1 -- 172.21.15.66:0/1089809669 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+25086 (secure 0 0 0) 0x7fd49c09ccd0 con 0x7fd4ac101ab0 2024-04-21T09:08:24.141 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:24.142 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:08:12.028415+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:24.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd4940a3db0 msgr2=0x7fd4940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd4940a3db0 0x7fd4940a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0006010 tx=0x7fd4a0004120 comp rx=0 tx=0).stop 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac101ab0 msgr2=0x7fd4ac198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac101ab0 0x7fd4ac198c70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c890 tx=0x7fd49c00cd50 comp rx=0 tx=0).stop 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 shutdown_connections 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd4940a3db0 0x7fd4940a6270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4ac107010 0x7fd4ac19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4ac101ab0 0x7fd4ac198c70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 --2- 172.21.15.66:0/1089809669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4ac101110 0x7fd4ac198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 >> 172.21.15.66:0/1089809669 conn(0x7fd4ac077e30 msgr2=0x7fd4ac0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 shutdown_connections 2024-04-21T09:08:24.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.143+0000 7fd4b28a8640 1 -- 172.21.15.66:0/1089809669 wait complete. 2024-04-21T09:08:24.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:08:24.155 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 -- 172.21.15.66:0/2105822699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101a40 msgr2=0x7fdfc0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/2105822699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101a40 0x7fdfc0105a80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009900 tx=0x7fdfb402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 -- 172.21.15.66:0/2105822699 shutdown_connections 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/2105822699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc0106160 0x7fdfc010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/2105822699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101a40 0x7fdfc0105a80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/2105822699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc0101090 0x7fdfc0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 -- 172.21.15.66:0/2105822699 >> 172.21.15.66:0/2105822699 conn(0x7fdfc0077f50 msgr2=0x7fdfc0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 -- 172.21.15.66:0/2105822699 shutdown_connections 2024-04-21T09:08:24.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 -- 172.21.15.66:0/2105822699 wait complete. 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 Processor -- start 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.260+0000 7fdfc72ba640 1 -- start start 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101090 0x7fdfc019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc0106160 0x7fdfc019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc0198650 0x7fdfc0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc0111100 con 0x7fdfc0101090 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc0110f80 con 0x7fdfc0198650 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc0111280 con 0x7fdfc0106160 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101090 0x7fdfc019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101090 0x7fdfc019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33306/0 (socket says 172.21.15.66:33306) 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 -- 172.21.15.66:0/3547322868 learned_addr learned my addr 172.21.15.66:0/3547322868 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5857640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc0198650 0x7fdfc0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 -- 172.21.15.66:0/3547322868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc0106160 msgr2=0x7fdfc019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc4855640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc0106160 0x7fdfc019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc0106160 0x7fdfc019ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 -- 172.21.15.66:0/3547322868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc0198650 msgr2=0x7fdfc0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc0198650 0x7fdfc0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 -- 172.21.15.66:0/3547322868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc0199250 con 0x7fdfc0101090 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5857640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc0198650 0x7fdfc0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc5056640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101090 0x7fdfc019e590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c890 tx=0x7fdfbc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfb27fc640 1 -- 172.21.15.66:0/3547322868 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc004220 con 0x7fdfc0101090 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfb27fc640 1 -- 172.21.15.66:0/3547322868 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfbc0043c0 con 0x7fdfc0101090 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc01a5330 con 0x7fdfc0101090 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfb27fc640 1 -- 172.21.15.66:0/3547322868 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc004a00 con 0x7fdfc0101090 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.261+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc01a5730 con 0x7fdfc0101090 2024-04-21T09:08:24.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.262+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc0102560 con 0x7fdfc0101090 2024-04-21T09:08:24.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.264+0000 7fdfb27fc640 1 -- 172.21.15.66:0/3547322868 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fdfbc0079b0 con 0x7fdfc0101090 2024-04-21T09:08:24.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.264+0000 7fdfb27fc640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfac0a3db0 0x7fdfac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.264+0000 7fdfc4855640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfac0a3db0 0x7fdfac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.264+0000 7fdfb27fc640 1 -- 172.21.15.66:0/3547322868 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdfbc0d0f10 con 0x7fdfc0101090 2024-04-21T09:08:24.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.265+0000 7fdfc4855640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfac0a3db0 0x7fdfac0a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4036040 tx=0x7fdfb4000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:24.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.268+0000 7fdfb27fc640 1 -- 172.21.15.66:0/3547322868 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfbc014030 con 0x7fdfc0101090 2024-04-21T09:08:24.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.455+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfc0068d70 con 0x7fdfc0101090 2024-04-21T09:08:24.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.455+0000 7fdfb27fc640 1 -- 172.21.15.66:0/3547322868 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fdfbc09d0d0 con 0x7fdfc0101090 2024-04-21T09:08:24.456 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfac0a3db0 msgr2=0x7fdfac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfac0a3db0 0x7fdfac0a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4036040 tx=0x7fdfb4000bc0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101090 msgr2=0x7fdfc019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101090 0x7fdfc019e590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c890 tx=0x7fdfbc00cd50 comp rx=0 tx=0).stop 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 shutdown_connections 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfac0a3db0 0x7fdfac0a6270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc0198650 0x7fdfc0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc0106160 0x7fdfc019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 --2- 172.21.15.66:0/3547322868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc0101090 0x7fdfc019e590 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 >> 172.21.15.66:0/3547322868 conn(0x7fdfc0077f50 msgr2=0x7fdfc00695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:24.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 shutdown_connections 2024-04-21T09:08:24.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.457+0000 7fdfc72ba640 1 -- 172.21.15.66:0/3547322868 wait complete. 2024-04-21T09:08:24.467 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:24 smithi066 ceph-mon[27953]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.467 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:24 smithi066 ceph-mon[27953]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.467 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:24 smithi066 ceph-mon[27953]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.467 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:24 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/604885666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:24.467 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:24 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1089809669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:24.468 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.573+0000 7f59636f4640 1 -- 172.21.15.66:0/1629798287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 msgr2=0x7f595c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 --2- 172.21.15.66:0/1629798287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c10bfd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f594c009900 tx=0x7f594c02bd60 comp rx=0 tx=0).stop 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- 172.21.15.66:0/1629798287 shutdown_connections 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 --2- 172.21.15.66:0/1629798287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595c10c510 0x7f595c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 --2- 172.21.15.66:0/1629798287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c10bfd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 --2- 172.21.15.66:0/1629798287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1031d0 0x7f595c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- 172.21.15.66:0/1629798287 >> 172.21.15.66:0/1629798287 conn(0x7f595c077f50 msgr2=0x7f595c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- 172.21.15.66:0/1629798287 shutdown_connections 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- 172.21.15.66:0/1629798287 wait complete. 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 Processor -- start 2024-04-21T09:08:24.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- start start 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1031d0 0x7f595c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595c10c510 0x7f595c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f595c112180 con 0x7f595c103af0 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f595c112000 con 0x7f595c10c510 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.574+0000 7f59636f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f595c112300 con 0x7f595c1031d0 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33328/0 (socket says 172.21.15.66:33328) 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 -- 172.21.15.66:0/3065465059 learned_addr learned my addr 172.21.15.66:0/3065465059 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5961490640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1031d0 0x7f595c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5961c91640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595c10c510 0x7f595c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 -- 172.21.15.66:0/3065465059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1031d0 msgr2=0x7f595c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1031d0 0x7f595c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 -- 172.21.15.66:0/3065465059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595c10c510 msgr2=0x7f595c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595c10c510 0x7f595c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 -- 172.21.15.66:0/3065465059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c198d10 con 0x7f595c103af0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5961490640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1031d0 0x7f595c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f5960c8f640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c19e920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f594c005940 tx=0x7f594c02d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f59527fc640 1 -- 172.21.15.66:0/3065465059 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c043020 con 0x7f595c103af0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f59527fc640 1 -- 172.21.15.66:0/3065465059 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f594c02dda0 con 0x7f595c103af0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c198fa0 con 0x7f595c103af0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f59527fc640 1 -- 172.21.15.66:0/3065465059 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c003bf0 con 0x7f595c103af0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.575+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c1a52a0 con 0x7f595c103af0 2024-04-21T09:08:24.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.576+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c068d70 con 0x7f595c103af0 2024-04-21T09:08:24.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.577+0000 7f59527fc640 1 -- 172.21.15.66:0/3065465059 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f594c046020 con 0x7f595c103af0 2024-04-21T09:08:24.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.578+0000 7f59527fc640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59480a3e30 0x7f59480a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.578+0000 7f5961490640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59480a3e30 0x7f59480a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.578+0000 7f59527fc640 1 -- 172.21.15.66:0/3065465059 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f594c0f3580 con 0x7f595c103af0 2024-04-21T09:08:24.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.578+0000 7f5961490640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59480a3e30 0x7f59480a62f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5954005c40 tx=0x7f595400a430 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:24.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.582+0000 7f59527fc640 1 -- 172.21.15.66:0/3065465059 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f594c04d020 con 0x7f595c103af0 2024-04-21T09:08:24.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:24 smithi186 ceph-mon[31505]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:24 smithi186 ceph-mon[31505]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:24 smithi186 ceph-mon[31505]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:24 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/604885666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:24 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1089809669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:24 smithi083 ceph-mon[31413]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:24 smithi083 ceph-mon[31413]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:24 smithi083 ceph-mon[31413]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T09:08:24.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:24 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/604885666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:24.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:24 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1089809669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:24.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.767+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f595c107100 con 0x7f595c103af0 2024-04-21T09:08:24.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.768+0000 7f59527fc640 1 -- 172.21.15.66:0/3065465059 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f594c0bf850 con 0x7f595c103af0 2024-04-21T09:08:24.769 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59480a3e30 msgr2=0x7f59480a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59480a3e30 0x7f59480a62f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5954005c40 tx=0x7f595400a430 comp rx=0 tx=0).stop 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 msgr2=0x7f595c19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c19e920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f594c005940 tx=0x7f594c02d9a0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 shutdown_connections 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59480a3e30 0x7f59480a62f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595c10c510 0x7f595c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595c103af0 0x7f595c19e920 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 --2- 172.21.15.66:0/3065465059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1031d0 0x7f595c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 >> 172.21.15.66:0/3065465059 conn(0x7f595c077f50 msgr2=0x7f595c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:24.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 shutdown_connections 2024-04-21T09:08:24.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.770+0000 7f59636f4640 1 -- 172.21.15.66:0/3065465059 wait complete. 2024-04-21T09:08:24.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 -- 172.21.15.66:0/2062177146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224101a50 msgr2=0x7f5224105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 --2- 172.21.15.66:0/2062177146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224101a50 0x7f5224105a90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5218009900 tx=0x7f521802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 -- 172.21.15.66:0/2062177146 shutdown_connections 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 --2- 172.21.15.66:0/2062177146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5224106170 0x7f522410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 --2- 172.21.15.66:0/2062177146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224101a50 0x7f5224105a90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 --2- 172.21.15.66:0/2062177146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52241010a0 0x7f5224101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 -- 172.21.15.66:0/2062177146 >> 172.21.15.66:0/2062177146 conn(0x7f5224077eb0 msgr2=0x7f52240782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 -- 172.21.15.66:0/2062177146 shutdown_connections 2024-04-21T09:08:24.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.886+0000 7f522be31640 1 -- 172.21.15.66:0/2062177146 wait complete. 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 Processor -- start 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 -- start start 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52241010a0 0x7f52241986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224106170 0x7f5224198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522419cfa0 0x7f522419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5224111090 con 0x7f5224106170 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5224110f10 con 0x7f52241010a0 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522be31640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5224111210 con 0x7f522419cfa0 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224106170 0x7f5224198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224106170 0x7f5224198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54508/0 (socket says 172.21.15.66:54508) 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 -- 172.21.15.66:0/2724065699 learned_addr learned my addr 172.21.15.66:0/2724065699 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f522a3ce640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522419cfa0 0x7f522419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f5229bcd640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52241010a0 0x7f52241986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 -- 172.21.15.66:0/2724065699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522419cfa0 msgr2=0x7f522419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522419cfa0 0x7f522419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 -- 172.21.15.66:0/2724065699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52241010a0 msgr2=0x7f52241986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52241010a0 0x7f52241986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.887+0000 7f52293cc640 1 -- 172.21.15.66:0/2724065699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522419da90 con 0x7f5224106170 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.888+0000 7f52293cc640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224106170 0x7f5224198c20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5218036040 tx=0x7f521802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.888+0000 7f5216ffd640 1 -- 172.21.15.66:0/2724065699 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218043020 con 0x7f5224106170 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.888+0000 7f5216ffd640 1 -- 172.21.15.66:0/2724065699 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f521802cbe0 con 0x7f5224106170 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.888+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522419dd20 con 0x7f5224106170 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.888+0000 7f5216ffd640 1 -- 172.21.15.66:0/2724065699 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218003bf0 con 0x7f5224106170 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.888+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52241a5540 con 0x7f5224106170 2024-04-21T09:08:24.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.889+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52240691c0 con 0x7f5224106170 2024-04-21T09:08:24.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.890+0000 7f5216ffd640 1 -- 172.21.15.66:0/2724065699 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f5218046020 con 0x7f5224106170 2024-04-21T09:08:24.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.890+0000 7f5216ffd640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52100a3db0 0x7f52100a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:24.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.890+0000 7f5229bcd640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52100a3db0 0x7f52100a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:24.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.891+0000 7f5216ffd640 1 -- 172.21.15.66:0/2724065699 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f52180f3410 con 0x7f5224106170 2024-04-21T09:08:24.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.891+0000 7f5229bcd640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52100a3db0 0x7f52100a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5220005d00 tx=0x7f522000b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:24.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:24.894+0000 7f5216ffd640 1 -- 172.21.15.66:0/2724065699 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f521804d020 con 0x7f5224106170 2024-04-21T09:08:25.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.082+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5224101a50 con 0x7f5224106170 2024-04-21T09:08:25.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.082+0000 7f5216ffd640 1 -- 172.21.15.66:0/2724065699 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f52180bf6e0 con 0x7f5224106170 2024-04-21T09:08:25.083 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52100a3db0 msgr2=0x7f52100a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52100a3db0 0x7f52100a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5220005d00 tx=0x7f522000b410 comp rx=0 tx=0).stop 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224106170 msgr2=0x7f5224198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224106170 0x7f5224198c20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5218036040 tx=0x7f521802c7e0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 shutdown_connections 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52100a3db0 0x7f52100a6270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522419cfa0 0x7f522419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5224106170 0x7f5224198c20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 --2- 172.21.15.66:0/2724065699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52241010a0 0x7f52241986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 >> 172.21.15.66:0/2724065699 conn(0x7f5224077eb0 msgr2=0x7f522406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 shutdown_connections 2024-04-21T09:08:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.084+0000 7f522be31640 1 -- 172.21.15.66:0/2724065699 wait complete. 2024-04-21T09:08:25.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:08:25.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.200+0000 7f5891a3f640 1 -- 172.21.15.66:0/627534186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c101a40 msgr2=0x7f588c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.200+0000 7f5891a3f640 1 --2- 172.21.15.66:0/627534186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c101a40 0x7f588c105a80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5880009900 tx=0x7f588002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:25.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.200+0000 7f5891a3f640 1 -- 172.21.15.66:0/627534186 shutdown_connections 2024-04-21T09:08:25.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.200+0000 7f5891a3f640 1 --2- 172.21.15.66:0/627534186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f588c106160 0x7f588c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.200+0000 7f5891a3f640 1 --2- 172.21.15.66:0/627534186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c101a40 0x7f588c105a80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.200+0000 7f5891a3f640 1 --2- 172.21.15.66:0/627534186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588c101090 0x7f588c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.200+0000 7f5891a3f640 1 -- 172.21.15.66:0/627534186 >> 172.21.15.66:0/627534186 conn(0x7f588c0771d0 msgr2=0x7f588c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 -- 172.21.15.66:0/627534186 shutdown_connections 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 -- 172.21.15.66:0/627534186 wait complete. 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 Processor -- start 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 -- start start 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f588c101090 0x7f588c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588c106160 0x7f588c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c1985a0 0x7f588c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f588c06abc0 con 0x7f588c1985a0 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f588c06aa40 con 0x7f588c101090 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.201+0000 7f5891a3f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f588c06ad40 con 0x7f588c106160 2024-04-21T09:08:25.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c1985a0 0x7f588c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c1985a0 0x7f588c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54520/0 (socket says 172.21.15.66:54520) 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 -- 172.21.15.66:0/3150032272 learned_addr learned my addr 172.21.15.66:0/3150032272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588a7fc640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588c106160 0x7f588c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588affd640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f588c101090 0x7f588c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 -- 172.21.15.66:0/3150032272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588c106160 msgr2=0x7f588c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588c106160 0x7f588c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 -- 172.21.15.66:0/3150032272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f588c101090 msgr2=0x7f588c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f588c101090 0x7f588c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 -- 172.21.15.66:0/3150032272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c1990c0 con 0x7f588c1985a0 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588affd640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f588c101090 0x7f588c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:25.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f588b7fe640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c1985a0 0x7f588c198a50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f588400eaa0 tx=0x7f588400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:25.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f5890a3d640 1 -- 172.21.15.66:0/3150032272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588400cdb0 con 0x7f588c1985a0 2024-04-21T09:08:25.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f5890a3d640 1 -- 172.21.15.66:0/3150032272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5884005020 con 0x7f588c1985a0 2024-04-21T09:08:25.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c1993b0 con 0x7f588c1985a0 2024-04-21T09:08:25.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f5890a3d640 1 -- 172.21.15.66:0/3150032272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58840116d0 con 0x7f588c1985a0 2024-04-21T09:08:25.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.202+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c1a5600 con 0x7f588c1985a0 2024-04-21T09:08:25.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.203+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588c0691c0 con 0x7f588c1985a0 2024-04-21T09:08:25.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.205+0000 7f5890a3d640 1 -- 172.21.15.66:0/3150032272 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f58840051d0 con 0x7f588c1985a0 2024-04-21T09:08:25.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.205+0000 7f5890a3d640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f58740a3e30 0x7f58740a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.205+0000 7f588affd640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f58740a3e30 0x7f58740a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.205+0000 7f5890a3d640 1 -- 172.21.15.66:0/3150032272 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f58840d11d0 con 0x7f588c1985a0 2024-04-21T09:08:25.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.206+0000 7f588affd640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f58740a3e30 0x7f58740a62f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f587c009b50 tx=0x7f587c005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:25.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.209+0000 7f5890a3d640 1 -- 172.21.15.66:0/3150032272 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5884016030 con 0x7f588c1985a0 2024-04-21T09:08:25.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:25 smithi066 ceph-mon[27953]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-04-21T09:08:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:25 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3547322868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:25 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3065465059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:25 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2724065699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.398+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f588c101ae0 con 0x7f588c1985a0 2024-04-21T09:08:25.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.398+0000 7f5890a3d640 1 -- 172.21.15.66:0/3150032272 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f588409d410 con 0x7f588c1985a0 2024-04-21T09:08:25.399 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:08:25.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f58740a3e30 msgr2=0x7f58740a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f58740a3e30 0x7f58740a62f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f587c009b50 tx=0x7f587c005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:25.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c1985a0 msgr2=0x7f588c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c1985a0 0x7f588c198a50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f588400eaa0 tx=0x7f588400ef60 comp rx=0 tx=0).stop 2024-04-21T09:08:25.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 shutdown_connections 2024-04-21T09:08:25.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f58740a3e30 0x7f58740a62f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f588c1985a0 0x7f588c198a50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588c106160 0x7f588c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 --2- 172.21.15.66:0/3150032272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f588c101090 0x7f588c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.400+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 >> 172.21.15.66:0/3150032272 conn(0x7f588c0771d0 msgr2=0x7f588c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:25.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.401+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 shutdown_connections 2024-04-21T09:08:25.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.401+0000 7f5891a3f640 1 -- 172.21.15.66:0/3150032272 wait complete. 2024-04-21T09:08:25.412 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 -- 172.21.15.66:0/3923748589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22881031d0 msgr2=0x7f22881035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3923748589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22881031d0 0x7f22881035b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2278009900 tx=0x7f227802bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 -- 172.21.15.66:0/3923748589 shutdown_connections 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3923748589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f228810c5a0 0x7f228810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3923748589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2288103af0 0x7f228810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3923748589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22881031d0 0x7f22881035b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 -- 172.21.15.66:0/3923748589 >> 172.21.15.66:0/3923748589 conn(0x7f22880771d0 msgr2=0x7f22880775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 -- 172.21.15.66:0/3923748589 shutdown_connections 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.514+0000 7f228fbb2640 1 -- 172.21.15.66:0/3923748589 wait complete. 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 Processor -- start 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 -- start start 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22881031d0 0x7f22881a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2288103af0 0x7f22881a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f228810c5a0 0x7f228819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f22881121d0 con 0x7f228810c5a0 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2288112050 con 0x7f2288103af0 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228fbb2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2288112350 con 0x7f22881031d0 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f228810c5a0 0x7f228819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f228810c5a0 0x7f228819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54538/0 (socket says 172.21.15.66:54538) 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 -- 172.21.15.66:0/3142037620 learned_addr learned my addr 172.21.15.66:0/3142037620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228d14d640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2288103af0 0x7f22881a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228d94e640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22881031d0 0x7f22881a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 -- 172.21.15.66:0/3142037620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22881031d0 msgr2=0x7f22881a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22881031d0 0x7f22881a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 -- 172.21.15.66:0/3142037620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2288103af0 msgr2=0x7f22881a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2288103af0 0x7f22881a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 -- 172.21.15.66:0/3142037620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228819b000 con 0x7f228810c5a0 2024-04-21T09:08:25.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.515+0000 7f228e14f640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f228810c5a0 0x7f228819a830 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f228000dac0 tx=0x7f228000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:25.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.516+0000 7f227effd640 1 -- 172.21.15.66:0/3142037620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280017070 con 0x7f228810c5a0 2024-04-21T09:08:25.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.516+0000 7f227effd640 1 -- 172.21.15.66:0/3142037620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f228000be30 con 0x7f228810c5a0 2024-04-21T09:08:25.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.516+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228819b2f0 con 0x7f228810c5a0 2024-04-21T09:08:25.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.516+0000 7f227effd640 1 -- 172.21.15.66:0/3142037620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280010460 con 0x7f228810c5a0 2024-04-21T09:08:25.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.516+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a7510 con 0x7f228810c5a0 2024-04-21T09:08:25.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.517+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228806ae60 con 0x7f228810c5a0 2024-04-21T09:08:25.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.518+0000 7f227effd640 1 -- 172.21.15.66:0/3142037620 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f22800040b0 con 0x7f228810c5a0 2024-04-21T09:08:25.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.518+0000 7f227effd640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f22740a3e30 0x7f22740a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.518+0000 7f228d94e640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f22740a3e30 0x7f22740a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.519+0000 7f227effd640 1 -- 172.21.15.66:0/3142037620 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f22800d0830 con 0x7f228810c5a0 2024-04-21T09:08:25.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.519+0000 7f228d94e640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f22740a3e30 0x7f22740a62f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2278006010 tx=0x7f2278005e40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:25.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.523+0000 7f227effd640 1 -- 172.21.15.66:0/3142037620 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f228009c3a0 con 0x7f228810c5a0 2024-04-21T09:08:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:25 smithi186 ceph-mon[31505]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-04-21T09:08:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:25 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3547322868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:25 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3065465059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:25 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2724065699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:25 smithi083 ceph-mon[31413]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-04-21T09:08:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:25 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3547322868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:25 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3065465059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:25 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2724065699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:08:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.750+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22881a7740 con 0x7f228810c5a0 2024-04-21T09:08:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.751+0000 7f227effd640 1 -- 172.21.15.66:0/3142037620 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f22800a06a0 con 0x7f228810c5a0 2024-04-21T09:08:25.751 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:25.752 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:08:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f22740a3e30 msgr2=0x7f22740a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f22740a3e30 0x7f22740a62f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2278006010 tx=0x7f2278005e40 comp rx=0 tx=0).stop 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f228810c5a0 msgr2=0x7f228819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f228810c5a0 0x7f228819a830 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f228000dac0 tx=0x7f228000df80 comp rx=0 tx=0).stop 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 shutdown_connections 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f22740a3e30 0x7f22740a62f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f228810c5a0 0x7f228819a830 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2288103af0 0x7f22881a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 --2- 172.21.15.66:0/3142037620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22881031d0 0x7f22881a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 >> 172.21.15.66:0/3142037620 conn(0x7f22880771d0 msgr2=0x7f2288100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 shutdown_connections 2024-04-21T09:08:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.753+0000 7f228fbb2640 1 -- 172.21.15.66:0/3142037620 wait complete. 2024-04-21T09:08:25.764 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:08:25.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 -- 172.21.15.66:0/3753626590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee81021a0 msgr2=0x7f2ee8102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 --2- 172.21.15.66:0/3753626590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee81021a0 0x7f2ee8102580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009900 tx=0x7f2ed802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:25.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 -- 172.21.15.66:0/3753626590 shutdown_connections 2024-04-21T09:08:25.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 --2- 172.21.15.66:0/3753626590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ee8107270 0x7f2ee810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 --2- 172.21.15.66:0/3753626590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee8102b50 0x7f2ee8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 --2- 172.21.15.66:0/3753626590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee81021a0 0x7f2ee8102580 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 -- 172.21.15.66:0/3753626590 >> 172.21.15.66:0/3753626590 conn(0x7f2ee80793a0 msgr2=0x7f2ee80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 -- 172.21.15.66:0/3753626590 shutdown_connections 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.866+0000 7f2eede88640 1 -- 172.21.15.66:0/3753626590 wait complete. 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 Processor -- start 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 -- start start 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ee81021a0 0x7f2ee8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee8107270 0x7f2ee8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee819cfe0 0x7f2ee819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81121e0 con 0x7f2ee8107270 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8112060 con 0x7f2ee819cfe0 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2eede88640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8112360 con 0x7f2ee81021a0 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2ee6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee8107270 0x7f2ee8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2ee6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee8107270 0x7f2ee8198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54552/0 (socket says 172.21.15.66:54552) 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.867+0000 7f2ee6ffd640 1 -- 172.21.15.66:0/3507637574 learned_addr learned my addr 172.21.15.66:0/3507637574 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee7fff640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee819cfe0 0x7f2ee819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee77fe640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ee81021a0 0x7f2ee8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee6ffd640 1 -- 172.21.15.66:0/3507637574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ee81021a0 msgr2=0x7f2ee8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee6ffd640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ee81021a0 0x7f2ee8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee6ffd640 1 -- 172.21.15.66:0/3507637574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee819cfe0 msgr2=0x7f2ee819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee6ffd640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee819cfe0 0x7f2ee819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee6ffd640 1 -- 172.21.15.66:0/3507637574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee819dad0 con 0x7f2ee8107270 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee77fe640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ee81021a0 0x7f2ee8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee6ffd640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee8107270 0x7f2ee8198c60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e9c0 tx=0x7f2edc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee4ff9640 1 -- 172.21.15.66:0/3507637574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00cd80 con 0x7f2ee8107270 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee819ddc0 con 0x7f2ee8107270 2024-04-21T09:08:25.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee4ff9640 1 -- 172.21.15.66:0/3507637574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2edc0054d0 con 0x7f2ee8107270 2024-04-21T09:08:25.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2ee4ff9640 1 -- 172.21.15.66:0/3507637574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc010430 con 0x7f2ee8107270 2024-04-21T09:08:25.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.868+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a55b0 con 0x7f2ee8107270 2024-04-21T09:08:25.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.869+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee8069230 con 0x7f2ee8107270 2024-04-21T09:08:25.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.870+0000 7f2ee4ff9640 1 -- 172.21.15.66:0/3507637574 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f2edc004fd0 con 0x7f2ee8107270 2024-04-21T09:08:25.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.871+0000 7f2ee4ff9640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2ed00a3db0 0x7f2ed00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:25.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.871+0000 7f2ee77fe640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2ed00a3db0 0x7f2ed00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:25.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.871+0000 7f2ee4ff9640 1 -- 172.21.15.66:0/3507637574 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2edc0d11c0 con 0x7f2ee8107270 2024-04-21T09:08:25.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.871+0000 7f2ee77fe640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2ed00a3db0 0x7f2ed00a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8036040 tx=0x7f2ed8000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:25.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:25.876+0000 7f2ee4ff9640 1 -- 172.21.15.66:0/3507637574 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2edc09d380 con 0x7f2ee8107270 2024-04-21T09:08:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.067+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ee8102b50 con 0x7f2ee8107270 2024-04-21T09:08:26.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.068+0000 7f2ee4ff9640 1 -- 172.21.15.66:0/3507637574 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+25086 (secure 0 0 0) 0x7f2edc09cc80 con 0x7f2ee8107270 2024-04-21T09:08:26.069 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:26.069 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":96,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:08:12.028415+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2ed00a3db0 msgr2=0x7f2ed00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2ed00a3db0 0x7f2ed00a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8036040 tx=0x7f2ed8000bc0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee8107270 msgr2=0x7f2ee8198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee8107270 0x7f2ee8198c60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e9c0 tx=0x7f2edc00ee80 comp rx=0 tx=0).stop 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 shutdown_connections 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2ed00a3db0 0x7f2ed00a6270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee819cfe0 0x7f2ee819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ee8107270 0x7f2ee8198c60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 --2- 172.21.15.66:0/3507637574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ee81021a0 0x7f2ee8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 >> 172.21.15.66:0/3507637574 conn(0x7f2ee80793a0 msgr2=0x7f2ee80fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 shutdown_connections 2024-04-21T09:08:26.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.070+0000 7f2eede88640 1 -- 172.21.15.66:0/3507637574 wait complete. 2024-04-21T09:08:26.082 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 809877 2024-04-21T09:08:26.082 DEBUG:tasks.quiescer:Yielding 2024-04-21T09:08:26.083 INFO:teuthology.run_tasks:Running task workunit... 2024-04-21T09:08:26.097 INFO:tasks.workunit:Pulling workunits from ref 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T09:08:26.097 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-21T09:08:26.097 INFO:tasks.workunit:timeout=3h 2024-04-21T09:08:26.097 INFO:tasks.workunit:cleanup=True 2024-04-21T09:08:26.097 DEBUG:teuthology.orchestra.run.smithi066:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:26.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-21 09:06:14.539065224 +0000 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-21 09:06:14.539065224 +0000 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-21 09:08:09.461642460 +0000 2024-04-21T09:08:26.114 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-21 09:06:14.539065224 +0000 2024-04-21T09:08:26.114 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-21T09:08:26.115 DEBUG:teuthology.orchestra.run.smithi066:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-21T09:08:26.187 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/leonid-s-usov/ceph-ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T09:08:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 -- 172.21.15.66:0/2439514701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84100e80 msgr2=0x7f0f8410ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2439514701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84100e80 0x7f0f8410ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009cd0 tx=0x7f0f7c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 -- 172.21.15.66:0/2439514701 shutdown_connections 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2439514701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84100e80 0x7f0f8410ede0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2439514701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f841004c0 0x7f0f84100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2439514701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f84106f50 0x7f0f84107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 -- 172.21.15.66:0/2439514701 >> 172.21.15.66:0/2439514701 conn(0x7f0f84077450 msgr2=0x7f0f840fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 -- 172.21.15.66:0/2439514701 shutdown_connections 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.201+0000 7f0f8890d640 1 -- 172.21.15.66:0/2439514701 wait complete. 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 Processor -- start 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 -- start start 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f841004c0 0x7f0f84198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f84100e80 0x7f0f84198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84106f50 0x7f0f8419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112280 con 0x7f0f84106f50 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112100 con 0x7f0f84100e80 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.202+0000 7f0f8890d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112400 con 0x7f0f841004c0 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84106f50 0x7f0f8419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84106f50 0x7f0f8419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54580/0 (socket says 172.21.15.66:54580) 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 -- 172.21.15.66:0/2346280358 learned_addr learned my addr 172.21.15.66:0/2346280358 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f8259c640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f841004c0 0x7f0f84198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f7bfff640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f84100e80 0x7f0f84198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 -- 172.21.15.66:0/2346280358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f841004c0 msgr2=0x7f0f84198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f841004c0 0x7f0f84198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 -- 172.21.15.66:0/2346280358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f84100e80 msgr2=0x7f0f84198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f84100e80 0x7f0f84198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 -- 172.21.15.66:0/2346280358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8419d730 con 0x7f0f84106f50 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f82d9d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84106f50 0x7f0f8419d000 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c004af0 tx=0x7f0f7c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f7b7fe640 1 -- 172.21.15.66:0/2346280358 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c041020 con 0x7f0f84106f50 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f7b7fe640 1 -- 172.21.15.66:0/2346280358 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c032960 con 0x7f0f84106f50 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8419d9c0 con 0x7f0f84106f50 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f7b7fe640 1 -- 172.21.15.66:0/2346280358 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c039040 con 0x7f0f84106f50 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.203+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a52b0 con 0x7f0f84106f50 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.204+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84069230 con 0x7f0f84106f50 2024-04-21T09:08:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.206+0000 7f0f7b7fe640 1 -- 172.21.15.66:0/2346280358 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f0f7c044020 con 0x7f0f84106f50 2024-04-21T09:08:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.206+0000 7f0f7b7fe640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f700a3e30 0x7f0f700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.207+0000 7f0f8259c640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f700a3e30 0x7f0f700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.207+0000 7f0f7b7fe640 1 -- 172.21.15.66:0/2346280358 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f0f7c0f2c70 con 0x7f0f84106f50 2024-04-21T09:08:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.207+0000 7f0f8259c640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f700a3e30 0x7f0f700a62f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009bd0 tx=0x7f0f74005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.210+0000 7f0f7b7fe640 1 -- 172.21.15.66:0/2346280358 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f7c04b020 con 0x7f0f84106f50 2024-04-21T09:08:26.218 INFO:tasks.workunit.client.0.smithi066.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-21T09:08:26.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:26 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3150032272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:26.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:26 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3142037620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:26.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:26 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3507637574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:26.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.438+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f8410a4a0 con 0x7f0f84106f50 2024-04-21T09:08:26.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.438+0000 7f0f7b7fe640 1 -- 172.21.15.66:0/2346280358 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f0f7c0bf060 con 0x7f0f84106f50 2024-04-21T09:08:26.439 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:26.439 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:08:26.439 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:08:26.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f700a3e30 msgr2=0x7f0f700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f700a3e30 0x7f0f700a62f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009bd0 tx=0x7f0f74005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:26.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84106f50 msgr2=0x7f0f8419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84106f50 0x7f0f8419d000 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c004af0 tx=0x7f0f7c034670 comp rx=0 tx=0).stop 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 shutdown_connections 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f700a3e30 0x7f0f700a62f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84106f50 0x7f0f8419d000 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f84100e80 0x7f0f84198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 --2- 172.21.15.66:0/2346280358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f841004c0 0x7f0f84198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 >> 172.21.15.66:0/2346280358 conn(0x7f0f84077450 msgr2=0x7f0f84104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.440+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 shutdown_connections 2024-04-21T09:08:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.441+0000 7f0f8890d640 1 -- 172.21.15.66:0/2346280358 wait complete. 2024-04-21T09:08:26.452 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T09:05:46.330734+0000', 'modified': '2024-04-21T09:08:10.160804+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24490, 'mds_2': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/2851815427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 2851815427}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 2851815427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/1405629184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 1405629184}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 1405629184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6837/1780100152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1780100152}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1780100152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.83:6837/1568837633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1568837633}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1568837633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.186:6839/599772329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 599772329}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 599772329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/4140026829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 4140026829}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 4140026829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24512, 24490, 24476]} 2024-04-21T09:08:26.452 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-21T09:08:26.452 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T09:08:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:26 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3150032272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:26 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3142037620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:26 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3507637574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 -- 172.21.15.66:0/3988213935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 msgr2=0x7f6f0410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 --2- 172.21.15.66:0/3988213935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f0410ee40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009ca0 tx=0x7f6efc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 -- 172.21.15.66:0/3988213935 shutdown_connections 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 --2- 172.21.15.66:0/3988213935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f0410ee40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 --2- 172.21.15.66:0/3988213935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0406b1f0 0x7f6f04106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 --2- 172.21.15.66:0/3988213935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f0406a840 0x7f6f0406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 -- 172.21.15.66:0/3988213935 >> 172.21.15.66:0/3988213935 conn(0x7f6f04077eb0 msgr2=0x7f6f040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 -- 172.21.15.66:0/3988213935 shutdown_connections 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.594+0000 7f6f0b070640 1 -- 172.21.15.66:0/3988213935 wait complete. 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 Processor -- start 2024-04-21T09:08:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 -- start start 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0406a840 0x7f6f04198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f04198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f0419cff0 0x7f6f0419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04112230 con 0x7f6f04107270 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041120b0 con 0x7f6f0419cff0 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0b070640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041123b0 con 0x7f6f0406a840 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f03fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f04198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f03fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f04198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54596/0 (socket says 172.21.15.66:54596) 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f03fff640 1 -- 172.21.15.66:0/2253051912 learned_addr learned my addr 172.21.15.66:0/2253051912 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.595+0000 7f6f0960d640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f0419cff0 0x7f6f0419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0406a840 0x7f6f04198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f03fff640 1 -- 172.21.15.66:0/2253051912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0406a840 msgr2=0x7f6f04198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f03fff640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0406a840 0x7f6f04198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f03fff640 1 -- 172.21.15.66:0/2253051912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f0419cff0 msgr2=0x7f6f0419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f03fff640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f0419cff0 0x7f6f0419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f03fff640 1 -- 172.21.15.66:0/2253051912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0419dae0 con 0x7f6f04107270 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0406a840 0x7f6f04198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f0960d640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f0419cff0 0x7f6f0419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f03fff640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f04198c70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b450 tx=0x7f6ef800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f01ffb640 1 -- 172.21.15.66:0/2253051912 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8015020 con 0x7f6f04107270 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f01ffb640 1 -- 172.21.15.66:0/2253051912 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef8011d40 con 0x7f6f04107270 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0419ddd0 con 0x7f6f04107270 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f01ffb640 1 -- 172.21.15.66:0/2253051912 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800f340 con 0x7f6f04107270 2024-04-21T09:08:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.596+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f041a55c0 con 0x7f6f04107270 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.598+0000 7f6f01ffb640 1 -- 172.21.15.66:0/2253051912 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f6ef8018020 con 0x7f6f04107270 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 Processor -- start 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 -- start start 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f01ffb640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6eec0a3db0 0x7f6eec0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142d9610 0x5641142d99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 0x5641142e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142e3120 0x5641142e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56411423db10 con 0x5641142d9610 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641141db240 con 0x5641142e3120 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0960d640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6eec0a3db0 0x7f6eec0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564114239ad0 con 0x5641142d9fc0 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f01ffb640 1 -- 172.21.15.66:0/2253051912 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6ef80d0750 con 0x7f6f04107270 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142d9610 0x5641142d99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142d9610 0x5641142d99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54608/0 (socket says 172.21.15.66:54608) 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 -- 172.21.15.66:0/358200459 learned_addr learned my addr 172.21.15.66:0/358200459 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0960d640 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142e3120 0x5641142e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f03fff640 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 0x5641142e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 -- 172.21.15.66:0/358200459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 msgr2=0x5641142e2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 0x5641142e2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 -- 172.21.15.66:0/358200459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142e3120 msgr2=0x5641142e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142e3120 0x5641142e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f08e0c640 1 -- 172.21.15.66:0/358200459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564113f502d0 con 0x5641142d9610 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.599+0000 7f6f0960d640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6eec0a3db0 0x7f6eec0a6270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005e30 tx=0x7f6efc02cf90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142d9610 0x5641142d99f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009c50 tx=0x7f6ef400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e888f2514b993f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6eeaffd640 1 -- 172.21.15.66:0/358200459 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4016070 con 0x5641142d9610 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6eeaffd640 1 -- 172.21.15.66:0/358200459 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef40054d0 con 0x5641142d9610 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6eeaffd640 1 -- 172.21.15.66:0/358200459 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4011470 con 0x5641142d9610 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 -- 172.21.15.66:0/358200459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142d9610 msgr2=0x5641142d99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142d9610 0x5641142d99f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009c50 tx=0x7f6ef400e6c0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 -- 172.21.15.66:0/358200459 shutdown_connections 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142e3120 0x5641142e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 0x5641142e2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/358200459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142d9610 0x5641142d99f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 -- 172.21.15.66:0/358200459 >> 172.21.15.66:0/358200459 conn(0x5641142d3a60 msgr2=0x5641142d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 -- 172.21.15.66:0/358200459 shutdown_connections 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.600+0000 7f6f0dc59740 1 -- 172.21.15.66:0/358200459 wait complete. 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 -- start start 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 Processor -- start 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x5641142e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5641142ea0a0 con 0x5641142e3120 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641141c6190 con 0x5641142d9fc0 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5641142ea220 con 0x5641142d9610 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0dc59740 10 client.0 _ll_register_callbacks cb 0x5641141c18b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0960d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0960d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54610/0 (socket says 172.21.15.66:54610) 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f0960d640 1 -- 172.21.15.66:0/1550253738 learned_addr learned my addr 172.21.15.66:0/1550253738 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.601+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x5641142e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f03fff640 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0960d640 1 -- 172.21.15.66:0/1550253738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 msgr2=0x5641142e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0960d640 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x5641142e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0960d640 1 -- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 msgr2=0x5641142e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0960d640 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0960d640 1 -- 172.21.15.66:0/1550253738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411401a6b0 con 0x5641142e3120 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0960d640 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e7690 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6efc036000 tx=0x7f6efc02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f5b09a6066dede server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6ee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6ee3fff640 1 -- 172.21.15.66:0/1550253738 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc052060 con 0x5641142e3120 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef40054d0 con 0x5641142e3120 2024-04-21T09:08:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6ee3fff640 1 -- 172.21.15.66:0/1550253738 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6efc004240 con 0x5641142e3120 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6ee3fff640 1 -- 172.21.15.66:0/1550253738 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6efc059050 con 0x5641142e3120 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0dc59740 10 client.15846 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5641142f63b0 con 0x5641142e3120 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6ee3fff640 1 -- 172.21.15.66:0/1550253738 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef40054d0 con 0x5641142e3120 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.602+0000 7f6ee3fff640 1 -- 172.21.15.66:0/1550253738 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6efc034ab0 con 0x5641142e3120 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f0dc59740 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f0dc59740 10 client.15846 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f0dc59740 20 client.15846 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641142f89e0 0x5641142f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f0dc59740 4 client.15846 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56411424b530 con 0x5641142f89e0 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641142f89e0 0x5641142f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6ee3fff640 10 client.15846 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.603+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641142f89e0 0x5641142f9e50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=604c65746e015a8d server_cookie=56cd0e7b05b08030 in_seq=0 out_seq=0 2024-04-21T09:08:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6ee3fff640 1 -- 172.21.15.66:0/1550253738 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56411424b530 con 0x5641142f89e0 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6ee3fff640 10 client.15846 handle_command_reply: tid=0 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 client.15846 shutdown 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 msgr2=0x5641142e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e7690 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6efc036000 tx=0x7f6efc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 shutdown_connections 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641142f89e0 0x5641142f9e50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e7690 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1550253738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x5641142e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6ee3fff640 0 client.15846 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.604+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 >> 172.21.15.66:0/1550253738 conn(0x5641142d3a60 msgr2=0x5641142d5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.605+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 shutdown_connections 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.605+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1550253738 wait complete. 2024-04-21T09:08:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.605+0000 7f6f0dc59740 20 client.15846 trim_cache size 0 max 16384 2024-04-21T09:08:26.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:26 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3150032272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:08:26.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:26 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3142037620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:26.613 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:26 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3507637574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 Processor -- start 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 -- start start 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9610 0x5641142e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 0x5641142f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5641142f8190 con 0x5641142e3120 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641142f8860 con 0x5641142d9610 2024-04-21T09:08:26.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5641142f8f30 con 0x5641142d9fc0 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54622/0 (socket says 172.21.15.66:54622) 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 -- 172.21.15.66:0/1303673147 learned_addr learned my addr 172.21.15.66:0/1303673147 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 -- 172.21.15.66:0/1303673147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 msgr2=0x5641142f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 0x5641142f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 -- 172.21.15.66:0/1303673147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9610 msgr2=0x5641142e92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9610 0x5641142e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9610 0x5641142e92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 -- 172.21.15.66:0/1303673147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641142d8840 con 0x5641142e3120 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9610 0x5641142e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6f0960d640 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4510 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6efc02f020 tx=0x7f6efc0093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cf2d7130249269 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.623+0000 7f6eea7fc640 1 -- 172.21.15.66:0/1303673147 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc052040 con 0x5641142e3120 2024-04-21T09:08:26.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6eea7fc640 1 -- 172.21.15.66:0/1303673147 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6efc062070 con 0x5641142e3120 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6eea7fc640 1 -- 172.21.15.66:0/1303673147 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc039040 con 0x5641142e3120 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1303673147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 msgr2=0x5641142e4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4510 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6efc02f020 tx=0x7f6efc0093a0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1303673147 shutdown_connections 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4510 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9fc0 0x5641142f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/1303673147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9610 0x5641142e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1303673147 >> 172.21.15.66:0/1303673147 conn(0x5641142d3a60 msgr2=0x5641142ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1303673147 shutdown_connections 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 -- 172.21.15.66:0/1303673147 wait complete. 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.624+0000 7f6f0dc59740 1 -- start start 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 1 Processor -- start 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x564114302640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5641142f9cd0 con 0x5641142e3120 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641142f9600 con 0x5641142d9fc0 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5641142fa3a0 con 0x5641142d9610 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0dc59740 10 client.0 _ll_register_callbacks cb 0x7f6efc047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54626/0 (socket says 172.21.15.66:54626) 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 -- 172.21.15.66:0/2167094363 learned_addr learned my addr 172.21.15.66:0/2167094363 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f03fff640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x564114302640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 -- 172.21.15.66:0/2167094363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 msgr2=0x564114302640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x564114302640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 -- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 msgr2=0x5641142e3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f0960d640 1 -- 172.21.15.66:0/2167094363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564114300160 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f03fff640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.625+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x564114302640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0960d640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4000 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6efc02c130 tx=0x7f6efc043ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f9b034675b8ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6ee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6ee3fff640 1 -- 172.21.15.66:0/2167094363 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc052040 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6ee3fff640 1 -- 172.21.15.66:0/2167094363 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6efc034640 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc062070 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6ee3fff640 1 -- 172.21.15.66:0/2167094363 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6efc076050 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 10 client.15852 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56411435eae0 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6ee3fff640 1 -- 172.21.15.66:0/2167094363 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6efc062070 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6ee3fff640 1 -- 172.21.15.66:0/2167094363 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6efc004320 con 0x5641142e3120 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 10 client.15852 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:26.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 10 client.15852 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 20 client.15852 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641143631b0 0x564114361ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 4 client.15852 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.626+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "status"}) -- 0x5641142453a0 con 0x5641143631b0 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641143631b0 0x564114361ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6ee3fff640 10 client.15852 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f08e0c640 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641143631b0 0x564114361ec0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c4fb8dd7aa73468 server_cookie=967e9d2fcb0b65ca in_seq=0 out_seq=0 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6ee3fff640 1 -- 172.21.15.66:0/2167094363 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5641142453a0 con 0x5641143631b0 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6ee3fff640 10 client.15852 handle_command_reply: tid=0 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 client.15852 shutdown 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 msgr2=0x5641142e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4000 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6efc02c130 tx=0x7f6efc043ab0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 shutdown_connections 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5641143631b0 0x564114361ec0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641142e3120 0x5641142e4000 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641142d9fc0 0x5641142e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6ee3fff640 0 client.15852 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 --2- 172.21.15.66:0/2167094363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641142d9610 0x564114302640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.627+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 >> 172.21.15.66:0/2167094363 conn(0x5641142d3a60 msgr2=0x5641142e0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 shutdown_connections 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0dc59740 1 -- 172.21.15.66:0/2167094363 wait complete. 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0dc59740 20 client.15852 trim_cache size 0 max 16384 2024-04-21T09:08:26.635 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 2, 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24476, 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 134.538154883, 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 25, 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 96, 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 95, 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 177.03649089499999 2024-04-21T09:08:26.636 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:26.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6eec0a3db0 msgr2=0x7f6eec0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6eec0a3db0 0x7f6eec0a6270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005e30 tx=0x7f6efc02cf90 comp rx=0 tx=0).stop 2024-04-21T09:08:26.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 msgr2=0x7f6f04198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f04198c70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b450 tx=0x7f6ef800b910 comp rx=0 tx=0).stop 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 shutdown_connections 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6eec0a3db0 0x7f6eec0a6270 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f0419cff0 0x7f6f0419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f04107270 0x7f6f04198c70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 --2- 172.21.15.66:0/2253051912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0406a840 0x7f6f04198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 >> 172.21.15.66:0/2253051912 conn(0x7f6f04077eb0 msgr2=0x7f6f04100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 shutdown_connections 2024-04-21T09:08:26.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.628+0000 7f6f0b070640 1 -- 172.21.15.66:0/2253051912 wait complete. 2024-04-21T09:08:26.639 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T09:08:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 -- 172.21.15.66:0/1095653410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4101a50 msgr2=0x7f4dc4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/1095653410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4101a50 0x7f4dc4105a90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009900 tx=0x7f4db402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 -- 172.21.15.66:0/1095653410 shutdown_connections 2024-04-21T09:08:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/1095653410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc4106170 0x7f4dc410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/1095653410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4101a50 0x7f4dc4105a90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/1095653410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dc41010a0 0x7f4dc4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 -- 172.21.15.66:0/1095653410 >> 172.21.15.66:0/1095653410 conn(0x7f4dc4077eb0 msgr2=0x7f4dc40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 -- 172.21.15.66:0/1095653410 shutdown_connections 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.742+0000 7f4dc9d96640 1 -- 172.21.15.66:0/1095653410 wait complete. 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 Processor -- start 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 -- start start 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc41010a0 0x7f4dc41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dc4106170 0x7f4dc4199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4199fa0 0x7f4dc41a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4111090 con 0x7f4dc4199fa0 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4110f10 con 0x7f4dc41010a0 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc9d96640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4111210 con 0x7f4dc4106170 2024-04-21T09:08:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4199fa0 0x7f4dc41a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4199fa0 0x7f4dc41a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54642/0 (socket says 172.21.15.66:54642) 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc3fff640 1 -- 172.21.15.66:0/164688255 learned_addr learned my addr 172.21.15.66:0/164688255 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.743+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc41010a0 0x7f4dc41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dc4106170 0x7f4dc4199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc3fff640 1 -- 172.21.15.66:0/164688255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dc4106170 msgr2=0x7f4dc4199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc3fff640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dc4106170 0x7f4dc4199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc3fff640 1 -- 172.21.15.66:0/164688255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc41010a0 msgr2=0x7f4dc41994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc3fff640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc41010a0 0x7f4dc41994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc3fff640 1 -- 172.21.15.66:0/164688255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc41a4d20 con 0x7f4dc4199fa0 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc41010a0 0x7f4dc41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc3fff640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4199fa0 0x7f4dc41a4550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00eb10 tx=0x7f4dbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc0ff9640 1 -- 172.21.15.66:0/164688255 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc004b20 con 0x7f4dc4199fa0 2024-04-21T09:08:26.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc0ff9640 1 -- 172.21.15.66:0/164688255 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc004cc0 con 0x7f4dc4199fa0 2024-04-21T09:08:26.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc41a5010 con 0x7f4dc4199fa0 2024-04-21T09:08:26.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc0ff9640 1 -- 172.21.15.66:0/164688255 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc010670 con 0x7f4dc4199fa0 2024-04-21T09:08:26.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.744+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc41a54e0 con 0x7f4dc4199fa0 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.746+0000 7f4dc0ff9640 1 -- 172.21.15.66:0/164688255 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f4dbc005020 con 0x7f4dc4199fa0 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 Processor -- start 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 -- start start 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb167b0 0x5622bbb16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb17160 0x5622bbb1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb22130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5622bba7ab10 con 0x5622bbb167b0 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622bba18240 con 0x5622bbb1fce0 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dcc987740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5622bba76ad0 con 0x5622bbb17160 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc0ff9640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db00a3db0 0x7f4db00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb167b0 0x5622bbb16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb167b0 0x5622bbb16b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54644/0 (socket says 172.21.15.66:54644) 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/3976476925 learned_addr learned my addr 172.21.15.66:0/3976476925 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc3fff640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db00a3db0 0x7f4db00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb17160 0x5622bbb1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb22130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc0ff9640 1 -- 172.21.15.66:0/164688255 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4dbc0d1350 con 0x7f4dc4199fa0 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/3976476925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb17160 msgr2=0x5622bbb1f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb17160 0x5622bbb1f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/3976476925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 msgr2=0x5622bbb22130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb22130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.747+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/3976476925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622bb78d2d0 con 0x5622bbb167b0 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.748+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb167b0 0x5622bbb16b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4db4005940 tx=0x7f4db40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986d3bc3752ab8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.748+0000 7f4dc3fff640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db00a3db0 0x7f4db00a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0ce6d0 tx=0x7f4dbc00efd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.748+0000 7f4db9ffb640 1 -- 172.21.15.66:0/3976476925 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db402c7e0 con 0x5622bbb167b0 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.748+0000 7f4db9ffb640 1 -- 172.21.15.66:0/3976476925 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db4048070 con 0x5622bbb167b0 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.748+0000 7f4db9ffb640 1 -- 172.21.15.66:0/3976476925 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4004920 con 0x5622bbb167b0 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.748+0000 7f4dcc987740 1 -- 172.21.15.66:0/3976476925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb167b0 msgr2=0x5622bbb16b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.748+0000 7f4dcc987740 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb167b0 0x5622bbb16b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4db4005940 tx=0x7f4db40058b0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- 172.21.15.66:0/3976476925 shutdown_connections 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb22130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb17160 0x5622bbb1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 --2- 172.21.15.66:0/3976476925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb167b0 0x5622bbb16b90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- 172.21.15.66:0/3976476925 >> 172.21.15.66:0/3976476925 conn(0x5622bbb10a60 msgr2=0x5622bbb10e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- 172.21.15.66:0/3976476925 shutdown_connections 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- 172.21.15.66:0/3976476925 wait complete. 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- start start 2024-04-21T09:08:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 Processor -- start 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb167b0 0x5622bbb1cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb1d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb22bf0 con 0x5622bbb17160 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622bba03190 con 0x5622bbb1fce0 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.749+0000 7f4dcc987740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb22d70 con 0x5622bbb167b0 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dcc987740 10 client.0 _ll_register_callbacks cb 0x5622bb9fe8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54646/0 (socket says 172.21.15.66:54646) 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 -- 172.21.15.66:0/1823050442 learned_addr learned my addr 172.21.15.66:0/1823050442 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb1d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb167b0 0x5622bbb1cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 -- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb167b0 msgr2=0x5622bbb1cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb167b0 0x5622bbb1cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 -- 172.21.15.66:0/1823050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 msgr2=0x5622bbb1d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb1d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 -- 172.21.15.66:0/1823050442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622bb8576b0 con 0x5622bbb17160 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4dac011880 tx=0x7f4dac011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa584072e680362a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4da2ffd640 1 -- 172.21.15.66:0/1823050442 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00f9a0 con 0x5622bbb17160 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4da2ffd640 1 -- 172.21.15.66:0/1823050442 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4dac00fb40 con 0x5622bbb17160 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db4048070 con 0x5622bbb17160 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4da2ffd640 1 -- 172.21.15.66:0/1823050442 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4dac00fd40 con 0x5622bbb17160 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dcc987740 10 client.15864 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.750+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622bbb2ef30 con 0x5622bbb17160 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4da2ffd640 1 -- 172.21.15.66:0/1823050442 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db4048070 con 0x5622bbb17160 2024-04-21T09:08:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4da2ffd640 1 -- 172.21.15.66:0/1823050442 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4dac018030 con 0x5622bbb17160 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4dcc987740 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4dcc987740 10 client.15864 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4dcc987740 20 client.15864 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4dcc987740 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb31560 0x5622bbb329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4dcc987740 4 client.15864 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622bba88530 con 0x5622bbb31560 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.751+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb31560 0x5622bbb329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb31560 0x5622bbb329d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b7fd8275e910fc7 server_cookie=5800b0a64a99a47f in_seq=0 out_seq=0 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4da2ffd640 10 client.15864 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4da2ffd640 1 -- 172.21.15.66:0/1823050442 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622bba88530 con 0x5622bbb31560 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4da2ffd640 10 client.15864 handle_command_reply: tid=0 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 client.15864 shutdown 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 msgr2=0x5622bbb1d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4dac011880 tx=0x7f4dac011d40 comp rx=0 tx=0).stop 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 shutdown_connections 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb1fce0 0x5622bbb1d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb31560 0x5622bbb329d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d170 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 --2- 172.21.15.66:0/1823050442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb167b0 0x5622bbb1cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4da2ffd640 0 client.15864 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.752+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 >> 172.21.15.66:0/1823050442 conn(0x5622bbb10a60 msgr2=0x5622bbb12bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.753+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 shutdown_connections 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.753+0000 7f4dcc987740 1 -- 172.21.15.66:0/1823050442 wait complete. 2024-04-21T09:08:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.753+0000 7f4dcc987740 20 client.15864 trim_cache size 0 max 16384 2024-04-21T09:08:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 Processor -- start 2024-04-21T09:08:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 -- start start 2024-04-21T09:08:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 0x5622bbb30e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 0x5622bbb1f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb30d10 con 0x5622bbb17160 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb313e0 con 0x5622bbb167b0 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dcc987740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb31ab0 con 0x5622bbb10a60 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54658/0 (socket says 172.21.15.66:54658) 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/630392102 learned_addr learned my addr 172.21.15.66:0/630392102 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/630392102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 msgr2=0x5622bbb30e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/630392102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 0x5622bbb30e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/630392102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 msgr2=0x5622bbb1f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/630392102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 0x5622bbb1f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/630392102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622bbb1b180 con 0x5622bbb17160 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.771+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/630392102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d4d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009850 tx=0x7f4db4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e85f80cc0e0511 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4db97fa640 1 -- 172.21.15.66:0/630392102 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4005d50 con 0x5622bbb17160 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4db97fa640 1 -- 172.21.15.66:0/630392102 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db4048040 con 0x5622bbb17160 2024-04-21T09:08:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4db97fa640 1 -- 172.21.15.66:0/630392102 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4003ec0 con 0x5622bbb17160 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 -- 172.21.15.66:0/630392102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 msgr2=0x5622bbb1d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 --2- 172.21.15.66:0/630392102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d4d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009850 tx=0x7f4db4005a20 comp rx=0 tx=0).stop 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 -- 172.21.15.66:0/630392102 shutdown_connections 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 --2- 172.21.15.66:0/630392102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1d4d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 --2- 172.21.15.66:0/630392102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 0x5622bbb1f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 --2- 172.21.15.66:0/630392102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 0x5622bbb30e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 -- 172.21.15.66:0/630392102 >> 172.21.15.66:0/630392102 conn(0x5622bbb178b0 msgr2=0x5622bbb1b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 -- 172.21.15.66:0/630392102 shutdown_connections 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 -- 172.21.15.66:0/630392102 wait complete. 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.772+0000 7f4dcc987740 1 -- start start 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 1 Processor -- start 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 0x5622bbb19940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 0x5622bbb19e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb32850 con 0x5622bbb17160 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb32180 con 0x5622bbb167b0 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5622bbb32f20 con 0x5622bbb10a60 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dcc987740 10 client.0 _ll_register_callbacks cb 0x7f4dac011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54660/0 (socket says 172.21.15.66:54660) 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/2162160449 learned_addr learned my addr 172.21.15.66:0/2162160449 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 0x5622bbb19940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 0x5622bbb19e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 msgr2=0x5622bbb19940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 0x5622bbb19940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/2162160449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 msgr2=0x5622bbb19e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 0x5622bbb19e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 -- 172.21.15.66:0/2162160449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622bbb38f30 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4dc2ffd640 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1a3c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4db40059d0 tx=0x7f4db4045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c4c18b41eac152 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.773+0000 7f4da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4da2ffd640 1 -- 172.21.15.66:0/2162160449 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db402c7e0 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4da2ffd640 1 -- 172.21.15.66:0/2162160449 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4db402c980 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db4048040 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4da2ffd640 1 -- 172.21.15.66:0/2162160449 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4db402cb80 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 10 client.15870 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622bbb97820 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4da2ffd640 1 -- 172.21.15.66:0/2162160449 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db4048040 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4da2ffd640 1 -- 172.21.15.66:0/2162160449 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4db4034520 con 0x5622bbb17160 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 10 client.15870 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 10 client.15870 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T09:08:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 20 client.15870 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb1fce0 0x5622bbb9aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 4 client.15870 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.774+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "status"}) -- 0x5622bba823a0 con 0x5622bbb1fce0 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb1fce0 0x5622bbb9aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4da2ffd640 10 client.15870 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dc37fe640 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb1fce0 0x5622bbb9aa00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25e6396bc3ef45bf server_cookie=a394125c0e3517a in_seq=0 out_seq=0 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4da2ffd640 1 -- 172.21.15.66:0/2162160449 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5622bba823a0 con 0x5622bbb1fce0 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4da2ffd640 10 client.15870 handle_command_reply: tid=0 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 client.15870 shutdown 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 msgr2=0x5622bbb1a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1a3c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4db40059d0 tx=0x7f4db4045040 comp rx=0 tx=0).stop 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 shutdown_connections 2024-04-21T09:08:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622bbb1fce0 0x5622bbb9aa00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622bbb17160 0x5622bbb1a3c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4da2ffd640 0 client.15870 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622bbb167b0 0x5622bbb19e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 --2- 172.21.15.66:0/2162160449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622bbb10a60 0x5622bbb19940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 >> 172.21.15.66:0/2162160449 conn(0x5622bbb178b0 msgr2=0x5622bbb1b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 shutdown_connections 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.775+0000 7f4dcc987740 1 -- 172.21.15.66:0/2162160449 wait complete. 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dcc987740 20 client.15870 trim_cache size 0 max 16384 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:26.783 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 1, 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24490, 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 135.69505423000001, 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 25, 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 96, 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 95, 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 174.470366826 2024-04-21T09:08:26.784 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db00a3db0 msgr2=0x7f4db00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db00a3db0 0x7f4db00a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0ce6d0 tx=0x7f4dbc00efd0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4199fa0 msgr2=0x7f4dc41a4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4199fa0 0x7f4dc41a4550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00eb10 tx=0x7f4dbc00c2d0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 shutdown_connections 2024-04-21T09:08:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db00a3db0 0x7f4db00a6270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dc4199fa0 0x7f4dc41a4550 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dc4106170 0x7f4dc4199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 --2- 172.21.15.66:0/164688255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc41010a0 0x7f4dc41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 >> 172.21.15.66:0/164688255 conn(0x7f4dc4077eb0 msgr2=0x7f4dc4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 shutdown_connections 2024-04-21T09:08:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.776+0000 7f4dc9d96640 1 -- 172.21.15.66:0/164688255 wait complete. 2024-04-21T09:08:26.787 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T09:08:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 -- 172.21.15.66:0/3590585927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 msgr2=0x7f4dcc1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 --2- 172.21.15.66:0/3590585927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc1008d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009930 tx=0x7f4dc002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 -- 172.21.15.66:0/3590585927 shutdown_connections 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 --2- 172.21.15.66:0/3590585927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dcc100e10 0x7f4dcc10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 --2- 172.21.15.66:0/3590585927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc1008d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 --2- 172.21.15.66:0/3590585927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dcc106f30 0x7f4dcc107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 -- 172.21.15.66:0/3590585927 >> 172.21.15.66:0/3590585927 conn(0x7f4dcc077410 msgr2=0x7f4dcc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 -- 172.21.15.66:0/3590585927 shutdown_connections 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.890+0000 7f4dd2677640 1 -- 172.21.15.66:0/3590585927 wait complete. 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 Processor -- start 2024-04-21T09:08:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 -- start start 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dcc100e10 0x7f4dcc19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dcc106f30 0x7f4dcc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1122f0 con 0x7f4dcc100450 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112170 con 0x7f4dcc106f30 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd2677640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112470 con 0x7f4dcc100e10 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54682/0 (socket says 172.21.15.66:54682) 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 -- 172.21.15.66:0/370490731 learned_addr learned my addr 172.21.15.66:0/370490731 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dcc106f30 0x7f4dcc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dcc100e10 0x7f4dcc19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 -- 172.21.15.66:0/370490731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dcc100e10 msgr2=0x7f4dcc19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dcc100e10 0x7f4dcc19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 -- 172.21.15.66:0/370490731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dcc106f30 msgr2=0x7f4dcc198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dcc106f30 0x7f4dcc198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.891+0000 7f4dcbfff640 1 -- 172.21.15.66:0/370490731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc198d90 con 0x7f4dcc100450 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.892+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc19e460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00d950 tx=0x7f4dbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.892+0000 7f4dc97fa640 1 -- 172.21.15.66:0/370490731 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc017070 con 0x7f4dcc100450 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.892+0000 7f4dc97fa640 1 -- 172.21.15.66:0/370490731 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc009c30 con 0x7f4dcc100450 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.892+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc199080 con 0x7f4dcc100450 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.892+0000 7f4dc97fa640 1 -- 172.21.15.66:0/370490731 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00f620 con 0x7f4dcc100450 2024-04-21T09:08:26.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.892+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc1a5320 con 0x7f4dcc100450 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.894+0000 7f4dc97fa640 1 -- 172.21.15.66:0/370490731 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f4dbc010460 con 0x7f4dcc100450 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.894+0000 7f4dd5262740 1 Processor -- start 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.894+0000 7f4dd5262740 1 -- start start 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dc97fa640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db40a3e30 0x7f4db40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d4160 0x560b5f7dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7dcce0 0x560b5f7df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd5262740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560b5f737b10 con 0x560b5f7d4160 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd5262740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b5f6d5240 con 0x560b5f7d37b0 2024-04-21T09:08:26.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd5262740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560b5f733ad0 con 0x560b5f7dcce0 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d4160 0x560b5f7dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db40a3e30 0x7f4db40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dc97fa640 1 -- 172.21.15.66:0/370490731 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4dbc0d08f0 con 0x7f4dcc100450 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d4160 0x560b5f7dc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54684/0 (socket says 172.21.15.66:54684) 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 -- 172.21.15.66:0/3077454606 learned_addr learned my addr 172.21.15.66:0/3077454606 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7dcce0 0x560b5f7df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 -- 172.21.15.66:0/3077454606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7dcce0 msgr2=0x560b5f7df130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7dcce0 0x560b5f7df130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 -- 172.21.15.66:0/3077454606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 msgr2=0x560b5f7d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 -- 172.21.15.66:0/3077454606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5f44a2d0 con 0x560b5f7d4160 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db40a3e30 0x7f4db40a62f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009fd0 tx=0x7f4dc4013b80 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d4160 0x560b5f7dc710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0005ab0 tx=0x7f4dc0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5981db13565084 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4db27fc640 1 -- 172.21.15.66:0/3077454606 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0043020 con 0x560b5f7d4160 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4db27fc640 1 -- 172.21.15.66:0/3077454606 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc002c7e0 con 0x560b5f7d4160 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.895+0000 7f4db27fc640 1 -- 172.21.15.66:0/3077454606 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00053c0 con 0x560b5f7d4160 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 -- 172.21.15.66:0/3077454606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d4160 msgr2=0x560b5f7dc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d4160 0x560b5f7dc710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0005ab0 tx=0x7f4dc0005dd0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 -- 172.21.15.66:0/3077454606 shutdown_connections 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7dcce0 0x560b5f7df130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d4160 0x560b5f7dc710 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3077454606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 -- 172.21.15.66:0/3077454606 >> 172.21.15.66:0/3077454606 conn(0x560b5f7cda60 msgr2=0x560b5f7cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 -- 172.21.15.66:0/3077454606 shutdown_connections 2024-04-21T09:08:26.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 -- 172.21.15.66:0/3077454606 wait complete. 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.896+0000 7f4dd5262740 1 -- start start 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 1 Processor -- start 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7da130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7dfbf0 con 0x560b5f7dcce0 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b5f6c0190 con 0x560b5f7d37b0 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7dfd70 con 0x560b5f7d4160 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd5262740 10 client.0 _ll_register_callbacks cb 0x560b5f6bb8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7da670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54690/0 (socket says 172.21.15.66:54690) 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 -- 172.21.15.66:0/1135470229 learned_addr learned my addr 172.21.15.66:0/1135470229 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7da130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 -- 172.21.15.66:0/1135470229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 msgr2=0x560b5f7da130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7da130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 -- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 msgr2=0x560b5f7d9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dd0c14640 1 -- 172.21.15.66:0/1135470229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5f5146b0 con 0x560b5f7dcce0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.897+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7da670 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400ff10 tx=0x7f4dc4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8976f43e1b60ac78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dab7fe640 1 -- 172.21.15.66:0/1135470229 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4024030 con 0x560b5f7dcce0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dab7fe640 1 -- 172.21.15.66:0/1135470229 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4dc4004740 con 0x560b5f7dcce0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc002c7e0 con 0x560b5f7dcce0 2024-04-21T09:08:26.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dab7fe640 1 -- 172.21.15.66:0/1135470229 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4dc4031050 con 0x560b5f7dcce0 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dd5262740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b5f7ebed0 con 0x560b5f7dcce0 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dab7fe640 1 -- 172.21.15.66:0/1135470229 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc002c7e0 con 0x560b5f7dcce0 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dab7fe640 1 -- 172.21.15.66:0/1135470229 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4dc401f070 con 0x560b5f7dcce0 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dd5262740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.898+0000 7f4dd5262740 10 client.15882 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.899+0000 7f4dd5262740 20 client.15882 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.899+0000 7f4dd5262740 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f7ee5a0 0x560b5f7efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.899+0000 7f4dd5262740 4 client.15882 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.899+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b5f745530 con 0x560b5f7ee5a0 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.899+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f7ee5a0 0x560b5f7efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.899+0000 7f4dab7fe640 10 client.15882 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.899+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f7ee5a0 0x560b5f7efa10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fcf8044209ee60e server_cookie=5bc2ae3ba5707611 in_seq=0 out_seq=0 2024-04-21T09:08:26.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dab7fe640 1 -- 172.21.15.66:0/1135470229 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b5f745530 con 0x560b5f7ee5a0 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dab7fe640 10 client.15882 handle_command_reply: tid=0 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 client.15882 shutdown 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 msgr2=0x560b5f7da670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7da670 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400ff10 tx=0x7f4dc4004ab0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 shutdown_connections 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f7ee5a0 0x560b5f7efa10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7da670 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7da130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 --2- 172.21.15.66:0/1135470229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d37b0 0x560b5f7d9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dab7fe640 0 client.15882 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 >> 172.21.15.66:0/1135470229 conn(0x560b5f7cda60 msgr2=0x560b5f7cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 shutdown_connections 2024-04-21T09:08:26.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 1 -- 172.21.15.66:0/1135470229 wait complete. 2024-04-21T09:08:26.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.900+0000 7f4dd5262740 20 client.15882 trim_cache size 0 max 16384 2024-04-21T09:08:26.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 Processor -- start 2024-04-21T09:08:26.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 -- start start 2024-04-21T09:08:26.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d37b0 0x560b5f7dc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d4160 0x560b5f7eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7edd50 con 0x560b5f7dcce0 2024-04-21T09:08:26.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7ee420 con 0x560b5f7d4160 2024-04-21T09:08:26.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd5262740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7eeaf0 con 0x560b5f7d37b0 2024-04-21T09:08:26.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7d84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54698/0 (socket says 172.21.15.66:54698) 2024-04-21T09:08:26.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 -- 172.21.15.66:0/3062237087 learned_addr learned my addr 172.21.15.66:0/3062237087 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d37b0 0x560b5f7dc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 -- 172.21.15.66:0/3062237087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d37b0 msgr2=0x560b5f7dc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d37b0 0x560b5f7dc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 -- 172.21.15.66:0/3062237087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d4160 msgr2=0x560b5f7eded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d4160 0x560b5f7eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d4160 0x560b5f7eded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.919+0000 7f4dd0c14640 1 -- 172.21.15.66:0/3062237087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5f7d3f00 con 0x560b5f7dcce0 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7d84c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009420 tx=0x7f4dc4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f860c2454f6e419 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4db1ffb640 1 -- 172.21.15.66:0/3062237087 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4024070 con 0x560b5f7dcce0 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4db1ffb640 1 -- 172.21.15.66:0/3062237087 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4009e20 con 0x560b5f7dcce0 2024-04-21T09:08:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4db1ffb640 1 -- 172.21.15.66:0/3062237087 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4020690 con 0x560b5f7dcce0 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 -- 172.21.15.66:0/3062237087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 msgr2=0x560b5f7d84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7d84c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009420 tx=0x7f4dc4009a50 comp rx=0 tx=0).stop 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 -- 172.21.15.66:0/3062237087 shutdown_connections 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7dcce0 0x560b5f7d84c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7d4160 0x560b5f7eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3062237087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d37b0 0x560b5f7dc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 -- 172.21.15.66:0/3062237087 >> 172.21.15.66:0/3062237087 conn(0x560b5f7cda60 msgr2=0x560b5f7c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 -- 172.21.15.66:0/3062237087 shutdown_connections 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.920+0000 7f4dd5262740 1 -- 172.21.15.66:0/3062237087 wait complete. 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 -- start start 2024-04-21T09:08:26.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 Processor -- start 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d37b0 0x560b5f7d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7dcce0 0x560b5f7f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7ef890 con 0x560b5f7d37b0 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7ef1c0 con 0x560b5f7dcce0 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560b5f7eff60 con 0x560b5f7d4160 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd5262740 10 client.0 _ll_register_callbacks cb 0x7f4dc4012110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d37b0 0x560b5f7d6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d37b0 0x560b5f7d6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54702/0 (socket says 172.21.15.66:54702) 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dcbfff640 1 -- 172.21.15.66:0/3033766975 learned_addr learned my addr 172.21.15.66:0/3033766975 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.921+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7dcce0 0x560b5f7f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dcbfff640 1 -- 172.21.15.66:0/3033766975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 msgr2=0x560b5f7d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7d7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dcbfff640 1 -- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7dcce0 msgr2=0x560b5f7f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7dcce0 0x560b5f7f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dcbfff640 1 -- 172.21.15.66:0/3033766975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5f7f67a0 con 0x560b5f7d37b0 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd0c14640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7dcce0 0x560b5f7f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dcbfff640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d37b0 0x560b5f7d6d20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005fd0 tx=0x7f4dbc009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f13420cf68bd62 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dab7fe640 1 -- 172.21.15.66:0/3033766975 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc020040 con 0x560b5f7d37b0 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dab7fe640 1 -- 172.21.15.66:0/3033766975 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4dbc09a6b0 con 0x560b5f7d37b0 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc4009e20 con 0x560b5f7d37b0 2024-04-21T09:08:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dab7fe640 1 -- 172.21.15.66:0/3033766975 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc0a5020 con 0x560b5f7d37b0 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 10 client.15888 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b5f854cf0 con 0x560b5f7d37b0 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dab7fe640 1 -- 172.21.15.66:0/3033766975 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4009e20 con 0x560b5f7d37b0 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dab7fe640 1 -- 172.21.15.66:0/3033766975 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4dbc081ce0 con 0x560b5f7d37b0 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 10 client.15888 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 10 client.15888 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 20 client.15888 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f8598b0 0x560b5f8585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 4 client.15888 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.922+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "status"}) -- 0x560b5f73f3a0 con 0x560b5f8598b0 2024-04-21T09:08:26.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.923+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f8598b0 0x560b5f8585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.923+0000 7f4dab7fe640 10 client.15888 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.923+0000 7f4dcb7fe640 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f8598b0 0x560b5f8585c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8792105ceec6826 server_cookie=68bf09780912d860 in_seq=0 out_seq=0 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.923+0000 7f4dab7fe640 1 -- 172.21.15.66:0/3033766975 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560b5f73f3a0 con 0x560b5f8598b0 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.923+0000 7f4dab7fe640 10 client.15888 handle_command_reply: tid=0 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.923+0000 7f4dd5262740 1 client.15888 shutdown 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d37b0 msgr2=0x560b5f7d6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d37b0 0x560b5f7d6d20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005fd0 tx=0x7f4dbc009ea0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 shutdown_connections 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560b5f8598b0 0x560b5f8585c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b5f7dcce0 0x560b5f7f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b5f7d4160 0x560b5f7d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dab7fe640 0 client.15888 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:26.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 --2- 172.21.15.66:0/3033766975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b5f7d37b0 0x560b5f7d6d20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 >> 172.21.15.66:0/3033766975 conn(0x560b5f7cda60 msgr2=0x560b5f7d8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 shutdown_connections 2024-04-21T09:08:26.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 1 -- 172.21.15.66:0/3033766975 wait complete. 2024-04-21T09:08:26.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd5262740 20 client.15888 trim_cache size 0 max 16384 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0cca84f6-ffbd-11ee-bc93-c7b262605968", 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24512, 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 159.959570101, 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 25, 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 96, 2024-04-21T09:08:26.935 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 95, 2024-04-21T09:08:26.936 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 166.059434208 2024-04-21T09:08:26.936 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:26.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db40a3e30 msgr2=0x7f4db40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db40a3e30 0x7f4db40a62f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009fd0 tx=0x7f4dc4013b80 comp rx=0 tx=0).stop 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 msgr2=0x7f4dcc19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc19e460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00d950 tx=0x7f4dbc00de10 comp rx=0 tx=0).stop 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 shutdown_connections 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4db40a3e30 0x7f4db40a62f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dcc106f30 0x7f4dcc198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dcc100e10 0x7f4dcc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 --2- 172.21.15.66:0/370490731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4dcc100450 0x7f4dcc19e460 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 >> 172.21.15.66:0/370490731 conn(0x7f4dcc077410 msgr2=0x7f4dcc105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 shutdown_connections 2024-04-21T09:08:26.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:26.924+0000 7f4dd2677640 1 -- 172.21.15.66:0/370490731 wait complete. 2024-04-21T09:08:26.939 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 120 sec 2024-04-21T09:08:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:27 smithi186 ceph-mon[31505]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-04-21T09:08:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:27 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2346280358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:27.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:27 smithi083 ceph-mon[31413]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-04-21T09:08:27.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:27 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2346280358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:27 smithi066 ceph-mon[27953]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-04-21T09:08:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:27 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2346280358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:29 smithi186 ceph-mon[31505]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-04-21T09:08:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:29 smithi083 ceph-mon[31413]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-04-21T09:08:29.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:29 smithi066 ceph-mon[27953]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-04-21T09:08:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:31 smithi186 ceph-mon[31505]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-21T09:08:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:31 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:08:31.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:31 smithi083 ceph-mon[31413]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-21T09:08:31.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:31 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:08:31.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:31 smithi066 ceph-mon[27953]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-21T09:08:31.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:31 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:08:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 -- 172.21.15.66:0/3002965871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da4101aa0 msgr2=0x7f1da4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 --2- 172.21.15.66:0/3002965871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da4101aa0 0x7f1da4105a90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d9802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 -- 172.21.15.66:0/3002965871 shutdown_connections 2024-04-21T09:08:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 --2- 172.21.15.66:0/3002965871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1da4106170 0x7f1da410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 --2- 172.21.15.66:0/3002965871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da4101aa0 0x7f1da4105a90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 --2- 172.21.15.66:0/3002965871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1da41010f0 0x7f1da41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 -- 172.21.15.66:0/3002965871 >> 172.21.15.66:0/3002965871 conn(0x7f1da4077fd0 msgr2=0x7f1da40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 -- 172.21.15.66:0/3002965871 shutdown_connections 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.348+0000 7f1dac167640 1 -- 172.21.15.66:0/3002965871 wait complete. 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 Processor -- start 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 -- start start 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da41010f0 0x7f1da4102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1da4106170 0x7f1da4103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1da41035b0 0x7f1da41a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41110b0 con 0x7f1da41010f0 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4110f30 con 0x7f1da4106170 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1dac167640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4111230 con 0x7f1da41035b0 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da41010f0 0x7f1da4102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da41010f0 0x7f1da4102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54730/0 (socket says 172.21.15.66:54730) 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 -- 172.21.15.66:0/447549199 learned_addr learned my addr 172.21.15.66:0/447549199 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9702640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1da4106170 0x7f1da4103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1daa704640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1da41035b0 0x7f1da41a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 -- 172.21.15.66:0/447549199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1da41035b0 msgr2=0x7f1da41a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1da41035b0 0x7f1da41a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 -- 172.21.15.66:0/447549199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1da4106170 msgr2=0x7f1da4103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1da4106170 0x7f1da4103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1da9f03640 1 -- 172.21.15.66:0/447549199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da41a6ff0 con 0x7f1da41010f0 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.349+0000 7f1daa704640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1da41035b0 0x7f1da41a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.350+0000 7f1da9f03640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da41010f0 0x7f1da4102b30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400ebc0 tx=0x7f1d9400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.350+0000 7f1da2ffd640 1 -- 172.21.15.66:0/447549199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94010630 con 0x7f1da41010f0 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.350+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da41a72e0 con 0x7f1da41010f0 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.350+0000 7f1da2ffd640 1 -- 172.21.15.66:0/447549199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d94010c80 con 0x7f1da41010f0 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.350+0000 7f1da2ffd640 1 -- 172.21.15.66:0/447549199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94018b60 con 0x7f1da41010f0 2024-04-21T09:08:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.350+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41a7800 con 0x7f1da41010f0 2024-04-21T09:08:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.352+0000 7f1da2ffd640 1 -- 172.21.15.66:0/447549199 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f1d94018d00 con 0x7f1da41010f0 2024-04-21T09:08:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 Processor -- start 2024-04-21T09:08:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 -- start start 2024-04-21T09:08:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff1a8d0 0x55783ff19620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1da2ffd640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1d8c0a3e90 0x7f1d8c0a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff19fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55783fe7e8d0 con 0x55783ff19b60 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55783fe1be80 con 0x55783ff230e0 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daee36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55783fe7a890 con 0x55783ff1a8d0 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1da9702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff19fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1daa704640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1d8c0a3e90 0x7f1d8c0a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1da2ffd640 1 -- 172.21.15.66:0/447549199 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1d940d1350 con 0x7f1da41010f0 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1da9702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff19fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54742/0 (socket says 172.21.15.66:54742) 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1da9702640 1 -- 172.21.15.66:0/1539981727 learned_addr learned my addr 172.21.15.66:0/1539981727 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.353+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff1a8d0 0x55783ff19620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1daa704640 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1da9702640 1 -- 172.21.15.66:0/1539981727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff1a8d0 msgr2=0x55783ff19620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1da9702640 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff1a8d0 0x55783ff19620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1da9702640 1 -- 172.21.15.66:0/1539981727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 msgr2=0x55783ff254d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1da9702640 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff254d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1da9702640 1 -- 172.21.15.66:0/1539981727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783fe3d320 con 0x55783ff19b60 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff1a8d0 0x55783ff19620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1daa704640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1d8c0a3e90 0x7f1d8c0a6350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009f60 tx=0x7f1d9c009f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1da9702640 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff19fe0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d980095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da636b94010427 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1d8bfff640 1 -- 172.21.15.66:0/1539981727 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98045020 con 0x55783ff19b60 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1d8bfff640 1 -- 172.21.15.66:0/1539981727 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9802c7e0 con 0x55783ff19b60 2024-04-21T09:08:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.354+0000 7f1d8bfff640 1 -- 172.21.15.66:0/1539981727 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98005450 con 0x55783ff19b60 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 -- 172.21.15.66:0/1539981727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 msgr2=0x55783ff19fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff19fe0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d980095f0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 -- 172.21.15.66:0/1539981727 shutdown_connections 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff19fe0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 --2- 172.21.15.66:0/1539981727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff1a8d0 0x55783ff19620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 -- 172.21.15.66:0/1539981727 >> 172.21.15.66:0/1539981727 conn(0x55783fee0c90 msgr2=0x55783ff14850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 -- 172.21.15.66:0/1539981727 shutdown_connections 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 -- 172.21.15.66:0/1539981727 wait complete. 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.355+0000 7f1daee36740 1 -- start start 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 1 Processor -- start 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff22a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff1a8d0 0x55783ff1fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff230e0 0x55783ff203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55783ff25db0 con 0x55783ff19b60 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55783fe06eb0 con 0x55783ff1a8d0 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55783ff25f30 con 0x55783ff230e0 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daee36740 10 client.0 _ll_register_callbacks cb 0x55783fe33b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff22a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff22a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54750/0 (socket says 172.21.15.66:54750) 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 -- 172.21.15.66:0/1352143132 learned_addr learned my addr 172.21.15.66:0/1352143132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daa704640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff230e0 0x55783ff203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9702640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff1a8d0 0x55783ff1fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 -- 172.21.15.66:0/1352143132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff230e0 msgr2=0x55783ff203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff230e0 0x55783ff203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 -- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff1a8d0 msgr2=0x55783ff1fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff1a8d0 0x55783ff1fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9f03640 1 -- 172.21.15.66:0/1352143132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783fb91210 con 0x55783ff19b60 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1da9702640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff1a8d0 0x55783ff1fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.356+0000 7f1daa704640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff230e0 0x55783ff203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff22a90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1d9409cda0 tx=0x7f1d9409ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9226612362cfa038 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1d897fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1d897fa640 1 -- 172.21.15.66:0/1352143132 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9409d510 con 0x55783ff19b60 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1d897fa640 1 -- 172.21.15.66:0/1352143132 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1d940ceb30 con 0x55783ff19b60 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9802c7e0 con 0x55783ff19b60 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1d897fa640 1 -- 172.21.15.66:0/1352143132 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d9409ce10 con 0x55783ff19b60 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 10 client.15900 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55783fc5d620 con 0x55783ff19b60 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1d897fa640 1 -- 172.21.15.66:0/1352143132 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9802c7e0 con 0x55783ff19b60 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1d897fa640 1 -- 172.21.15.66:0/1352143132 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1d940ce080 con 0x55783ff19b60 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 20 client.15900 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff33090 0x55783ff354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 4 client.15900 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.357+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55783fe92880 con 0x55783ff33090 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.358+0000 7f1da9702640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff33090 0x55783ff354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.358+0000 7f1da9702640 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff33090 0x55783ff354a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a46b9300935d6f3 server_cookie=2a046c8368cf56e9 in_seq=0 out_seq=0 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.358+0000 7f1d897fa640 10 client.15900 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1d897fa640 1 -- 172.21.15.66:0/1352143132 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55783fe92880 con 0x55783ff33090 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1d897fa640 10 client.15900 handle_command_reply: tid=0 2024-04-21T09:08:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 client.15900 shutdown 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 msgr2=0x55783ff22a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff22a90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1d9409cda0 tx=0x7f1d9409ca10 comp rx=0 tx=0).stop 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 shutdown_connections 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff33090 0x55783ff354a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff230e0 0x55783ff203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff1a8d0 0x55783ff1fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1d897fa640 0 client.15900 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 --2- 172.21.15.66:0/1352143132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff19b60 0x55783ff22a90 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 >> 172.21.15.66:0/1352143132 conn(0x55783fee0c90 msgr2=0x55783ff165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 shutdown_connections 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 1 -- 172.21.15.66:0/1352143132 wait complete. 2024-04-21T09:08:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.359+0000 7f1daee36740 20 client.15900 trim_cache size 0 max 16384 2024-04-21T09:08:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.374+0000 7f1daee36740 1 Processor -- start 2024-04-21T09:08:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.374+0000 7f1daee36740 1 -- start start 2024-04-21T09:08:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.374+0000 7f1daee36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.374+0000 7f1daee36740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff22550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1daee36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff22a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1daee36740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b3ca0 con 0x55783ff1a8d0 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1daee36740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b1570 con 0x55783ff230e0 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1daee36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41af3a0 con 0x55783ff19b60 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff22550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff22550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54758/0 (socket says 172.21.15.66:54758) 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 -- 172.21.15.66:0/2856554476 learned_addr learned my addr 172.21.15.66:0/2856554476 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9f03640 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1daa704640 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff22a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 -- 172.21.15.66:0/2856554476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 msgr2=0x55783ff2ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff2ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 -- 172.21.15.66:0/2856554476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 msgr2=0x55783ff22a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff22a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 -- 172.21.15.66:0/2856554476 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783ff1b8f0 con 0x55783ff1a8d0 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1da9702640 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff22550 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d513431872ea59a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1d8b7fe640 1 -- 172.21.15.66:0/2856554476 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98045040 con 0x55783ff1a8d0 2024-04-21T09:08:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1d8b7fe640 1 -- 172.21.15.66:0/2856554476 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9802c7e0 con 0x55783ff1a8d0 2024-04-21T09:08:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.375+0000 7f1d8b7fe640 1 -- 172.21.15.66:0/2856554476 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d980512c0 con 0x55783ff1a8d0 2024-04-21T09:08:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 -- 172.21.15.66:0/2856554476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 msgr2=0x55783ff22550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff22550 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d98000bc0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 -- 172.21.15.66:0/2856554476 shutdown_connections 2024-04-21T09:08:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff22a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff22550 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 --2- 172.21.15.66:0/2856554476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 -- 172.21.15.66:0/2856554476 >> 172.21.15.66:0/2856554476 conn(0x55783fee0c90 msgr2=0x55783ff14380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 -- 172.21.15.66:0/2856554476 shutdown_connections 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.376+0000 7f1daee36740 1 -- 172.21.15.66:0/2856554476 wait complete. 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 -- start start 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 Processor -- start 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff23e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41afa30 con 0x55783ff1a8d0 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b0ea0 con 0x55783ff230e0 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b0100 con 0x55783ff19b60 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daee36740 10 client.0 _ll_register_callbacks cb 0x7f1d9400eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1da9702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1da9702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff243a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54766/0 (socket says 172.21.15.66:54766) 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1da9702640 1 -- 172.21.15.66:0/1967969167 learned_addr learned my addr 172.21.15.66:0/1967969167 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1daa704640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.377+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff23e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1da9702640 1 -- 172.21.15.66:0/1967969167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 msgr2=0x55783ff23e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1da9702640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff23e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1da9702640 1 -- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 msgr2=0x55783ff248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1da9702640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1da9702640 1 -- 172.21.15.66:0/1967969167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783ff297a0 con 0x55783ff1a8d0 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff23e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1da9702640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff243a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1d9802c170 tx=0x7f1d9802c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598c2dd09df893de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1d897fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1d897fa640 1 -- 172.21.15.66:0/1967969167 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98045070 con 0x55783ff1a8d0 2024-04-21T09:08:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1d897fa640 1 -- 172.21.15.66:0/1967969167 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1d98005410 con 0x55783ff1a8d0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9802c7e0 con 0x55783ff1a8d0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1d897fa640 1 -- 172.21.15.66:0/1967969167 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d98005610 con 0x55783ff1a8d0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1daee36740 10 client.15906 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55783ff99a20 con 0x55783ff1a8d0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.378+0000 7f1d897fa640 1 -- 172.21.15.66:0/1967969167 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9802c7e0 con 0x55783ff1a8d0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1d897fa640 1 -- 172.21.15.66:0/1967969167 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1d98041480 con 0x55783ff1a8d0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1daee36740 10 client.15906 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1daee36740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1daee36740 20 client.15906 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1daee36740 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff99cf0 0x55783ff9c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1daee36740 4 client.15906 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55783fe1e3d0 con 0x55783ff99cf0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff99cf0 0x55783ff9c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1d897fa640 10 client.15906 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.379+0000 7f1da9f03640 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff99cf0 0x55783ff9c0e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d1e0f25ec718490 server_cookie=79201ec0f3076b47 in_seq=0 out_seq=0 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1d897fa640 1 -- 172.21.15.66:0/1967969167 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55783fe1e3d0 con 0x55783ff99cf0 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1d897fa640 10 client.15906 handle_command_reply: tid=0 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 client.15906 shutdown 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 msgr2=0x55783ff243a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff243a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1d9802c170 tx=0x7f1d9802c610 comp rx=0 tx=0).stop 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 shutdown_connections 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55783ff99cf0 0x55783ff9c0e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55783ff230e0 0x55783ff248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55783ff1a8d0 0x55783ff243a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1d897fa640 0 client.15906 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 --2- 172.21.15.66:0/1967969167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55783ff19b60 0x55783ff23e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 >> 172.21.15.66:0/1967969167 conn(0x55783fee0c90 msgr2=0x55783ff1e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 shutdown_connections 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 1 -- 172.21.15.66:0/1967969167 wait complete. 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.392+0000 7f1daee36740 20 client.15906 trim_cache size 0 max 16384 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:08:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:08:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:08:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:08:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1d8c0a3e90 msgr2=0x7f1d8c0a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1d8c0a3e90 0x7f1d8c0a6350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009f60 tx=0x7f1d9c009f90 comp rx=0 tx=0).stop 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da41010f0 msgr2=0x7f1da4102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da41010f0 0x7f1da4102b30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400ebc0 tx=0x7f1d9400c6a0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 shutdown_connections 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1d8c0a3e90 0x7f1d8c0a6350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1da41035b0 0x7f1da41a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1da4106170 0x7f1da4103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 --2- 172.21.15.66:0/447549199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1da41010f0 0x7f1da4102b30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 >> 172.21.15.66:0/447549199 conn(0x7f1da4077fd0 msgr2=0x7f1da40699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 shutdown_connections 2024-04-21T09:08:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:32.393+0000 7f1dac167640 1 -- 172.21.15.66:0/447549199 wait complete. 2024-04-21T09:08:33.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:33 smithi186 ceph-mon[31505]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-21T09:08:33.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:33 smithi083 ceph-mon[31413]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-21T09:08:33.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:33 smithi066 ceph-mon[27953]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-21T09:08:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:35 smithi186 ceph-mon[31505]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-21T09:08:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:35 smithi083 ceph-mon[31413]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-21T09:08:35.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:35 smithi066 ceph-mon[27953]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-21T09:08:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:37 smithi186 ceph-mon[31505]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-21T09:08:37.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:37 smithi083 ceph-mon[31413]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-21T09:08:37.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:37 smithi066 ceph-mon[27953]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-21T09:08:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:39 smithi186 ceph-mon[31505]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-21T09:08:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:39 smithi083 ceph-mon[31413]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-21T09:08:39.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:39 smithi066 ceph-mon[27953]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-21T09:08:41.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:41 smithi186 ceph-mon[31505]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-21T09:08:41.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:41 smithi083 ceph-mon[31413]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-21T09:08:41.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:41 smithi066 ceph-mon[27953]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-21T09:08:43.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:43 smithi186 ceph-mon[31505]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T09:08:43.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:43 smithi083 ceph-mon[31413]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T09:08:43.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:43 smithi066 ceph-mon[27953]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T09:08:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 -- 172.21.15.66:0/2312643022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff494101aa0 msgr2=0x7ff494105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 --2- 172.21.15.66:0/2312643022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff494101aa0 0x7ff494105a90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff488009900 tx=0x7ff48802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 -- 172.21.15.66:0/2312643022 shutdown_connections 2024-04-21T09:08:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 --2- 172.21.15.66:0/2312643022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff494106170 0x7ff49410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 --2- 172.21.15.66:0/2312643022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff494101aa0 0x7ff494105a90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 --2- 172.21.15.66:0/2312643022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4941010f0 0x7ff4941014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 -- 172.21.15.66:0/2312643022 >> 172.21.15.66:0/2312643022 conn(0x7ff494077fd0 msgr2=0x7ff4940783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 -- 172.21.15.66:0/2312643022 shutdown_connections 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.525+0000 7ff499946640 1 -- 172.21.15.66:0/2312643022 wait complete. 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 Processor -- start 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 -- start start 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4941010f0 0x7ff4941986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff494106170 0x7ff494198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff49419cf90 0x7ff49419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941121f0 con 0x7ff4941010f0 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff494112070 con 0x7ff494106170 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff499946640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff494112370 con 0x7ff49419cf90 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4941010f0 0x7ff4941986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4941010f0 0x7ff4941986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50670/0 (socket says 172.21.15.66:50670) 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff492ffd640 1 -- 172.21.15.66:0/2682625132 learned_addr learned my addr 172.21.15.66:0/2682625132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.526+0000 7ff4927fc640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff494106170 0x7ff494198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff4937fe640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff49419cf90 0x7ff49419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff492ffd640 1 -- 172.21.15.66:0/2682625132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff49419cf90 msgr2=0x7ff49419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff492ffd640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff49419cf90 0x7ff49419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff492ffd640 1 -- 172.21.15.66:0/2682625132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff494106170 msgr2=0x7ff494198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff492ffd640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff494106170 0x7ff494198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff492ffd640 1 -- 172.21.15.66:0/2682625132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49419da80 con 0x7ff4941010f0 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff492ffd640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4941010f0 0x7ff4941986d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff48400e9f0 tx=0x7ff48400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff498944640 1 -- 172.21.15.66:0/2682625132 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48400cd80 con 0x7ff4941010f0 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff498944640 1 -- 172.21.15.66:0/2682625132 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4840054d0 con 0x7ff4941010f0 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49419dd70 con 0x7ff4941010f0 2024-04-21T09:08:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff498944640 1 -- 172.21.15.66:0/2682625132 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484010690 con 0x7ff4941010f0 2024-04-21T09:08:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.527+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49410e8f0 con 0x7ff4941010f0 2024-04-21T09:08:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff498944640 1 -- 172.21.15.66:0/2682625132 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7ff484004e90 con 0x7ff4941010f0 2024-04-21T09:08:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 Processor -- start 2024-04-21T09:08:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 -- start start 2024-04-21T09:08:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb5ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb600ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5630cb5518d0 con 0x5630cb5ecb60 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5630cb4eee80 con 0x5630cb5ed8d0 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.530+0000 7ff49c60c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5630cb54d890 con 0x5630cb5fe8f0 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb5ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb5ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50682/0 (socket says 172.21.15.66:50682) 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff492ffd640 1 -- 172.21.15.66:0/972030992 learned_addr learned my addr 172.21.15.66:0/972030992 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4937fe640 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4927fc640 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb600ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff498944640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff47c0a3ee0 0x7ff47c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4937fe640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff47c0a3ee0 0x7ff47c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4927fc640 1 -- 172.21.15.66:0/972030992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 msgr2=0x5630cb5ec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4927fc640 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5ec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4927fc640 1 -- 172.21.15.66:0/972030992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 msgr2=0x5630cb5ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4927fc640 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb5ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff4927fc640 1 -- 172.21.15.66:0/972030992 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630cb510320 con 0x5630cb5fe8f0 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff498944640 1 -- 172.21.15.66:0/2682625132 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff4840d10c0 con 0x7ff4941010f0 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.531+0000 7ff492ffd640 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb5ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.532+0000 7ff4927fc640 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb600ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff488036040 tx=0x7ff488005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f09f711524f83806 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.532+0000 7ff4937fe640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff47c0a3ee0 0x7ff47c0a63a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005fd0 tx=0x7ff48c004ab0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.532+0000 7ff479ffb640 1 -- 172.21.15.66:0/972030992 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488045020 con 0x5630cb5fe8f0 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.532+0000 7ff479ffb640 1 -- 172.21.15.66:0/972030992 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48802c7e0 con 0x5630cb5fe8f0 2024-04-21T09:08:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.532+0000 7ff479ffb640 1 -- 172.21.15.66:0/972030992 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4880055c0 con 0x5630cb5fe8f0 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.532+0000 7ff49c60c740 1 -- 172.21.15.66:0/972030992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 msgr2=0x5630cb600ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb600ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff488036040 tx=0x7ff488005bf0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 -- 172.21.15.66:0/972030992 shutdown_connections 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb600ce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb5ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 --2- 172.21.15.66:0/972030992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 -- 172.21.15.66:0/972030992 >> 172.21.15.66:0/972030992 conn(0x5630cb5b3c90 msgr2=0x5630cb5e7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 -- 172.21.15.66:0/972030992 shutdown_connections 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 -- 172.21.15.66:0/972030992 wait complete. 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 -- start start 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.533+0000 7ff49c60c740 1 Processor -- start 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ecb60 0x5630cb5fe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff49c60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5630cb6015c0 con 0x5630cb5fe8f0 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff49c60c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5630cb4d9eb0 con 0x5630cb5ed8d0 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff49c60c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5630cb601740 con 0x5630cb5ecb60 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff49c60c740 10 client.0 _ll_register_callbacks cb 0x5630cb506b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ecb60 0x5630cb5fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ecb60 0x5630cb5fe280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:45274/0 (socket says 172.21.15.66:45274) 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 -- 172.21.15.66:0/4207302702 learned_addr learned my addr 172.21.15.66:0/4207302702 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff4937fe640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff4927fc640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 -- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 msgr2=0x5630cb5fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 -- 172.21.15.66:0/4207302702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5fe8f0 msgr2=0x5630cb5fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff492ffd640 1 -- 172.21.15.66:0/4207302702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630cb264210 con 0x5630cb5ecb60 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.534+0000 7ff4937fe640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff492ffd640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ecb60 0x5630cb5fe280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c6a0 tx=0x7ff4840ce3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e9b17e7f626ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff472ffd640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff472ffd640 1 -- 172.21.15.66:0/4207302702 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48409c4e0 con 0x5630cb5ecb60 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff472ffd640 1 -- 172.21.15.66:0/4207302702 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff4840cd7f0 con 0x5630cb5ecb60 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48802c7e0 con 0x5630cb5ecb60 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff472ffd640 1 -- 172.21.15.66:0/4207302702 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff484005180 con 0x5630cb5ecb60 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff49c60c740 10 client.25447 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630cb330620 con 0x5630cb5ecb60 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.535+0000 7ff472ffd640 1 -- 172.21.15.66:0/4207302702 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48802c7e0 con 0x5630cb5ecb60 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff472ffd640 1 -- 172.21.15.66:0/4207302702 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff4840cdb70 con 0x5630cb5ecb60 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff49c60c740 10 client.25447 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff49c60c740 10 client.25447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff49c60c740 20 client.25447 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff49c60c740 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb60eda0 0x5630cb611210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff49c60c740 4 client.25447 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630cb565880 con 0x5630cb60eda0 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff4927fc640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb60eda0 0x5630cb611210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff4927fc640 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb60eda0 0x5630cb611210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503a56d08b4b22a9 server_cookie=6a2bd4441e4bdcce in_seq=0 out_seq=0 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.536+0000 7ff472ffd640 10 client.25447 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.537+0000 7ff472ffd640 1 -- 172.21.15.66:0/4207302702 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630cb565880 con 0x5630cb60eda0 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.537+0000 7ff472ffd640 10 client.25447 handle_command_reply: tid=0 2024-04-21T09:08:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.537+0000 7ff49c60c740 1 client.25447 shutdown 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.537+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ecb60 msgr2=0x5630cb5fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.537+0000 7ff49c60c740 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ecb60 0x5630cb5fe280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c6a0 tx=0x7ff4840ce3b0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.537+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 shutdown_connections 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.537+0000 7ff49c60c740 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb60eda0 0x5630cb611210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff49c60c740 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff49c60c740 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb5fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff49c60c740 1 --2- 172.21.15.66:0/4207302702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ecb60 0x5630cb5fe280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 >> 172.21.15.66:0/4207302702 conn(0x5630cb5b3c90 msgr2=0x5630cb5e95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff472ffd640 0 client.25447 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 shutdown_connections 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff49c60c740 1 -- 172.21.15.66:0/4207302702 wait complete. 2024-04-21T09:08:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.538+0000 7ff49c60c740 20 client.25447 trim_cache size 0 max 16384 2024-04-21T09:08:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 Processor -- start 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 -- start start 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb60a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb610db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941af820 con 0x5630cb5ecb60 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941ad0f0 con 0x5630cb5ed8d0 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff49c60c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941aaf20 con 0x5630cb5fe8f0 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb60a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb60a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50702/0 (socket says 172.21.15.66:50702) 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff492ffd640 1 -- 172.21.15.66:0/1094758162 learned_addr learned my addr 172.21.15.66:0/1094758162 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.553+0000 7ff4927fc640 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb610db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff4937fe640 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff492ffd640 1 -- 172.21.15.66:0/1094758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 msgr2=0x5630cb5fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff492ffd640 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff492ffd640 1 -- 172.21.15.66:0/1094758162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 msgr2=0x5630cb610db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff492ffd640 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb610db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff492ffd640 1 -- 172.21.15.66:0/1094758162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630cb611580 con 0x5630cb5ecb60 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff4937fe640 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff4927fc640 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb610db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff492ffd640 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb60a4c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff484098f90 tx=0x7ff4840ce320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbc74351bf15a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff4797fa640 1 -- 172.21.15.66:0/1094758162 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48409c4e0 con 0x5630cb5ecb60 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff4797fa640 1 -- 172.21.15.66:0/1094758162 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4840a9070 con 0x5630cb5ecb60 2024-04-21T09:08:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.554+0000 7ff4797fa640 1 -- 172.21.15.66:0/1094758162 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48409c950 con 0x5630cb5ecb60 2024-04-21T09:08:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 -- 172.21.15.66:0/1094758162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 msgr2=0x5630cb60a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb60a4c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff484098f90 tx=0x7ff4840ce320 comp rx=0 tx=0).stop 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 -- 172.21.15.66:0/1094758162 shutdown_connections 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5fe8f0 0x5630cb5fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5ed8d0 0x5630cb610db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 --2- 172.21.15.66:0/1094758162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb60a4c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 -- 172.21.15.66:0/1094758162 >> 172.21.15.66:0/1094758162 conn(0x5630cb5b3c90 msgr2=0x5630cb5f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 -- 172.21.15.66:0/1094758162 shutdown_connections 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 -- 172.21.15.66:0/1094758162 wait complete. 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.555+0000 7ff49c60c740 1 -- start start 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 1 Processor -- start 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb606f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ed8d0 0x5630cb5f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5fe8f0 0x5630cb5f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941ab5b0 con 0x5630cb5ecb60 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb606f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ed8d0 0x5630cb5f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941aca20 con 0x5630cb5fe8f0 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941abc80 con 0x5630cb5ed8d0 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ed8d0 0x5630cb5f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:45292/0 (socket says 172.21.15.66:45292) 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 -- 172.21.15.66:0/836600686 learned_addr learned my addr 172.21.15.66:0/836600686 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff49c60c740 10 client.0 _ll_register_callbacks cb 0x7ff48400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4937fe640 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5fe8f0 0x5630cb5f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 -- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5fe8f0 msgr2=0x5630cb5f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5fe8f0 0x5630cb5f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 -- 172.21.15.66:0/836600686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 msgr2=0x5630cb606f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb606f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.556+0000 7ff4927fc640 1 -- 172.21.15.66:0/836600686 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630cb5f7e00 con 0x5630cb5ed8d0 2024-04-21T09:08:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff4927fc640 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ed8d0 0x5630cb5f8db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff488044040 tx=0x7ff488004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=658c07e102841a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff472ffd640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff472ffd640 1 -- 172.21.15.66:0/836600686 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488057060 con 0x5630cb5ed8d0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff472ffd640 1 -- 172.21.15.66:0/836600686 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff488052dc0 con 0x5630cb5ed8d0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4840a9070 con 0x5630cb5ed8d0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff472ffd640 1 -- 172.21.15.66:0/836600686 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff48805e050 con 0x5630cb5ed8d0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff49c60c740 10 client.25450 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630cb674910 con 0x5630cb5ed8d0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.557+0000 7ff472ffd640 1 -- 172.21.15.66:0/836600686 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4840a9070 con 0x5630cb5ed8d0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff472ffd640 1 -- 172.21.15.66:0/836600686 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff4840cd470 con 0x5630cb5ed8d0 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff49c60c740 10 client.25450 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff49c60c740 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff49c60c740 20 client.25450 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff49c60c740 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb674c50 0x5630cb677040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff49c60c740 4 client.25450 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:08:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630cb4f13d0 con 0x5630cb674c50 2024-04-21T09:08:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff492ffd640 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb674c50 0x5630cb677040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff472ffd640 10 client.25450 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.558+0000 7ff492ffd640 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb674c50 0x5630cb677040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aea9b4635480e45 server_cookie=306506ff2fc937a6 in_seq=0 out_seq=0 2024-04-21T09:08:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff472ffd640 1 -- 172.21.15.66:0/836600686 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630cb4f13d0 con 0x5630cb674c50 2024-04-21T09:08:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff472ffd640 10 client.25450 handle_command_reply: tid=0 2024-04-21T09:08:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 client.25450 shutdown 2024-04-21T09:08:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ed8d0 msgr2=0x5630cb5f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ed8d0 0x5630cb5f8db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff488044040 tx=0x7ff488004560 comp rx=0 tx=0).stop 2024-04-21T09:08:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 shutdown_connections 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5630cb674c50 0x5630cb677040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff472ffd640 0 client.25450 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb5fe8f0 0x5630cb5f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630cb5ed8d0 0x5630cb5f8db0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 --2- 172.21.15.66:0/836600686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5630cb5ecb60 0x5630cb606f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 >> 172.21.15.66:0/836600686 conn(0x5630cb5b3c90 msgr2=0x5630cb5e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 shutdown_connections 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.562+0000 7ff49c60c740 1 -- 172.21.15.66:0/836600686 wait complete. 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff49c60c740 20 client.25450 trim_cache size 0 max 16384 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:08:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:08:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:08:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:08:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff47c0a3ee0 msgr2=0x7ff47c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff47c0a3ee0 0x7ff47c0a63a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005fd0 tx=0x7ff48c004ab0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4941010f0 msgr2=0x7ff4941986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4941010f0 0x7ff4941986d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff48400e9f0 tx=0x7ff48400edf0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 shutdown_connections 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff47c0a3ee0 0x7ff47c0a63a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff49419cf90 0x7ff49419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff494106170 0x7ff494198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 --2- 172.21.15.66:0/2682625132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4941010f0 0x7ff4941986d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 >> 172.21.15.66:0/2682625132 conn(0x7ff494077fd0 msgr2=0x7ff4940fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 shutdown_connections 2024-04-21T09:08:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:45.563+0000 7ff499946640 1 -- 172.21.15.66:0/2682625132 wait complete. 2024-04-21T09:08:45.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:45 smithi186 ceph-mon[31505]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T09:08:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:45 smithi083 ceph-mon[31413]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T09:08:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:45 smithi066 ceph-mon[27953]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T09:08:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:47 smithi186 ceph-mon[31505]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-21T09:08:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:47 smithi083 ceph-mon[31413]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-21T09:08:47.622 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:47 smithi066 ceph-mon[27953]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-21T09:08:47.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:08:47.902 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:08:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 -- 172.21.15.66:0/1828609466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844101080 msgr2=0x7f0844101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1828609466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844101080 0x7f0844101460 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0834009900 tx=0x7f083402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 -- 172.21.15.66:0/1828609466 shutdown_connections 2024-04-21T09:08:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1828609466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0844106150 0x7f084410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1828609466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0844101a30 0x7f0844105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1828609466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844101080 0x7f0844101460 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 -- 172.21.15.66:0/1828609466 >> 172.21.15.66:0/1828609466 conn(0x7f0844078000 msgr2=0x7f0844078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 -- 172.21.15.66:0/1828609466 shutdown_connections 2024-04-21T09:08:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.015+0000 7f084a7b2640 1 -- 172.21.15.66:0/1828609466 wait complete. 2024-04-21T09:08:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 Processor -- start 2024-04-21T09:08:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 -- start start 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0844101080 0x7f0844198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844106150 0x7f0844198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084419cfe0 0x7f084419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08441110b0 con 0x7f0844106150 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0844110f30 con 0x7f084419cfe0 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f084a7b2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0844111230 con 0x7f0844101080 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f08437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844106150 0x7f0844198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f08437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844106150 0x7f0844198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50724/0 (socket says 172.21.15.66:50724) 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.016+0000 7f08437fe640 1 -- 172.21.15.66:0/1939067073 learned_addr learned my addr 172.21.15.66:0/1939067073 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f0843fff640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0844101080 0x7f0844198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f0848d4f640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084419cfe0 0x7f084419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08437fe640 1 -- 172.21.15.66:0/1939067073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0844101080 msgr2=0x7f0844198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08437fe640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0844101080 0x7f0844198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08437fe640 1 -- 172.21.15.66:0/1939067073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084419cfe0 msgr2=0x7f084419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08437fe640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084419cfe0 0x7f084419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08437fe640 1 -- 172.21.15.66:0/1939067073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084419dad0 con 0x7f0844106150 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f0843fff640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0844101080 0x7f0844198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f0848d4f640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084419cfe0 0x7f084419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08437fe640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844106150 0x7f0844198c60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f083800d740 tx=0x7f083800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08417fa640 1 -- 172.21.15.66:0/1939067073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838017070 con 0x7f0844106150 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084419ddc0 con 0x7f0844106150 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08417fa640 1 -- 172.21.15.66:0/1939067073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800de50 con 0x7f0844106150 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f08417fa640 1 -- 172.21.15.66:0/1939067073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838004ce0 con 0x7f0844106150 2024-04-21T09:08:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.017+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08441a55b0 con 0x7f0844106150 2024-04-21T09:08:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 Processor -- start 2024-04-21T09:08:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 -- start start 2024-04-21T09:08:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdf580 0x55a2e4cdf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ceae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a2e4c441e0 con 0x55a2e4cdf580 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a2e4be1390 con 0x55a2e4cdfea0 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f084d39f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a2e4c401a0 con 0x55a2e4ce8a80 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f0848d4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdf580 0x55a2e4cdf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f0848d4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdf580 0x55a2e4cdf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50734/0 (socket says 172.21.15.66:50734) 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f0848d4f640 1 -- 172.21.15.66:0/746885717 learned_addr learned my addr 172.21.15.66:0/746885717 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.020+0000 7f0843fff640 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f0848d4f640 1 -- 172.21.15.66:0/746885717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 msgr2=0x55a2e4ceae70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f0848d4f640 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ceae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f0848d4f640 1 -- 172.21.15.66:0/746885717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 msgr2=0x55a2e4ce8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f0848d4f640 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f0848d4f640 1 -- 172.21.15.66:0/746885717 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e4a207d0 con 0x55a2e4cdf580 2024-04-21T09:08:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f08417fa640 1 -- 172.21.15.66:0/1939067073 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f0838004e80 con 0x7f0844106150 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f0848d4f640 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdf580 0x55a2e4cdf960 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f083c009fd0 tx=0x7f083c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b208510037dd2dce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f082a7fc640 1 -- 172.21.15.66:0/746885717 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c011700 con 0x55a2e4cdf580 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f082a7fc640 1 -- 172.21.15.66:0/746885717 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c011d50 con 0x55a2e4cdf580 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.021+0000 7f082a7fc640 1 -- 172.21.15.66:0/746885717 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c00e3c0 con 0x55a2e4cdf580 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f08417fa640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f082c0a3db0 0x7f082c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f08417fa640 1 -- 172.21.15.66:0/1939067073 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f08380d07b0 con 0x7f0844106150 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f0843fff640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f082c0a3db0 0x7f082c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f084d39f740 1 -- 172.21.15.66:0/746885717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdf580 msgr2=0x55a2e4cdf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f084d39f740 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdf580 0x55a2e4cdf960 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f083c009fd0 tx=0x7f083c010860 comp rx=0 tx=0).stop 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f0843fff640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f082c0a3db0 0x7f082c0a6270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0834000c00 tx=0x7f08340058b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f084d39f740 1 -- 172.21.15.66:0/746885717 shutdown_connections 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f084d39f740 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ceae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f084d39f740 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f084d39f740 1 --2- 172.21.15.66:0/746885717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdf580 0x55a2e4cdf960 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.022+0000 7f084d39f740 1 -- 172.21.15.66:0/746885717 >> 172.21.15.66:0/746885717 conn(0x55a2e4cd9cb0 msgr2=0x55a2e4cda090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.023+0000 7f084d39f740 1 -- 172.21.15.66:0/746885717 shutdown_connections 2024-04-21T09:08:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.023+0000 7f084d39f740 1 -- 172.21.15.66:0/746885717 wait complete. 2024-04-21T09:08:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.023+0000 7f084d39f740 1 -- start start 2024-04-21T09:08:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.023+0000 7f084d39f740 1 Processor -- start 2024-04-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f084d39f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f084d39f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f084d39f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f08437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f08437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50740/0 (socket says 172.21.15.66:50740) 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f08437fe640 1 -- 172.21.15.66:0/2701918613 learned_addr learned my addr 172.21.15.66:0/2701918613 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a2e4ceb870 con 0x55a2e4cdfea0 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a2e4bfd2f0 con 0x55a2e4cdf580 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a2e4ceb9f0 con 0x55a2e4ce8a80 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f0843fff640 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f084d39f740 10 client.0 _ll_register_callbacks cb 0x55a2e4c0a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f0843fff640 1 -- 172.21.15.66:0/2701918613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 msgr2=0x55a2e4ce6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f0843fff640 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f0843fff640 1 -- 172.21.15.66:0/2701918613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 msgr2=0x55a2e4ce6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f0843fff640 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.024+0000 7f0843fff640 1 -- 172.21.15.66:0/2701918613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e49563f0 con 0x55a2e4cdf580 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f0843fff640 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce5bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0834009230 tx=0x7f083402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4afd0f022bd5b018 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f08237fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f08237fe640 1 -- 172.21.15.66:0/2701918613 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083404e030 con 0x55a2e4cdf580 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083c011d50 con 0x55a2e4cdf580 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f08237fe640 1 -- 172.21.15.66:0/2701918613 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f083402cb20 con 0x55a2e4cdf580 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f08237fe640 1 -- 172.21.15.66:0/2701918613 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f083405b050 con 0x55a2e4cdf580 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f084d39f740 10 client.25544 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2e49160f0 con 0x55a2e4cdf580 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.025+0000 7f08237fe640 1 -- 172.21.15.66:0/2701918613 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0834004b70 con 0x55a2e4cdf580 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f08237fe640 1 -- 172.21.15.66:0/2701918613 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0834003b80 con 0x55a2e4cdf580 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f084d39f740 10 client.25544 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f084d39f740 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f084d39f740 20 client.25544 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f084d39f740 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4cf8b20 0x55a2e4cfaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f084d39f740 4 client.25544 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f083c011d50 con 0x55a2e4cf8b20 2024-04-21T09:08:48.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f08437fe640 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4cf8b20 0x55a2e4cfaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f08237fe640 10 client.25544 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.026+0000 7f08437fe640 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4cf8b20 0x55a2e4cfaf90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c24f1290a160f5f server_cookie=4be82607bc922920 in_seq=0 out_seq=0 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.027+0000 7f08237fe640 1 -- 172.21.15.66:0/2701918613 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f083c011d50 con 0x55a2e4cf8b20 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.027+0000 7f08237fe640 10 client.25544 handle_command_reply: tid=0 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.027+0000 7f084d39f740 1 client.25544 shutdown 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 msgr2=0x55a2e4ce5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce5bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0834009230 tx=0x7f083402bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 shutdown_connections 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4cf8b20 0x55a2e4cfaf90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 --2- 172.21.15.66:0/2701918613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce5bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 >> 172.21.15.66:0/2701918613 conn(0x55a2e4cd9cb0 msgr2=0x55a2e4cdba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f08237fe640 0 client.25544 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:48.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 shutdown_connections 2024-04-21T09:08:48.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 1 -- 172.21.15.66:0/2701918613 wait complete. 2024-04-21T09:08:48.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.028+0000 7f084d39f740 20 client.25544 trim_cache size 0 max 16384 2024-04-21T09:08:48.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.056+0000 7f084d39f740 1 Processor -- start 2024-04-21T09:08:48.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f084d39f740 1 -- start start 2024-04-21T09:08:48.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f084d39f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f084d39f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4cf4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f084d39f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f084d39f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08441b1c20 con 0x55a2e4ce8a80 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f084d39f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08441af4f0 con 0x55a2e4cdfea0 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f084d39f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ad320 con 0x55a2e4cdf580 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f0848d4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f0848d4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50750/0 (socket says 172.21.15.66:50750) 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f0848d4f640 1 -- 172.21.15.66:0/1737236432 learned_addr learned my addr 172.21.15.66:0/1737236432 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.057+0000 7f08437fe640 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4cf4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0843fff640 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0848d4f640 1 -- 172.21.15.66:0/1737236432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4cdf580 msgr2=0x55a2e4ce8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0848d4f640 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0848d4f640 1 -- 172.21.15.66:0/1737236432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 msgr2=0x55a2e4cf4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0848d4f640 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4cf4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0848d4f640 1 -- 172.21.15.66:0/1737236432 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e4cdfcd0 con 0x55a2e4ce8a80 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f08437fe640 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4cf4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0848d4f640 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce4d30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f083c005bf0 tx=0x7f083c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28900d87f0912e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0829ffb640 1 -- 172.21.15.66:0/1737236432 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c004aa0 con 0x55a2e4ce8a80 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0829ffb640 1 -- 172.21.15.66:0/1737236432 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c004c40 con 0x55a2e4ce8a80 2024-04-21T09:08:48.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.058+0000 7f0829ffb640 1 -- 172.21.15.66:0/1737236432 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0052a0 con 0x55a2e4ce8a80 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.059+0000 7f084d39f740 1 -- 172.21.15.66:0/1737236432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4ce8a80 msgr2=0x55a2e4ce4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.059+0000 7f084d39f740 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce4d30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f083c005bf0 tx=0x7f083c009c80 comp rx=0 tx=0).stop 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.059+0000 7f084d39f740 1 -- 172.21.15.66:0/1737236432 shutdown_connections 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.059+0000 7f084d39f740 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce4d30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.059+0000 7f084d39f740 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdfea0 0x55a2e4cf4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.059+0000 7f084d39f740 1 --2- 172.21.15.66:0/1737236432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4cdf580 0x55a2e4ce8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.059+0000 7f084d39f740 1 -- 172.21.15.66:0/1737236432 >> 172.21.15.66:0/1737236432 conn(0x55a2e4cd9cb0 msgr2=0x55a2e4cd84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.060+0000 7f084d39f740 1 -- 172.21.15.66:0/1737236432 shutdown_connections 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.060+0000 7f084d39f740 1 -- 172.21.15.66:0/1737236432 wait complete. 2024-04-21T09:08:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.060+0000 7f084d39f740 1 -- start start 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.060+0000 7f084d39f740 1 Processor -- start 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f084d39f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4cf0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f084d39f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f084d39f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f08437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f08437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50756/0 (socket says 172.21.15.66:50756) 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f08437fe640 1 -- 172.21.15.66:0/1328543745 learned_addr learned my addr 172.21.15.66:0/1328543745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ad9b0 con 0x55a2e4cdfea0 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08441aee20 con 0x55a2e4cdf580 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ae080 con 0x55a2e4ce8a80 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f0843fff640 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4cf0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f084d39f740 10 client.0 _ll_register_callbacks cb 0x7f0834042050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f08437fe640 1 -- 172.21.15.66:0/1328543745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 msgr2=0x55a2e4ce2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.061+0000 7f08437fe640 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08437fe640 1 -- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 msgr2=0x55a2e4cf0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08437fe640 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4cf0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08437fe640 1 -- 172.21.15.66:0/1328543745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e4ce2d40 con 0x55a2e4cdfea0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f0843fff640 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4cf0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08437fe640 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce22c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f083809bf70 tx=0x7f083809c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95c55fc34b49059 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08237fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08237fe640 1 -- 172.21.15.66:0/1328543745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08380b5070 con 0x55a2e4cdfea0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083c004c40 con 0x55a2e4cdfea0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08237fe640 1 -- 172.21.15.66:0/1328543745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f08380988e0 con 0x55a2e4cdfea0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f08237fe640 1 -- 172.21.15.66:0/1328543745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0838020040 con 0x55a2e4cdfea0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f084d39f740 10 client.15942 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.062+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2e4d63220 con 0x55a2e4cdfea0 2024-04-21T09:08:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.063+0000 7f08237fe640 1 -- 172.21.15.66:0/1328543745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c004c40 con 0x55a2e4cdfea0 2024-04-21T09:08:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.063+0000 7f08237fe640 1 -- 172.21.15.66:0/1328543745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f08380a4a50 con 0x55a2e4cdfea0 2024-04-21T09:08:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.063+0000 7f084d39f740 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.063+0000 7f084d39f740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.063+0000 7f084d39f740 20 client.15942 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.064+0000 7f084d39f740 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4d639d0 0x55a2e4d65de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.064+0000 7f084d39f740 4 client.15942 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.064+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2e4c51c00 con 0x55a2e4d639d0 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.064+0000 7f0843fff640 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4d639d0 0x55a2e4d65de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.064+0000 7f08237fe640 10 client.15942 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.064+0000 7f0843fff640 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4d639d0 0x55a2e4d65de0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f00826e58e5f98c server_cookie=35566b38a3d9a1d7 in_seq=0 out_seq=0 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f08237fe640 1 -- 172.21.15.66:0/1328543745 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a2e4c51c00 con 0x55a2e4d639d0 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f08237fe640 10 client.15942 handle_command_reply: tid=0 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 client.15942 shutdown 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 msgr2=0x55a2e4ce22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce22c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f083809bf70 tx=0x7f083809c130 comp rx=0 tx=0).stop 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 shutdown_connections 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a2e4d639d0 0x55a2e4d65de0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2e4ce8a80 0x55a2e4ce2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2e4cdfea0 0x55a2e4ce22c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 --2- 172.21.15.66:0/1328543745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2e4cdf580 0x55a2e4cf0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 >> 172.21.15.66:0/1328543745 conn(0x55a2e4cd9cb0 msgr2=0x55a2e4cdb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f08237fe640 0 client.15942 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 shutdown_connections 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 1 -- 172.21.15.66:0/1328543745 wait complete. 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.065+0000 7f084d39f740 20 client.15942 trim_cache size 0 max 16384 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:08:48.068 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f082c0a3db0 msgr2=0x7f082c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f082c0a3db0 0x7f082c0a6270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0834000c00 tx=0x7f08340058b0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844106150 msgr2=0x7f0844198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844106150 0x7f0844198c60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f083800d740 tx=0x7f083800dc00 comp rx=0 tx=0).stop 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 shutdown_connections 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f082c0a3db0 0x7f082c0a6270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084419cfe0 0x7f084419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0844106150 0x7f0844198c60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 --2- 172.21.15.66:0/1939067073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0844101080 0x7f0844198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 >> 172.21.15.66:0/1939067073 conn(0x7f0844078000 msgr2=0x7f08440694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 shutdown_connections 2024-04-21T09:08:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.066+0000 7f084a7b2640 1 -- 172.21.15.66:0/1939067073 wait complete. 2024-04-21T09:08:48.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:08:48.078 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:08:48.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.195+0000 7fe08b7fe640 1 -- 172.21.15.66:0/1207542080 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07802cbe0 con 0x7fe08c069620 2024-04-21T09:08:48.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 -- 172.21.15.66:0/1207542080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c069620 msgr2=0x7fe08c069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 --2- 172.21.15.66:0/1207542080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c069620 0x7fe08c069aa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe078009900 tx=0x7fe07802bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:48.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 -- 172.21.15.66:0/1207542080 shutdown_connections 2024-04-21T09:08:48.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 --2- 172.21.15.66:0/1207542080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe08c10e810 0x7fe08c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 --2- 172.21.15.66:0/1207542080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c069620 0x7fe08c069aa0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 --2- 172.21.15.66:0/1207542080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c102e30 0x7fe08c0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 -- 172.21.15.66:0/1207542080 >> 172.21.15.66:0/1207542080 conn(0x7fe08c077e30 msgr2=0x7fe08c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 -- 172.21.15.66:0/1207542080 shutdown_connections 2024-04-21T09:08:48.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.196+0000 7fe092fca640 1 -- 172.21.15.66:0/1207542080 wait complete. 2024-04-21T09:08:48.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 Processor -- start 2024-04-21T09:08:48.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 -- start start 2024-04-21T09:08:48.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c069620 0x7fe08c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe08c102e30 0x7fe08c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c10e810 0x7fe08c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c114390 con 0x7fe08c10e810 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c114210 con 0x7fe08c069620 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe092fca640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c114510 con 0x7fe08c102e30 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe091567640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c10e810 0x7fe08c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe090d66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c069620 0x7fe08c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe091567640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c10e810 0x7fe08c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50778/0 (socket says 172.21.15.66:50778) 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe091567640 1 -- 172.21.15.66:0/3531395083 learned_addr learned my addr 172.21.15.66:0/3531395083 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.197+0000 7fe08bfff640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe08c102e30 0x7fe08c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe091567640 1 -- 172.21.15.66:0/3531395083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe08c102e30 msgr2=0x7fe08c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe091567640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe08c102e30 0x7fe08c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe091567640 1 -- 172.21.15.66:0/3531395083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c069620 msgr2=0x7fe08c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe091567640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c069620 0x7fe08c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe091567640 1 -- 172.21.15.66:0/3531395083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c19b010 con 0x7fe08c10e810 2024-04-21T09:08:48.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe091567640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c10e810 0x7fe08c19a750 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe08400eb10 tx=0x7fe08400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe089ffb640 1 -- 172.21.15.66:0/3531395083 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084004b20 con 0x7fe08c10e810 2024-04-21T09:08:48.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe089ffb640 1 -- 172.21.15.66:0/3531395083 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe084004cc0 con 0x7fe08c10e810 2024-04-21T09:08:48.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c19b300 con 0x7fe08c10e810 2024-04-21T09:08:48.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.198+0000 7fe089ffb640 1 -- 172.21.15.66:0/3531395083 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840107b0 con 0x7fe08c10e810 2024-04-21T09:08:48.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.199+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c1a7610 con 0x7fe08c10e810 2024-04-21T09:08:48.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.200+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08c101430 con 0x7fe08c10e810 2024-04-21T09:08:48.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.201+0000 7fe089ffb640 1 -- 172.21.15.66:0/3531395083 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fe084005000 con 0x7fe08c10e810 2024-04-21T09:08:48.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.202+0000 7fe089ffb640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe07c0a3e30 0x7fe07c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.202+0000 7fe090d66640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe07c0a3e30 0x7fe07c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.202+0000 7fe089ffb640 1 -- 172.21.15.66:0/3531395083 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe0840d0f40 con 0x7fe08c10e810 2024-04-21T09:08:48.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.203+0000 7fe090d66640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe07c0a3e30 0x7fe07c0a62f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe080009c20 tx=0x7fe080005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.207+0000 7fe089ffb640 1 -- 172.21.15.66:0/3531395083 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe08409d100 con 0x7fe08c10e810 2024-04-21T09:08:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.637+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe08c19bcb0 con 0x7fe08c10e810 2024-04-21T09:08:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.638+0000 7fe089ffb640 1 -- 172.21.15.66:0/3531395083 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7fe08c19bcb0 con 0x7fe08c10e810 2024-04-21T09:08:48.637 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:08:48.638 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:08:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:08:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe07c0a3e30 msgr2=0x7fe07c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe07c0a3e30 0x7fe07c0a62f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe080009c20 tx=0x7fe080005f90 comp rx=0 tx=0).stop 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c10e810 msgr2=0x7fe08c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c10e810 0x7fe08c19a750 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe08400eb10 tx=0x7fe08400c2d0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 shutdown_connections 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe07c0a3e30 0x7fe07c0a62f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe08c10e810 0x7fe08c19a750 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe08c102e30 0x7fe08c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 --2- 172.21.15.66:0/3531395083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c069620 0x7fe08c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 >> 172.21.15.66:0/3531395083 conn(0x7fe08c077e30 msgr2=0x7fe08c10f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 shutdown_connections 2024-04-21T09:08:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.640+0000 7fe092fca640 1 -- 172.21.15.66:0/3531395083 wait complete. 2024-04-21T09:08:48.651 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 -- 172.21.15.66:0/141230378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c106160 msgr2=0x7fc78c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 --2- 172.21.15.66:0/141230378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c106160 0x7fc78c10dd30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc784009ca0 tx=0x7fc78402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 -- 172.21.15.66:0/141230378 shutdown_connections 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 --2- 172.21.15.66:0/141230378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c106160 0x7fc78c10dd30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 --2- 172.21.15.66:0/141230378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc78c101a40 0x7fc78c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 --2- 172.21.15.66:0/141230378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc78c101090 0x7fc78c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 -- 172.21.15.66:0/141230378 >> 172.21.15.66:0/141230378 conn(0x7fc78c077f50 msgr2=0x7fc78c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 -- 172.21.15.66:0/141230378 shutdown_connections 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.776+0000 7fc791787640 1 -- 172.21.15.66:0/141230378 wait complete. 2024-04-21T09:08:48.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 Processor -- start 2024-04-21T09:08:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 -- start start 2024-04-21T09:08:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc78c101090 0x7fc78c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc78c106160 0x7fc78c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c19d000 0x7fc78c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc78b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c19d000 0x7fc78c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc78b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c19d000 0x7fc78c19d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50792/0 (socket says 172.21.15.66:50792) 2024-04-21T09:08:48.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc78b7fe640 1 -- 172.21.15.66:0/3257326851 learned_addr learned my addr 172.21.15.66:0/3257326851 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c111120 con 0x7fc78c19d000 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c110fa0 con 0x7fc78c101090 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc78c106160 0x7fc78c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.777+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1112a0 con 0x7fc78c106160 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc78b7fe640 1 -- 172.21.15.66:0/3257326851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc78c106160 msgr2=0x7fc78c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc78c106160 0x7fc78c198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc78b7fe640 1 -- 172.21.15.66:0/3257326851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc78c101090 msgr2=0x7fc78c198790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc78c101090 0x7fc78c198790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc78b7fe640 1 -- 172.21.15.66:0/3257326851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c19daf0 con 0x7fc78c19d000 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c19d000 0x7fc78c19d4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc784005e30 tx=0x7fc784035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc773fff640 1 -- 172.21.15.66:0/3257326851 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784039040 con 0x7fc78c19d000 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c19dd80 con 0x7fc78c19d000 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc773fff640 1 -- 172.21.15.66:0/3257326851 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc784044070 con 0x7fc78c19d000 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc773fff640 1 -- 172.21.15.66:0/3257326851 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840042a0 con 0x7fc78c19d000 2024-04-21T09:08:48.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.778+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1a55a0 con 0x7fc78c19d000 2024-04-21T09:08:48.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.781+0000 7fc794370740 1 Processor -- start 2024-04-21T09:08:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc773fff640 1 -- 172.21.15.66:0/3257326851 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7fc784004440 con 0x7fc78c19d000 2024-04-21T09:08:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc794370740 1 -- start start 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc794370740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840e180 0x55ee0840e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc794370740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08420010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc794370740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee08422940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc794370740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ee083731f0 con 0x55ee0840eaa0 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc794370740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee083104b0 con 0x55ee0840e180 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc794370740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ee0836f1b0 con 0x55ee08420550 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.782+0000 7fc78affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08420010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08420010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50804/0 (socket says 172.21.15.66:50804) 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 -- 172.21.15.66:0/3856011794 learned_addr learned my addr 172.21.15.66:0/3856011794 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840e180 0x55ee0840e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee08422940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 -- 172.21.15.66:0/3856011794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 msgr2=0x55ee08422940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee08422940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 -- 172.21.15.66:0/3856011794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840e180 msgr2=0x55ee0840e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840e180 0x55ee0840e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 -- 172.21.15.66:0/3856011794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee083021b0 con 0x55ee0840eaa0 2024-04-21T09:08:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840e180 0x55ee0840e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc78affd640 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08420010 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0097b0 tx=0x7fc77c009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e6ca910e6071af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc7717fa640 1 -- 172.21.15.66:0/3856011794 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c015020 con 0x55ee0840eaa0 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc7717fa640 1 -- 172.21.15.66:0/3856011794 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c011420 con 0x55ee0840eaa0 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.783+0000 7fc7717fa640 1 -- 172.21.15.66:0/3856011794 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0105c0 con 0x55ee0840eaa0 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc773fff640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7740a3e30 0x7fc7740a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc773fff640 1 -- 172.21.15.66:0/3257326851 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc7840f2fa0 con 0x7fc78c19d000 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 -- 172.21.15.66:0/3856011794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 msgr2=0x55ee08420010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08420010 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0097b0 tx=0x7fc77c009c70 comp rx=0 tx=0).stop 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 -- 172.21.15.66:0/3856011794 shutdown_connections 2024-04-21T09:08:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee08422940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08420010 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 --2- 172.21.15.66:0/3856011794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840e180 0x55ee0840e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 -- 172.21.15.66:0/3856011794 >> 172.21.15.66:0/3856011794 conn(0x55ee08408ba0 msgr2=0x55ee08408f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7740a3e30 0x7fc7740a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 -- 172.21.15.66:0/3856011794 shutdown_connections 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.784+0000 7fc794370740 1 -- 172.21.15.66:0/3856011794 wait complete. 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 -- start start 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 Processor -- start 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7740a3e30 0x7fc7740a62f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55ee08409fc0 tx=0x7fc780013040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840e180 0x55ee0841fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840eaa0 0x55ee0841d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee0841d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ee08423180 con 0x55ee0840e180 2024-04-21T09:08:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee0832c500 con 0x55ee0840eaa0 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ee08423300 con 0x55ee08420550 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc794370740 10 client.0 _ll_register_callbacks cb 0x55ee082fab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc78affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840e180 0x55ee0841fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc78affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840e180 0x55ee0841fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50810/0 (socket says 172.21.15.66:50810) 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc78affd640 1 -- 172.21.15.66:0/3659270986 learned_addr learned my addr 172.21.15.66:0/3659270986 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840eaa0 0x55ee0841d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.785+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee0841d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78affd640 1 -- 172.21.15.66:0/3659270986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 msgr2=0x55ee0841d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78affd640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee0841d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78affd640 1 -- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840eaa0 msgr2=0x55ee0841d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78affd640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840eaa0 0x55ee0841d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78affd640 1 -- 172.21.15.66:0/3659270986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee080853f0 con 0x55ee0840e180 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee0841d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840eaa0 0x55ee0841d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc78affd640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840e180 0x55ee0841fe40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc77c011e20 tx=0x7fc77c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c94cf856937b93f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc76eff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc76eff5640 1 -- 172.21.15.66:0/3659270986 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c015030 con 0x55ee0840e180 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc76eff5640 1 -- 172.21.15.66:0/3659270986 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc77c00b870 con 0x55ee0840e180 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77c011420 con 0x55ee0840e180 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc76eff5640 1 -- 172.21.15.66:0/3659270986 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc77c020070 con 0x55ee0840e180 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc794370740 10 client.15960 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.786+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee0814f7d0 con 0x55ee0840e180 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc76eff5640 1 -- 172.21.15.66:0/3659270986 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c011420 con 0x55ee0840e180 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc76eff5640 1 -- 172.21.15.66:0/3659270986 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc77c01aa90 con 0x55ee0840e180 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc794370740 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc794370740 10 client.15960 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc794370740 20 client.15960 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc794370740 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08432b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc794370740 4 client.15960 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee08312930 con 0x55ee08430740 2024-04-21T09:08:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.787+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08432b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.788+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08432b90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c89cfd9ad32abc6d server_cookie=447944cad942cede in_seq=0 out_seq=0 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.788+0000 7fc76eff5640 10 client.15960 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc76eff5640 1 -- 172.21.15.66:0/3659270986 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee08312930 con 0x55ee08430740 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc76eff5640 10 client.15960 handle_command_reply: tid=0 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 client.15960 shutdown 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840e180 msgr2=0x55ee0841fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840e180 0x55ee0841fe40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc77c011e20 tx=0x7fc77c011c10 comp rx=0 tx=0).stop 2024-04-21T09:08:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 shutdown_connections 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08432b90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc76eff5640 0 client.15960 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee08420550 0x55ee0841d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee0840eaa0 0x55ee0841d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 --2- 172.21.15.66:0/3659270986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840e180 0x55ee0841fe40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 >> 172.21.15.66:0/3659270986 conn(0x55ee08408ba0 msgr2=0x55ee0840a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 shutdown_connections 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 1 -- 172.21.15.66:0/3659270986 wait complete. 2024-04-21T09:08:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.789+0000 7fc794370740 20 client.15960 trim_cache size 0 max 16384 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.803+0000 7fc794370740 1 Processor -- start 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.803+0000 7fc794370740 1 -- start start 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.803+0000 7fc794370740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee0842c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.803+0000 7fc794370740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee0840a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.803+0000 7fc794370740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee0841f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc794370740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1af910 con 0x55ee0840eaa0 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc794370740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1ad1e0 con 0x55ee08420550 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc794370740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1ab010 con 0x55ee0840e180 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee0840a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee0842c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee0842c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:45406/0 (socket says 172.21.15.66:45406) 2024-04-21T09:08:48.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee0840a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50824/0 (socket says 172.21.15.66:50824) 2024-04-21T09:08:48.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 -- 172.21.15.66:0/758159730 learned_addr learned my addr 172.21.15.66:0/758159730 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 -- 172.21.15.66:0/758159730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 msgr2=0x55ee0841f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee0841f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee0841f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 -- 172.21.15.66:0/758159730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 msgr2=0x55ee0840a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee0840a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 -- 172.21.15.66:0/758159730 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0840e8d0 con 0x55ee0840e180 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee0840a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.804+0000 7fc78affd640 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee0842c0b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc77c018ce0 tx=0x7fc77c004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=487bbd07c580cb9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.805+0000 7fc770ff9640 1 -- 172.21.15.66:0/758159730 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c015070 con 0x55ee0840e180 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.805+0000 7fc770ff9640 1 -- 172.21.15.66:0/758159730 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c00b4b0 con 0x55ee0840e180 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.805+0000 7fc770ff9640 1 -- 172.21.15.66:0/758159730 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c017b50 con 0x55ee0840e180 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 -- 172.21.15.66:0/758159730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 msgr2=0x55ee0842c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee0842c0b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc77c018ce0 tx=0x7fc77c004640 comp rx=0 tx=0).stop 2024-04-21T09:08:48.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 -- 172.21.15.66:0/758159730 shutdown_connections 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee0841f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee0840a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 --2- 172.21.15.66:0/758159730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee0842c0b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 -- 172.21.15.66:0/758159730 >> 172.21.15.66:0/758159730 conn(0x55ee08408ba0 msgr2=0x55ee08408f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 -- 172.21.15.66:0/758159730 shutdown_connections 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 -- 172.21.15.66:0/758159730 wait complete. 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 -- start start 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 Processor -- start 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee08428a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08411d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.806+0000 7fc794370740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee08410880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc794370740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1ab6a0 con 0x55ee0840eaa0 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc794370740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1acb10 con 0x55ee08420550 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc794370740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1abd70 con 0x55ee0840e180 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc794370740 10 client.0 _ll_register_callbacks cb 0x7fc77c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08411d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08411d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50832/0 (socket says 172.21.15.66:50832) 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78a7fc640 1 -- 172.21.15.66:0/1659386694 learned_addr learned my addr 172.21.15.66:0/1659386694 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78affd640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee08428a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee08410880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78affd640 1 -- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 msgr2=0x55ee08410880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78affd640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee08410880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78affd640 1 -- 172.21.15.66:0/1659386694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 msgr2=0x55ee08411d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78affd640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08411d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78affd640 1 -- 172.21.15.66:0/1659386694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee08412660 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08411d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78b7fe640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee08410880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc78affd640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee08428a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc77c018c90 tx=0x7fc77c0107c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bdb972d2bf8960 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.807+0000 7fc76eff5640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc76eff5640 1 -- 172.21.15.66:0/1659386694 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c015040 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77c00b4b0 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc76eff5640 1 -- 172.21.15.66:0/1659386694 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc77c022350 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc76eff5640 1 -- 172.21.15.66:0/1659386694 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc77c011420 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc794370740 10 client.25480 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee084962f0 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc76eff5640 1 -- 172.21.15.66:0/1659386694 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c00b4b0 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.808+0000 7fc76eff5640 1 -- 172.21.15.66:0/1659386694 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc77c01aa90 con 0x55ee0840e180 2024-04-21T09:08:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc794370740 10 client.25480 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc794370740 10 client.25480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc794370740 20 client.25480 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc794370740 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08498700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc794370740 4 client.25480 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee08380c10 con 0x55ee08430740 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08498700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc76eff5640 10 client.25480 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.809+0000 7fc78a7fc640 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08498700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f53c7bf8d70764b server_cookie=16f16cc8ac74a64a in_seq=0 out_seq=0 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc76eff5640 1 -- 172.21.15.66:0/1659386694 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee08380c10 con 0x55ee08430740 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc76eff5640 10 client.25480 handle_command_reply: tid=0 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 client.25480 shutdown 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 msgr2=0x55ee08428a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee08428a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc77c018c90 tx=0x7fc77c0107c0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 shutdown_connections 2024-04-21T09:08:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55ee08430740 0x55ee08498700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee08420550 0x55ee08410880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc76eff5640 0 client.25480 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee0840eaa0 0x55ee08411d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 --2- 172.21.15.66:0/1659386694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee0840e180 0x55ee08428a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 >> 172.21.15.66:0/1659386694 conn(0x55ee08408ba0 msgr2=0x55ee0840a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 shutdown_connections 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 1 -- 172.21.15.66:0/1659386694 wait complete. 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.810+0000 7fc794370740 20 client.25480 trim_cache size 0 max 16384 2024-04-21T09:08:48.814 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7740a3e30 msgr2=0x7fc7740a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7740a3e30 0x7fc7740a62f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55ee08409fc0 tx=0x7fc780013040 comp rx=0 tx=0).stop 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c19d000 msgr2=0x7fc78c19d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c19d000 0x7fc78c19d4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc784005e30 tx=0x7fc784035d60 comp rx=0 tx=0).stop 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 shutdown_connections 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7740a3e30 0x7fc7740a62f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc78c19d000 0x7fc78c19d4b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc78c106160 0x7fc78c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 --2- 172.21.15.66:0/3257326851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc78c101090 0x7fc78c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 >> 172.21.15.66:0/3257326851 conn(0x7fc78c077f50 msgr2=0x7fc78c0694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 shutdown_connections 2024-04-21T09:08:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.811+0000 7fc791787640 1 -- 172.21.15.66:0/3257326851 wait complete. 2024-04-21T09:08:48.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 -- 172.21.15.66:0/174780529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9654103af0 msgr2=0x7f965410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 --2- 172.21.15.66:0/174780529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9654103af0 0x7f965410bfd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9648009900 tx=0x7f964802bd60 comp rx=0 tx=0).stop 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 -- 172.21.15.66:0/174780529 shutdown_connections 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 --2- 172.21.15.66:0/174780529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965410c510 0x7f965410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 --2- 172.21.15.66:0/174780529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9654103af0 0x7f965410bfd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 --2- 172.21.15.66:0/174780529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96541031d0 0x7f96541035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 -- 172.21.15.66:0/174780529 >> 172.21.15.66:0/174780529 conn(0x7f9654077f50 msgr2=0x7f9654078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 -- 172.21.15.66:0/174780529 shutdown_connections 2024-04-21T09:08:48.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.933+0000 7f965a96d640 1 -- 172.21.15.66:0/174780529 wait complete. 2024-04-21T09:08:48.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 Processor -- start 2024-04-21T09:08:48.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 -- start start 2024-04-21T09:08:48.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96541031d0 0x7f9654196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965410c510 0x7f9654196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f965419afe0 0x7f965419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9654110000 con 0x7f96541031d0 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f965410fe80 con 0x7f965419afe0 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.934+0000 7f965a96d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9654110180 con 0x7f965410c510 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96541031d0 0x7f9654196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96541031d0 0x7f9654196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50848/0 (socket says 172.21.15.66:50848) 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 -- 172.21.15.66:0/2947228316 learned_addr learned my addr 172.21.15.66:0/2947228316 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f96537fe640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965410c510 0x7f9654196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9658f0a640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f965419afe0 0x7f965419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 -- 172.21.15.66:0/2947228316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965410c510 msgr2=0x7f9654196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965410c510 0x7f9654196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 -- 172.21.15.66:0/2947228316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f965419afe0 msgr2=0x7f965419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f965419afe0 0x7f965419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 -- 172.21.15.66:0/2947228316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965419bad0 con 0x7f96541031d0 2024-04-21T09:08:48.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f9653fff640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96541031d0 0x7f9654196720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9644009be0 tx=0x7f96440071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.935+0000 7f96517fa640 1 -- 172.21.15.66:0/2947228316 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644007e00 con 0x7f96541031d0 2024-04-21T09:08:48.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.936+0000 7f96517fa640 1 -- 172.21.15.66:0/2947228316 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964401e070 con 0x7f96541031d0 2024-04-21T09:08:48.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.936+0000 7f96517fa640 1 -- 172.21.15.66:0/2947228316 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644018730 con 0x7f96541031d0 2024-04-21T09:08:48.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.936+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965419bdc0 con 0x7f96541031d0 2024-04-21T09:08:48.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.936+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541a3530 con 0x7f96541031d0 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 Processor -- start 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 -- start start 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f96517fa640 1 -- 172.21.15.66:0/2947228316 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f9644019ac0 con 0x7f96541031d0 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75ce38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ceca00 0x558b75ceee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558b75c481f0 con 0x558b75ceca00 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b75be54b0 con 0x558b75ce34d0 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f965d635740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558b75c441b0 con 0x558b75ce3e80 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f96537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ceca00 0x558b75ceee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f96537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ceca00 0x558b75ceee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50858/0 (socket says 172.21.15.66:50858) 2024-04-21T09:08:48.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.939+0000 7f96537fe640 1 -- 172.21.15.66:0/1534033282 learned_addr learned my addr 172.21.15.66:0/1534033282 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9653fff640 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9658f0a640 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75ce38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9653fff640 1 -- 172.21.15.66:0/1534033282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 msgr2=0x558b75ce38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9653fff640 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75ce38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f96517fa640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f963c0a3e30 0x7f963c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9653fff640 1 -- 172.21.15.66:0/1534033282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ceca00 msgr2=0x558b75ceee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9653fff640 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ceca00 0x558b75ceee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9653fff640 1 -- 172.21.15.66:0/1534033282 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b75bd71b0 con 0x558b75ce3e80 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f96537fe640 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ceca00 0x558b75ceee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9658f0a640 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75ce38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9658f0a640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f963c0a3e30 0x7f963c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f9653fff640 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96440d1b20 tx=0x7f96440b49a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f6d3d5c5c1822f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.940+0000 7f96517fa640 1 -- 172.21.15.66:0/2947228316 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f96440d4800 con 0x7f96541031d0 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f963a7fc640 1 -- 172.21.15.66:0/1534033282 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440a0090 con 0x558b75ce3e80 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f963a7fc640 1 -- 172.21.15.66:0/1534033282 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440a09c0 con 0x558b75ce3e80 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f963a7fc640 1 -- 172.21.15.66:0/1534033282 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964409fc20 con 0x558b75ce3e80 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f965d635740 1 -- 172.21.15.66:0/1534033282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 msgr2=0x558b75cec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f965d635740 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96440d1b20 tx=0x7f96440b49a0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f9658f0a640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f963c0a3e30 0x7f963c0a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f964c005fd0 tx=0x7f964c009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f965d635740 1 -- 172.21.15.66:0/1534033282 shutdown_connections 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f965d635740 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ceca00 0x558b75ceee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f965d635740 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f965d635740 1 --2- 172.21.15.66:0/1534033282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75ce38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.941+0000 7f965d635740 1 -- 172.21.15.66:0/1534033282 >> 172.21.15.66:0/1534033282 conn(0x558b75cddba0 msgr2=0x558b75cddf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 -- 172.21.15.66:0/1534033282 shutdown_connections 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 -- 172.21.15.66:0/1534033282 wait complete. 2024-04-21T09:08:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 -- start start 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 Processor -- start 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce3e80 0x558b75ce96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75ce9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558b75cef970 con 0x558b75ce34d0 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b75c01500 con 0x558b75ce3e80 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.942+0000 7f965d635740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558b75cefaf0 con 0x558b75ceca00 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f965d635740 10 client.0 _ll_register_callbacks cb 0x558b75bcfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50862/0 (socket says 172.21.15.66:50862) 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 -- 172.21.15.66:0/1557011031 learned_addr learned my addr 172.21.15.66:0/1557011031 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9658f0a640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75ce9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f96537fe640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce3e80 0x558b75ce96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 -- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 msgr2=0x558b75ce9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75ce9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 -- 172.21.15.66:0/1557011031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce3e80 msgr2=0x558b75ce96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce3e80 0x558b75ce96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 -- 172.21.15.66:0/1557011031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7595a3f0 con 0x558b75ce34d0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f96537fe640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce3e80 0x558b75ce96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f9653fff640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cec340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f964409cc90 tx=0x7f964409bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cf935e446c0b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f96337fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f96337fe640 1 -- 172.21.15.66:0/1557011031 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964409fe00 con 0x558b75ce34d0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f96337fe640 1 -- 172.21.15.66:0/1557011031 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f96440d4430 con 0x558b75ce34d0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.943+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96440a09c0 con 0x558b75ce34d0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f96337fe640 1 -- 172.21.15.66:0/1557011031 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f964409fc20 con 0x558b75ce34d0 2024-04-21T09:08:48.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 10 client.15978 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b75a247d0 con 0x558b75ce34d0 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f96337fe640 1 -- 172.21.15.66:0/1557011031 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440a09c0 con 0x558b75ce34d0 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f96337fe640 1 -- 172.21.15.66:0/1557011031 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f96440a0520 con 0x558b75ce34d0 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 10 client.15978 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 10 client.15978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 20 client.15978 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d052d0 0x558b75d076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 4 client.15978 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.944+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b75be7930 con 0x558b75d052d0 2024-04-21T09:08:48.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.945+0000 7f96537fe640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d052d0 0x558b75d076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.945+0000 7f96337fe640 10 client.15978 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:48.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.945+0000 7f96537fe640 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d052d0 0x558b75d076c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e12987fed69c4429 server_cookie=fc315db4293c4880 in_seq=0 out_seq=0 2024-04-21T09:08:48.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f96337fe640 1 -- 172.21.15.66:0/1557011031 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b75be7930 con 0x558b75d052d0 2024-04-21T09:08:48.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f96337fe640 10 client.15978 handle_command_reply: tid=0 2024-04-21T09:08:48.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 client.15978 shutdown 2024-04-21T09:08:48.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 msgr2=0x558b75cec340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cec340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f964409cc90 tx=0x7f964409bef0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 shutdown_connections 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75ce9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce3e80 0x558b75ce96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d052d0 0x558b75d076c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 --2- 172.21.15.66:0/1557011031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cec340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f96337fe640 0 client.15978 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 >> 172.21.15.66:0/1557011031 conn(0x558b75cddba0 msgr2=0x558b75cdf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 shutdown_connections 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 1 -- 172.21.15.66:0/1557011031 wait complete. 2024-04-21T09:08:48.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.946+0000 7f965d635740 20 client.15978 trim_cache size 0 max 16384 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 Processor -- start 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 -- start start 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ceca00 0x558b75d0a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96541aff30 con 0x558b75ce34d0 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96541ad800 con 0x558b75ceca00 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f965d635740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f96541ab630 con 0x558b75ce3e80 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f9653fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f9653fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cebe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50870/0 (socket says 172.21.15.66:50870) 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.972+0000 7f9653fff640 1 -- 172.21.15.66:0/2494305179 learned_addr learned my addr 172.21.15.66:0/2494305179 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9658f0a640 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ceca00 0x558b75d0a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f96537fe640 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9653fff640 1 -- 172.21.15.66:0/2494305179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 msgr2=0x558b75cec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9653fff640 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9653fff640 1 -- 172.21.15.66:0/2494305179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ceca00 msgr2=0x558b75d0a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9653fff640 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ceca00 0x558b75d0a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9653fff640 1 -- 172.21.15.66:0/2494305179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b75cea2e0 con 0x558b75ce34d0 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9658f0a640 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ceca00 0x558b75d0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9653fff640 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cebe00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f964409c720 tx=0x7f964409e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e9e6fa7ecf99ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9639ffb640 1 -- 172.21.15.66:0/2494305179 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440a0090 con 0x558b75ce34d0 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9639ffb640 1 -- 172.21.15.66:0/2494305179 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440bd020 con 0x558b75ce34d0 2024-04-21T09:08:48.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.973+0000 7f9639ffb640 1 -- 172.21.15.66:0/2494305179 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440da570 con 0x558b75ce34d0 2024-04-21T09:08:48.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 -- 172.21.15.66:0/2494305179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 msgr2=0x558b75cebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cebe00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f964409c720 tx=0x7f964409e7d0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 -- 172.21.15.66:0/2494305179 shutdown_connections 2024-04-21T09:08:48.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ceca00 0x558b75d0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ce3e80 0x558b75cec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 --2- 172.21.15.66:0/2494305179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce34d0 0x558b75cebe00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 -- 172.21.15.66:0/2494305179 >> 172.21.15.66:0/2494305179 conn(0x558b75cddba0 msgr2=0x558b75d0cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 -- 172.21.15.66:0/2494305179 shutdown_connections 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.974+0000 7f965d635740 1 -- 172.21.15.66:0/2494305179 wait complete. 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 -- start start 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 Processor -- start 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75d07e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce3e80 0x558b75d08360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75d088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96541abcc0 con 0x558b75ce3e80 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96541ad130 con 0x558b75ce34d0 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f96541ac390 con 0x558b75ceca00 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f96537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce3e80 0x558b75d08360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f965d635740 10 client.0 _ll_register_callbacks cb 0x7f96440da830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f96537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce3e80 0x558b75d08360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50876/0 (socket says 172.21.15.66:50876) 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.975+0000 7f96537fe640 1 -- 172.21.15.66:0/2976825627 learned_addr learned my addr 172.21.15.66:0/2976825627 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f9653fff640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75d07e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f9658f0a640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75d088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96537fe640 1 -- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 msgr2=0x558b75d088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96537fe640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75d088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96537fe640 1 -- 172.21.15.66:0/2976825627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 msgr2=0x558b75d07e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96537fe640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75d07e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96537fe640 1 -- 172.21.15.66:0/2976825627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b75d6a530 con 0x558b75ce3e80 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f9653fff640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75d07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96537fe640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce3e80 0x558b75d08360 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9648005ec0 tx=0x7f9648004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aacfab3fa62a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96337fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96337fe640 1 -- 172.21.15.66:0/2976825627 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964802c910 con 0x558b75ce3e80 2024-04-21T09:08:48.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96337fe640 1 -- 172.21.15.66:0/2976825627 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f964802cab0 con 0x558b75ce3e80 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96440bd020 con 0x558b75ce3e80 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f96337fe640 1 -- 172.21.15.66:0/2976825627 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9648056050 con 0x558b75ce3e80 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f965d635740 10 client.15984 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.976+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b75d6b840 con 0x558b75ce3e80 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f96337fe640 1 -- 172.21.15.66:0/2976825627 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440bd020 con 0x558b75ce3e80 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f96337fe640 1 -- 172.21.15.66:0/2976825627 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f96480047e0 con 0x558b75ce3e80 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f965d635740 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f965d635740 10 client.15984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f965d635740 20 client.15984 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f965d635740 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d6bff0 0x558b75d6e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f965d635740 4 client.15984 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b75c55c10 con 0x558b75d6bff0 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.977+0000 7f9653fff640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d6bff0 0x558b75d6e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f96337fe640 10 client.15984 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f9653fff640 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d6bff0 0x558b75d6e410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=831a0311b5192994 server_cookie=3a07bcb2bbcce391 in_seq=0 out_seq=0 2024-04-21T09:08:48.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f96337fe640 1 -- 172.21.15.66:0/2976825627 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b75c55c10 con 0x558b75d6bff0 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f96337fe640 10 client.15984 handle_command_reply: tid=0 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 client.15984 shutdown 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce3e80 msgr2=0x558b75d08360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce3e80 0x558b75d08360 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9648005ec0 tx=0x7f9648004200 comp rx=0 tx=0).stop 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 shutdown_connections 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558b75ceca00 0x558b75d088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x558b75d6bff0 0x558b75d6e410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b75ce3e80 0x558b75d08360 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f96337fe640 0 client.15984 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 --2- 172.21.15.66:0/2976825627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b75ce34d0 0x558b75d07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 >> 172.21.15.66:0/2976825627 conn(0x558b75cddba0 msgr2=0x558b75d005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 shutdown_connections 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 1 -- 172.21.15.66:0/2976825627 wait complete. 2024-04-21T09:08:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.978+0000 7f965d635740 20 client.15984 trim_cache size 0 max 16384 2024-04-21T09:08:48.983 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f963c0a3e30 msgr2=0x7f963c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f963c0a3e30 0x7f963c0a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f964c005fd0 tx=0x7f964c009500 comp rx=0 tx=0).stop 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96541031d0 msgr2=0x7f9654196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96541031d0 0x7f9654196720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9644009be0 tx=0x7f96440071c0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 shutdown_connections 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f963c0a3e30 0x7f963c0a62f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f965419afe0 0x7f965419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965410c510 0x7f9654196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 --2- 172.21.15.66:0/2947228316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96541031d0 0x7f9654196720 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 >> 172.21.15.66:0/2947228316 conn(0x7f9654077f50 msgr2=0x7f96540fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.979+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 shutdown_connections 2024-04-21T09:08:48.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:48.980+0000 7f965a96d640 1 -- 172.21.15.66:0/2947228316 wait complete. 2024-04-21T09:08:48.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 -- 172.21.15.66:0/1223749697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c074b70 msgr2=0x7f536c10e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 --2- 172.21.15.66:0/1223749697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c074b70 0x7f536c10e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f535c009900 tx=0x7f535c02bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 -- 172.21.15.66:0/1223749697 shutdown_connections 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 --2- 172.21.15.66:0/1223749697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c10e830 0x7f536c110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 --2- 172.21.15.66:0/1223749697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c074b70 0x7f536c10e2f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 --2- 172.21.15.66:0/1223749697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536c074250 0x7f536c074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 -- 172.21.15.66:0/1223749697 >> 172.21.15.66:0/1223749697 conn(0x7f536c0793e0 msgr2=0x7f536c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 -- 172.21.15.66:0/1223749697 shutdown_connections 2024-04-21T09:08:49.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.097+0000 7f537479f640 1 -- 172.21.15.66:0/1223749697 wait complete. 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 Processor -- start 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 -- start start 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c074250 0x7f536c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536c074b70 0x7f536c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c10e830 0x7f536c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1143e0 con 0x7f536c10e830 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f536c114260 con 0x7f536c074b70 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.098+0000 7f537479f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f536c114560 con 0x7f536c074250 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c10e830 0x7f536c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c10e830 0x7f536c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50890/0 (socket says 172.21.15.66:50890) 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 -- 172.21.15.66:0/3385612438 learned_addr learned my addr 172.21.15.66:0/3385612438 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f537253b640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c074250 0x7f536c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5371d3a640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536c074b70 0x7f536c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 -- 172.21.15.66:0/3385612438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c074250 msgr2=0x7f536c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c074250 0x7f536c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 -- 172.21.15.66:0/3385612438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536c074b70 msgr2=0x7f536c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536c074b70 0x7f536c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 -- 172.21.15.66:0/3385612438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c19afc0 con 0x7f536c10e830 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5371d3a640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536c074b70 0x7f536c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f5372d3c640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c10e830 0x7f536c19a700 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f536400e960 tx=0x7f536400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f53637fe640 1 -- 172.21.15.66:0/3385612438 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364004d20 con 0x7f536c10e830 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.099+0000 7f53637fe640 1 -- 172.21.15.66:0/3385612438 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364005020 con 0x7f536c10e830 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.100+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c19b2b0 con 0x7f536c10e830 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.100+0000 7f53637fe640 1 -- 172.21.15.66:0/3385612438 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640116d0 con 0x7f536c10e830 2024-04-21T09:08:49.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.100+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c1a75c0 con 0x7f536c10e830 2024-04-21T09:08:49.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f53637fe640 1 -- 172.21.15.66:0/3385612438 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f53640051d0 con 0x7f536c10e830 2024-04-21T09:08:49.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 Processor -- start 2024-04-21T09:08:49.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 -- start start 2024-04-21T09:08:49.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecbc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecc5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecc5a00 0x55998ecc7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55998ec211f0 con 0x55998ecbc4d0 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55998ebbe4b0 con 0x55998ecc5a00 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.103+0000 7f537738a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55998ec1d1b0 con 0x55998ecbce80 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecbc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecbc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50892/0 (socket says 172.21.15.66:50892) 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 -- 172.21.15.66:0/672733061 learned_addr learned my addr 172.21.15.66:0/672733061 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f537253b640 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecc5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f53637fe640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f53580a3e30 0x7f53580a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecc5a00 0x55998ecc7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 -- 172.21.15.66:0/672733061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 msgr2=0x55998ecc5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecc5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 -- 172.21.15.66:0/672733061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecc5a00 msgr2=0x55998ecc7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecc5a00 0x55998ecc7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 -- 172.21.15.66:0/672733061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998ebb01b0 con 0x55998ecbc4d0 2024-04-21T09:08:49.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f53637fe640 1 -- 172.21.15.66:0/3385612438 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f53640d10b0 con 0x7f536c10e830 2024-04-21T09:08:49.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5372d3c640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f53580a3e30 0x7f53580a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.104+0000 7f5371d3a640 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecbc8b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f535c005c40 tx=0x7f535c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93d796ac0ea4941 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:49.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.105+0000 7f5360ff9640 1 -- 172.21.15.66:0/672733061 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c048070 con 0x55998ecbc4d0 2024-04-21T09:08:49.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.105+0000 7f5360ff9640 1 -- 172.21.15.66:0/672733061 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c0054f0 con 0x55998ecbc4d0 2024-04-21T09:08:49.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.105+0000 7f5360ff9640 1 -- 172.21.15.66:0/672733061 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c0035f0 con 0x55998ecbc4d0 2024-04-21T09:08:49.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.105+0000 7f537738a740 1 -- 172.21.15.66:0/672733061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 msgr2=0x55998ecbc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.105+0000 7f537738a740 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecbc8b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f535c005c40 tx=0x7f535c005bd0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.105+0000 7f5372d3c640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f53580a3e30 0x7f53580a62f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f53640ce430 tx=0x7f5364009270 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:49.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- 172.21.15.66:0/672733061 shutdown_connections 2024-04-21T09:08:49.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecc5a00 0x55998ecc7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecc5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 --2- 172.21.15.66:0/672733061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecbc8b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- 172.21.15.66:0/672733061 >> 172.21.15.66:0/672733061 conn(0x55998ecb6ba0 msgr2=0x55998ecb6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- 172.21.15.66:0/672733061 shutdown_connections 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- 172.21.15.66:0/672733061 wait complete. 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- start start 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 Processor -- start 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecc53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbce80 0x55998ecc2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecc5a00 0x55998ecc2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55998ecc8910 con 0x55998ecbc4d0 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55998ebda500 con 0x55998ecbce80 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.106+0000 7f537738a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55998ecc8a90 con 0x55998ecc5a00 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f537738a740 10 client.0 _ll_register_callbacks cb 0x55998eba8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecc53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecc53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50896/0 (socket says 172.21.15.66:50896) 2024-04-21T09:08:49.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 -- 172.21.15.66:0/3285927727 learned_addr learned my addr 172.21.15.66:0/3285927727 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecc5a00 0x55998ecc2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 -- 172.21.15.66:0/3285927727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecc5a00 msgr2=0x55998ecc2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecc5a00 0x55998ecc2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f537253b640 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbce80 0x55998ecc2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 -- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbce80 msgr2=0x55998ecc2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbce80 0x55998ecc2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 -- 172.21.15.66:0/3285927727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998e9333f0 con 0x55998ecbc4d0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5371d3a640 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecc53a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f535c005ec0 tx=0x7f535c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ed73a296d2f310 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.107+0000 7f5355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f5355ffb640 1 -- 172.21.15.66:0/3285927727 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c048070 con 0x55998ecbc4d0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f535c0054f0 con 0x55998ecbc4d0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f5355ffb640 1 -- 172.21.15.66:0/3285927727 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f535c02cc90 con 0x55998ecbc4d0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f5355ffb640 1 -- 172.21.15.66:0/3285927727 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f535c004c30 con 0x55998ecbc4d0 2024-04-21T09:08:49.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 10 client.15996 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55998e9fd7d0 con 0x55998ecbc4d0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f5355ffb640 1 -- 172.21.15.66:0/3285927727 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c0054f0 con 0x55998ecbc4d0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f5355ffb640 1 -- 172.21.15.66:0/3285927727 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f535c0038f0 con 0x55998ecbc4d0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 10 client.15996 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 20 client.15996 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ecd5ab0 0x55998ecd7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 4 client.15996 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.108+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55998ebc0930 con 0x55998ecd5ab0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.109+0000 7f537253b640 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ecd5ab0 0x55998ecd7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.109+0000 7f537253b640 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ecd5ab0 0x55998ecd7ea0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6269f198a97659d server_cookie=24b35fbbe5d747a4 in_seq=0 out_seq=0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.109+0000 7f5355ffb640 10 client.15996 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f5355ffb640 1 -- 172.21.15.66:0/3285927727 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55998ebc0930 con 0x55998ecd5ab0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f5355ffb640 10 client.15996 handle_command_reply: tid=0 2024-04-21T09:08:49.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 client.15996 shutdown 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 msgr2=0x55998ecc53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecc53a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f535c005ec0 tx=0x7f535c004b50 comp rx=0 tx=0).stop 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 shutdown_connections 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ecd5ab0 0x55998ecd7ea0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecc5a00 0x55998ecc2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbce80 0x55998ecc2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 --2- 172.21.15.66:0/3285927727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbc4d0 0x55998ecc53a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 >> 172.21.15.66:0/3285927727 conn(0x55998ecb6ba0 msgr2=0x55998ecb88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f5355ffb640 0 client.15996 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 shutdown_connections 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 1 -- 172.21.15.66:0/3285927727 wait complete. 2024-04-21T09:08:49.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.110+0000 7f537738a740 20 client.15996 trim_cache size 0 max 16384 2024-04-21T09:08:49.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 Processor -- start 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 -- start start 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbc4d0 0x55998ecdaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecdb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecc5a00 0x55998ecc4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1b3dd0 con 0x55998ecc5a00 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1b16a0 con 0x55998ecbc4d0 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537738a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1af4d0 con 0x55998ecbce80 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecc5a00 0x55998ecc4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecc5a00 0x55998ecc4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50910/0 (socket says 172.21.15.66:50910) 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 -- 172.21.15.66:0/1197801188 learned_addr learned my addr 172.21.15.66:0/1197801188 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f537253b640 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecdb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbc4d0 0x55998ecdaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 -- 172.21.15.66:0/1197801188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 msgr2=0x55998ecdb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecdb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 -- 172.21.15.66:0/1197801188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbc4d0 msgr2=0x55998ecdaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbc4d0 0x55998ecdaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.124+0000 7f5371d3a640 1 -- 172.21.15.66:0/1197801188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998ecc31e0 con 0x55998ecc5a00 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.125+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecc5a00 0x55998ecc4e60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f535c001eb0 tx=0x7f535c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d69fe459dc7c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.125+0000 7f5357fff640 1 -- 172.21.15.66:0/1197801188 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c048040 con 0x55998ecc5a00 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.125+0000 7f5357fff640 1 -- 172.21.15.66:0/1197801188 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c049040 con 0x55998ecc5a00 2024-04-21T09:08:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.125+0000 7f5357fff640 1 -- 172.21.15.66:0/1197801188 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c003960 con 0x55998ecc5a00 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 -- 172.21.15.66:0/1197801188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecc5a00 msgr2=0x55998ecc4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecc5a00 0x55998ecc4e60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f535c001eb0 tx=0x7f535c005fd0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 -- 172.21.15.66:0/1197801188 shutdown_connections 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecc5a00 0x55998ecc4e60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbce80 0x55998ecdb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 --2- 172.21.15.66:0/1197801188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecbc4d0 0x55998ecdaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 -- 172.21.15.66:0/1197801188 >> 172.21.15.66:0/1197801188 conn(0x55998ecb6ba0 msgr2=0x55998ecd1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 -- 172.21.15.66:0/1197801188 shutdown_connections 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 -- 172.21.15.66:0/1197801188 wait complete. 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.126+0000 7f537738a740 1 -- start start 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537738a740 1 Processor -- start 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537738a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbc4d0 0x55998ecdaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537738a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbce80 0x55998ecd7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537738a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecd8320 0x55998ecd87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537253b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbce80 0x55998ecd7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537253b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbce80 0x55998ecd7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50914/0 (socket says 172.21.15.66:50914) 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537253b640 1 -- 172.21.15.66:0/1546766730 learned_addr learned my addr 172.21.15.66:0/1546766730 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1afb60 con 0x55998ecbce80 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbc4d0 0x55998ecdaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1b0fd0 con 0x55998ecd8320 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1b0230 con 0x55998ecbc4d0 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecd8320 0x55998ecd87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537253b640 1 -- 172.21.15.66:0/1546766730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbc4d0 msgr2=0x55998ecdaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537253b640 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbc4d0 0x55998ecdaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.127+0000 7f537253b640 1 -- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecd8320 msgr2=0x55998ecd87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f537253b640 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecd8320 0x55998ecd87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f537253b640 1 -- 172.21.15.66:0/1546766730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998ecdb2f0 con 0x55998ecbce80 2024-04-21T09:08:49.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f537253b640 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbce80 0x55998ecd7de0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55998ecb7c10 tx=0x7f5368009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a26401fa21ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f537738a740 10 client.0 _ll_register_callbacks cb 0x7f535c0043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f5355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f535c049040 con 0x55998ecbce80 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f5355ffb640 1 -- 172.21.15.66:0/1546766730 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536802d050 con 0x55998ecbce80 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f5355ffb640 1 -- 172.21.15.66:0/1546766730 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f5368003a00 con 0x55998ecbce80 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f5355ffb640 1 -- 172.21.15.66:0/1546766730 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5368023070 con 0x55998ecbce80 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f537738a740 10 client.16002 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.128+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55998ed3bb40 con 0x55998ecbce80 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f5355ffb640 1 -- 172.21.15.66:0/1546766730 <== mon.0 v2:172.21.15.66:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5368003c20 con 0x55998ecbce80 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f537738a740 10 client.16002 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f537738a740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f537738a740 20 client.16002 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f537738a740 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ed3c150 0x55998ed3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f537738a740 4 client.16002 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:08:49.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55998ec2ec10 con 0x55998ed3c150 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.129+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ed3c150 0x55998ed3e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f5355ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f5371d3a640 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ed3c150 0x55998ed3e610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690abee55ccc52d5 server_cookie=1bd97e9758a67c83 in_seq=0 out_seq=0 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f5355ffb640 1 -- 172.21.15.66:0/1546766730 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55998ec2ec10 con 0x55998ed3c150 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f5355ffb640 10 client.16002 handle_command_reply: tid=0 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 client.16002 shutdown 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbce80 msgr2=0x55998ecd7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbce80 0x55998ecd7de0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55998ecb7c10 tx=0x7f5368009e20 comp rx=0 tx=0).stop 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 shutdown_connections 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55998ed3c150 0x55998ed3e610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55998ecd8320 0x55998ecd87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55998ecbce80 0x55998ecd7de0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 --2- 172.21.15.66:0/1546766730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55998ecbc4d0 0x55998ecdaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f5355ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 >> 172.21.15.66:0/1546766730 conn(0x55998ecb6ba0 msgr2=0x55998ecd0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 shutdown_connections 2024-04-21T09:08:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 1 -- 172.21.15.66:0/1546766730 wait complete. 2024-04-21T09:08:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.130+0000 7f537738a740 20 client.16002 trim_cache size 0 max 16384 2024-04-21T09:08:49.135 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f53580a3e30 msgr2=0x7f53580a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f53580a3e30 0x7f53580a62f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f53640ce430 tx=0x7f5364009270 comp rx=0 tx=0).stop 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c10e830 msgr2=0x7f536c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c10e830 0x7f536c19a700 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f536400e960 tx=0x7f536400ee20 comp rx=0 tx=0).stop 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 shutdown_connections 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f53580a3e30 0x7f53580a62f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f536c10e830 0x7f536c19a700 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536c074b70 0x7f536c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 --2- 172.21.15.66:0/3385612438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c074250 0x7f536c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 >> 172.21.15.66:0/3385612438 conn(0x7f536c0793e0 msgr2=0x7f536c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 shutdown_connections 2024-04-21T09:08:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:49.131+0000 7f537479f640 1 -- 172.21.15.66:0/3385612438 wait complete. 2024-04-21T09:08:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:49 smithi186 ceph-mon[31505]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:08:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:49 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3531395083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:49.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:49 smithi083 ceph-mon[31413]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:08:49.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:49 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3531395083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:49 smithi066 ceph-mon[27953]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:08:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:49 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3531395083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:08:50.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:08:50.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:08:50.146 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 -- 172.21.15.66:0/3314652746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 msgr2=0x7f2f20101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 --2- 172.21.15.66:0/3314652746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f20101470 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009900 tx=0x7f2f1002bc30 comp rx=0 tx=0).stop 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 -- 172.21.15.66:0/3314652746 shutdown_connections 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 --2- 172.21.15.66:0/3314652746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f20106160 0x7f2f2010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 --2- 172.21.15.66:0/3314652746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f20101a40 0x7f2f20105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 --2- 172.21.15.66:0/3314652746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f20101470 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 -- 172.21.15.66:0/3314652746 >> 172.21.15.66:0/3314652746 conn(0x7f2f20077f50 msgr2=0x7f2f20078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 -- 172.21.15.66:0/3314652746 shutdown_connections 2024-04-21T09:08:50.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.257+0000 7f2f26625640 1 -- 172.21.15.66:0/3314652746 wait complete. 2024-04-21T09:08:50.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 Processor -- start 2024-04-21T09:08:50.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 -- start start 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f201995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f20106160 0x7f2f20199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2019a090 0x7f2f201a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201110a0 con 0x7f2f20101090 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20110f20 con 0x7f2f2019a090 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f26625640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20111220 con 0x7f2f20106160 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f201995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f201995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50924/0 (socket says 172.21.15.66:50924) 2024-04-21T09:08:50.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.258+0000 7f2f1ffff640 1 -- 172.21.15.66:0/1802094555 learned_addr learned my addr 172.21.15.66:0/1802094555 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f20106160 0x7f2f20199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2019a090 0x7f2f201a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1ffff640 1 -- 172.21.15.66:0/1802094555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f20106160 msgr2=0x7f2f20199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f20106160 0x7f2f20199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1ffff640 1 -- 172.21.15.66:0/1802094555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2019a090 msgr2=0x7f2f201a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2019a090 0x7f2f201a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1ffff640 1 -- 172.21.15.66:0/1802094555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201a4e10 con 0x7f2f20101090 2024-04-21T09:08:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f201995c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005940 tx=0x7f2f1002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1d7fa640 1 -- 172.21.15.66:0/1802094555 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10043020 con 0x7f2f20101090 2024-04-21T09:08:50.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201a50a0 con 0x7f2f20101090 2024-04-21T09:08:50.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1d7fa640 1 -- 172.21.15.66:0/1802094555 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f1002cbe0 con 0x7f2f20101090 2024-04-21T09:08:50.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f1d7fa640 1 -- 172.21.15.66:0/1802094555 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10003bf0 con 0x7f2f20101090 2024-04-21T09:08:50.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.259+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a5640 con 0x7f2f20101090 2024-04-21T09:08:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 Processor -- start 2024-04-21T09:08:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 -- start start 2024-04-21T09:08:50.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f4fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fd230 0x56048f509c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f50c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56048f460f00 con 0x56048f4fc880 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56048f3fe590 con 0x56048f4fd230 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f29214740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56048f45cec0 con 0x56048f50a220 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f24bc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f4fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f24bc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f4fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50938/0 (socket says 172.21.15.66:50938) 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.262+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2005755112 learned_addr learned my addr 172.21.15.66:0/2005755112 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f50c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2005755112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 msgr2=0x56048f50c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f50c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2005755112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fd230 msgr2=0x56048f509c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fd230 0x56048f509c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2005755112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048f23b7e0 con 0x56048f4fc880 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f4fcc60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009e60 tx=0x7f2f18010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424550f0a075c198 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f067fc640 1 -- 172.21.15.66:0/2005755112 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18016070 con 0x56048f4fc880 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f067fc640 1 -- 172.21.15.66:0/2005755112 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f18004d20 con 0x56048f4fc880 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.263+0000 7f2f067fc640 1 -- 172.21.15.66:0/2005755112 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f180127d0 con 0x56048f4fc880 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 -- 172.21.15.66:0/2005755112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 msgr2=0x56048f4fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f4fcc60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009e60 tx=0x7f2f18010740 comp rx=0 tx=0).stop 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f1d7fa640 1 -- 172.21.15.66:0/1802094555 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f2f10046020 con 0x7f2f20101090 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 -- 172.21.15.66:0/2005755112 shutdown_connections 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f50c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fd230 0x56048f509c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 --2- 172.21.15.66:0/2005755112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f4fcc60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 -- 172.21.15.66:0/2005755112 >> 172.21.15.66:0/2005755112 conn(0x56048f4f6fb0 msgr2=0x56048f4f7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 -- 172.21.15.66:0/2005755112 shutdown_connections 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 -- 172.21.15.66:0/2005755112 wait complete. 2024-04-21T09:08:50.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.264+0000 7f2f29214740 1 -- start start 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 1 Processor -- start 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f507250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f507790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f50a220 0x56048f507cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56048f50d150 con 0x56048f4fc880 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56048f41a580 con 0x56048f50a220 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56048f50d2d0 con 0x56048f4fd230 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f29214740 10 client.0 _ll_register_callbacks cb 0x56048f420650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f507250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f507250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50942/0 (socket says 172.21.15.66:50942) 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2294844939 learned_addr learned my addr 172.21.15.66:0/2294844939 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f1d7fa640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2f080a3cc0 0x7f2f080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f507790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2f080a3cc0 0x7f2f080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f50a220 0x56048f507cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2294844939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 msgr2=0x56048f507790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f1d7fa640 1 -- 172.21.15.66:0/1802094555 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2f100f2f50 con 0x7f2f20101090 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f507790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f50a220 msgr2=0x56048f507cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f50a220 0x56048f507cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.265+0000 7f2f24bc2640 1 -- 172.21.15.66:0/2294844939 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048f29bf70 con 0x56048f4fc880 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f50a220 0x56048f507cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f507790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f507250 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800a010 tx=0x7f2f18005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd887ea7a247642b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2eff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2f080a3cc0 0x7f2f080a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009c20 tx=0x7f2f1802b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2eff7fe640 1 -- 172.21.15.66:0/2294844939 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18023050 con 0x56048f4fc880 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2eff7fe640 1 -- 172.21.15.66:0/2294844939 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2f1801a410 con 0x56048f4fc880 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.266+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f18004d20 con 0x56048f4fc880 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.267+0000 7f2eff7fe640 1 -- 172.21.15.66:0/2294844939 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f1802c020 con 0x56048f4fc880 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.267+0000 7f2f29214740 10 client.16014 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.267+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56048f133170 con 0x56048f4fc880 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.267+0000 7f2eff7fe640 1 -- 172.21.15.66:0/2294844939 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f18004d20 con 0x56048f4fc880 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2eff7fe640 1 -- 172.21.15.66:0/2294844939 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2f1800f040 con 0x56048f4fc880 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f29214740 10 client.16014 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f29214740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f29214740 20 client.16014 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f29214740 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f51a670 0x56048f51ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f29214740 4 client.16014 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56048f46e920 con 0x56048f51a670 2024-04-21T09:08:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f51a670 0x56048f51ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2eff7fe640 10 client.16014 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.268+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f51a670 0x56048f51ca60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5559de2528a883d7 server_cookie=501d8455c3419ba3 in_seq=0 out_seq=0 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.269+0000 7f2eff7fe640 1 -- 172.21.15.66:0/2294844939 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56048f46e920 con 0x56048f51a670 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.269+0000 7f2eff7fe640 10 client.16014 handle_command_reply: tid=0 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.269+0000 7f2f29214740 1 client.16014 shutdown 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 msgr2=0x56048f507250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f507250 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800a010 tx=0x7f2f18005270 comp rx=0 tx=0).stop 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 shutdown_connections 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f51a670 0x56048f51ca60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f50a220 0x56048f507cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2eff7fe640 0 client.16014 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f507790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 --2- 172.21.15.66:0/2294844939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fc880 0x56048f507250 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 >> 172.21.15.66:0/2294844939 conn(0x56048f4f6fb0 msgr2=0x56048f4f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 shutdown_connections 2024-04-21T09:08:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 1 -- 172.21.15.66:0/2294844939 wait complete. 2024-04-21T09:08:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.270+0000 7f2f29214740 20 client.16014 trim_cache size 0 max 16384 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 Processor -- start 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 -- start start 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f516080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fd230 0x56048f506a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f507920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201afb40 con 0x56048f4fd230 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201ad410 con 0x56048f4fc880 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f29214740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201ab240 con 0x56048f50a220 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fd230 0x56048f506a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fd230 0x56048f506a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50956/0 (socket says 172.21.15.66:50956) 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f1f7fe640 1 -- 172.21.15.66:0/2644624223 learned_addr learned my addr 172.21.15.66:0/2644624223 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.305+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f516080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f507920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f1f7fe640 1 -- 172.21.15.66:0/2644624223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 msgr2=0x56048f507920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f507920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f1f7fe640 1 -- 172.21.15.66:0/2644624223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 msgr2=0x56048f516080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f516080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f1f7fe640 1 -- 172.21.15.66:0/2644624223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048f51cd90 con 0x56048f4fd230 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f516080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fd230 0x56048f506a20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400fc20 tx=0x7f2f14013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cad0068713848d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f05ffb640 1 -- 172.21.15.66:0/2644624223 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14004030 con 0x56048f4fd230 2024-04-21T09:08:50.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f05ffb640 1 -- 172.21.15.66:0/2644624223 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f140041d0 con 0x56048f4fd230 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.306+0000 7f2f05ffb640 1 -- 172.21.15.66:0/2644624223 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1401c840 con 0x56048f4fd230 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 -- 172.21.15.66:0/2644624223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fd230 msgr2=0x56048f506a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fd230 0x56048f506a20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400fc20 tx=0x7f2f14013830 comp rx=0 tx=0).stop 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 -- 172.21.15.66:0/2644624223 shutdown_connections 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f50a220 0x56048f507920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f4fd230 0x56048f506a20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 --2- 172.21.15.66:0/2644624223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f516080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 -- 172.21.15.66:0/2644624223 >> 172.21.15.66:0/2644624223 conn(0x56048f4f6fb0 msgr2=0x56048f507210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 -- 172.21.15.66:0/2644624223 shutdown_connections 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 -- 172.21.15.66:0/2644624223 wait complete. 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.307+0000 7f2f29214740 1 -- start start 2024-04-21T09:08:50.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 1 Processor -- start 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f512ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f50de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f50a220 0x56048f50e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201ab8d0 con 0x56048f50a220 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201acd40 con 0x56048f4fc880 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201abfa0 con 0x56048f4fd230 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f29214740 10 client.0 _ll_register_callbacks cb 0x7f2f18009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f512ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f512ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:49854/0 (socket says 172.21.15.66:49854) 2024-04-21T09:08:50.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f1ffff640 1 -- 172.21.15.66:0/128552296 learned_addr learned my addr 172.21.15.66:0/128552296 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f24bc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f50a220 0x56048f50e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.308+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f50de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f24bc2640 1 -- 172.21.15.66:0/128552296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 msgr2=0x56048f50de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f50de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f24bc2640 1 -- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 msgr2=0x56048f512ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f512ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f24bc2640 1 -- 172.21.15.66:0/128552296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048f57f870 con 0x56048f50a220 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f1f7fe640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f50de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f512ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f24bc2640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f50a220 0x56048f50e550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2f18028820 tx=0x7f2f18005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effe87351ec6fca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2eff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2eff7fe640 1 -- 172.21.15.66:0/128552296 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1803e070 con 0x56048f50a220 2024-04-21T09:08:50.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f140041d0 con 0x56048f50a220 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2eff7fe640 1 -- 172.21.15.66:0/128552296 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2f1801b480 con 0x56048f50a220 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2eff7fe640 1 -- 172.21.15.66:0/128552296 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f1804a050 con 0x56048f50a220 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f29214740 10 client.16020 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.309+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56048f580a30 con 0x56048f50a220 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2eff7fe640 1 -- 172.21.15.66:0/128552296 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f140041d0 con 0x56048f50a220 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2eff7fe640 1 -- 172.21.15.66:0/128552296 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2f1801bba0 con 0x56048f50a220 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2f29214740 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2f29214740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2f29214740 20 client.16020 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2f29214740 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f581180 0x56048f5835a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2f29214740 4 client.16020 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:08:50.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56048f468790 con 0x56048f581180 2024-04-21T09:08:50.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.310+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f581180 0x56048f5835a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.311+0000 7f2eff7fe640 10 client.16020 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.311+0000 7f2f1ffff640 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f581180 0x56048f5835a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fcff253fb3ff8c server_cookie=1561683c867f6e92 in_seq=0 out_seq=0 2024-04-21T09:08:50.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.312+0000 7f2eff7fe640 1 -- 172.21.15.66:0/128552296 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56048f468790 con 0x56048f581180 2024-04-21T09:08:50.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.312+0000 7f2eff7fe640 10 client.16020 handle_command_reply: tid=0 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.312+0000 7f2f29214740 1 client.16020 shutdown 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.312+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f50a220 msgr2=0x56048f50e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.312+0000 7f2f29214740 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f50a220 0x56048f50e550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2f18028820 tx=0x7f2f18005360 comp rx=0 tx=0).stop 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 shutdown_connections 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56048f581180 0x56048f5835a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56048f50a220 0x56048f50e550 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2eff7fe640 0 client.16020 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56048f4fd230 0x56048f50de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 --2- 172.21.15.66:0/128552296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56048f4fc880 0x56048f512ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 >> 172.21.15.66:0/128552296 conn(0x56048f4f6fb0 msgr2=0x56048f507a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 shutdown_connections 2024-04-21T09:08:50.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 1 -- 172.21.15.66:0/128552296 wait complete. 2024-04-21T09:08:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f29214740 20 client.16020 trim_cache size 0 max 16384 2024-04-21T09:08:50.320 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:50.320 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:08:50.320 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "4400eb14-48d0-4ca3-9b02-2c494cc77a65", 2024-04-21T09:08:50.320 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:08:50.320 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:50.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2f080a3cc0 msgr2=0x7f2f080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f26625640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2f080a3cc0 0x7f2f080a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009c20 tx=0x7f2f1802b000 comp rx=0 tx=0).stop 2024-04-21T09:08:50.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 msgr2=0x7f2f201995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.313+0000 7f2f26625640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f201995c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005940 tx=0x7f2f1002c7e0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 shutdown_connections 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2f080a3cc0 0x7f2f080a6180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2019a090 0x7f2f201a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f20106160 0x7f2f20199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 --2- 172.21.15.66:0/1802094555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f20101090 0x7f2f201995c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 >> 172.21.15.66:0/1802094555 conn(0x7f2f20077f50 msgr2=0x7f2f200ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 shutdown_connections 2024-04-21T09:08:50.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.314+0000 7f2f26625640 1 -- 172.21.15.66:0/1802094555 wait complete. 2024-04-21T09:08:50.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:08:50.325 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:08:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.432+0000 7f453e421640 1 -- 172.21.15.66:0/3718855452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 msgr2=0x7f453810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.432+0000 7f453e421640 1 --2- 172.21.15.66:0/3718855452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453810e9a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4530009ca0 tx=0x7f453002c230 comp rx=0 tx=0).stop 2024-04-21T09:08:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.432+0000 7f453e421640 1 -- 172.21.15.66:0/3718855452 shutdown_connections 2024-04-21T09:08:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.432+0000 7f453e421640 1 --2- 172.21.15.66:0/3718855452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453810e9a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.432+0000 7f453e421640 1 --2- 172.21.15.66:0/3718855452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4538103ae0 0x7f453810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.433+0000 7f453e421640 1 --2- 172.21.15.66:0/3718855452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45381031c0 0x7f45381035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.433+0000 7f453e421640 1 -- 172.21.15.66:0/3718855452 >> 172.21.15.66:0/3718855452 conn(0x7f4538078000 msgr2=0x7f4538078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.433+0000 7f453e421640 1 -- 172.21.15.66:0/3718855452 shutdown_connections 2024-04-21T09:08:50.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.433+0000 7f453e421640 1 -- 172.21.15.66:0/3718855452 wait complete. 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.433+0000 7f453e421640 1 Processor -- start 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.433+0000 7f453e421640 1 -- start start 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453e421640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45381031c0 0x7f4538198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453e421640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4538103ae0 0x7f4538198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453e421640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453e421640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4538112170 con 0x7f453810c500 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453e421640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4538111ff0 con 0x7f45381031c0 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453e421640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45381122f0 con 0x7f4538103ae0 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50988/0 (socket says 172.21.15.66:50988) 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f453c9be640 1 -- 172.21.15.66:0/2062386060 learned_addr learned my addr 172.21.15.66:0/2062386060 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f45377fe640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4538103ae0 0x7f4538198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f4537fff640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45381031c0 0x7f4538198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f45377fe640 1 -- 172.21.15.66:0/2062386060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45381031c0 msgr2=0x7f4538198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f45377fe640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45381031c0 0x7f4538198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f45377fe640 1 -- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 msgr2=0x7f453819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f45377fe640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.434+0000 7f45377fe640 1 -- 172.21.15.66:0/2062386060 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453819d700 con 0x7f4538103ae0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.435+0000 7f453c9be640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.435+0000 7f45377fe640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4538103ae0 0x7f4538198c50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.435+0000 7f45357fa640 1 -- 172.21.15.66:0/2062386060 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c009c30 con 0x7f4538103ae0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.435+0000 7f45357fa640 1 -- 172.21.15.66:0/2062386060 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452c009dd0 con 0x7f4538103ae0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.435+0000 7f45357fa640 1 -- 172.21.15.66:0/2062386060 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c010460 con 0x7f4538103ae0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.435+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453819d9f0 con 0x7f4538103ae0 2024-04-21T09:08:50.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.435+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4538100d40 con 0x7f4538103ae0 2024-04-21T09:08:50.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 Processor -- start 2024-04-21T09:08:50.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 -- start start 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44bce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5556a44161e0 con 0x5556a44baa80 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5556a43b3390 con 0x5556a44b1580 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f454100b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5556a44121a0 con 0x5556a44b1ea0 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f4537fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f4537fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44ba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:45626/0 (socket says 172.21.15.66:45626) 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f453c9be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f4537fff640 1 -- 172.21.15.66:0/3612509138 learned_addr learned my addr 172.21.15.66:0/3612509138 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.438+0000 7f453c9be640 1 -- 172.21.15.66:0/3612509138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 msgr2=0x5556a44ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f453c9be640 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f453c9be640 1 -- 172.21.15.66:0/3612509138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 msgr2=0x5556a44bce70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T09:08:50.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f453c9be640 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44bce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f453c9be640 1 -- 172.21.15.66:0/3612509138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a41f27d0 con 0x5556a44b1580 2024-04-21T09:08:50.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f4537fff640 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:50.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f45357fa640 1 -- 172.21.15.66:0/2062386060 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f452c003e80 con 0x7f4538103ae0 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f453c9be640 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b1960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4530005e30 tx=0x7f45300099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1445c0e98de19671 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f451e7fc640 1 -- 172.21.15.66:0/3612509138 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530043070 con 0x5556a44b1580 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f451e7fc640 1 -- 172.21.15.66:0/3612509138 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4530032440 con 0x5556a44b1580 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f451e7fc640 1 -- 172.21.15.66:0/3612509138 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530039040 con 0x5556a44b1580 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.439+0000 7f45357fa640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f45200a3db0 0x7f45200a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f45357fa640 1 -- 172.21.15.66:0/2062386060 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f452c0d0b50 con 0x7f4538103ae0 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f4537fff640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f45200a3db0 0x7f45200a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 -- 172.21.15.66:0/3612509138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 msgr2=0x5556a44b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b1960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4530005e30 tx=0x7f45300099f0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 -- 172.21.15.66:0/3612509138 shutdown_connections 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44bce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 --2- 172.21.15.66:0/3612509138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b1960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 -- 172.21.15.66:0/3612509138 >> 172.21.15.66:0/3612509138 conn(0x5556a44abcb0 msgr2=0x5556a44ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f4537fff640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f45200a3db0 0x7f45200a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5556a44b1050 tx=0x7f4528013040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 -- 172.21.15.66:0/3612509138 shutdown_connections 2024-04-21T09:08:50.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.440+0000 7f454100b740 1 -- 172.21.15.66:0/3612509138 wait complete. 2024-04-21T09:08:50.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 -- start start 2024-04-21T09:08:50.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 Processor -- start 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44b8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f453c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f453c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50996/0 (socket says 172.21.15.66:50996) 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f453c9be640 1 -- 172.21.15.66:0/2279138570 learned_addr learned my addr 172.21.15.66:0/2279138570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f4537fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5556a44bd810 con 0x5556a44baa80 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5556a43cf2f0 con 0x5556a44b1580 2024-04-21T09:08:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.441+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5556a44bd990 con 0x5556a44b1ea0 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f45377fe640 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f454100b740 10 client.0 _ll_register_callbacks cb 0x5556a43dc580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f4537fff640 1 -- 172.21.15.66:0/2279138570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 msgr2=0x5556a44b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f4537fff640 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f4537fff640 1 -- 172.21.15.66:0/2279138570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 msgr2=0x5556a44b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f4537fff640 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f4537fff640 1 -- 172.21.15.66:0/2279138570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a41283f0 con 0x5556a44b1580 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f453c9be640 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f4537fff640 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b7b00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4528009d10 tx=0x7f4528004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b16f6f1f503922b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f45177fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f45177fe640 1 -- 172.21.15.66:0/2279138570 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528009d40 con 0x5556a44b1580 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4530032440 con 0x5556a44b1580 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.442+0000 7f45177fe640 1 -- 172.21.15.66:0/2279138570 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4528023040 con 0x5556a44b1580 2024-04-21T09:08:50.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f45177fe640 1 -- 172.21.15.66:0/2279138570 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4528035050 con 0x5556a44b1580 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 10 client.25628 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556a40e80f0 con 0x5556a44b1580 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f45177fe640 1 -- 172.21.15.66:0/2279138570 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4530032440 con 0x5556a44b1580 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f45177fe640 1 -- 172.21.15.66:0/2279138570 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4528027070 con 0x5556a44b1580 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 10 client.25628 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 20 client.25628 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a44caad0 0x5556a44ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 4 client.25628 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556a4423c00 con 0x5556a44caad0 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.443+0000 7f45377fe640 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a44caad0 0x5556a44ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.444+0000 7f45177fe640 10 client.25628 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.444+0000 7f45377fe640 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a44caad0 0x5556a44ccf00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3dffa4cfbd6768b server_cookie=c55d13feae265e00 in_seq=0 out_seq=0 2024-04-21T09:08:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.444+0000 7f45177fe640 1 -- 172.21.15.66:0/2279138570 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556a4423c00 con 0x5556a44caad0 2024-04-21T09:08:50.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.444+0000 7f45177fe640 10 client.25628 handle_command_reply: tid=0 2024-04-21T09:08:50.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.445+0000 7f454100b740 1 client.25628 shutdown 2024-04-21T09:08:50.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 msgr2=0x5556a44b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b7b00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4528009d10 tx=0x7f4528004730 comp rx=0 tx=0).stop 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 shutdown_connections 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a44caad0 0x5556a44ccf00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f45177fe640 0 client.25628 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1ea0 0x5556a44b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 --2- 172.21.15.66:0/2279138570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1580 0x5556a44b7b00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 >> 172.21.15.66:0/2279138570 conn(0x5556a44abcb0 msgr2=0x5556a44ada00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 shutdown_connections 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 1 -- 172.21.15.66:0/2279138570 wait complete. 2024-04-21T09:08:50.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.448+0000 7f454100b740 20 client.25628 trim_cache size 0 max 16384 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.483+0000 7f454100b740 1 Processor -- start 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.483+0000 7f454100b740 1 -- start start 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f454100b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44ba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f454100b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44b1ea0 0x5556a44c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f454100b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44baa80 0x5556a44b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f454100b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45381af9a0 con 0x5556a44b1ea0 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f454100b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45381ad270 con 0x5556a44baa80 2024-04-21T09:08:50.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f454100b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45381ab0a0 con 0x5556a44b1580 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44b1ea0 0x5556a44c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44b1ea0 0x5556a44c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51002/0 (socket says 172.21.15.66:51002) 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 -- 172.21.15.66:0/2052831312 learned_addr learned my addr 172.21.15.66:0/2052831312 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f4537fff640 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f453c9be640 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44baa80 0x5556a44b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 -- 172.21.15.66:0/2052831312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 msgr2=0x5556a44ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44ba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 -- 172.21.15.66:0/2052831312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44baa80 msgr2=0x5556a44b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44baa80 0x5556a44b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f45377fe640 1 -- 172.21.15.66:0/2052831312 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a44ab7e0 con 0x5556a44b1ea0 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.484+0000 7f453c9be640 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44baa80 0x5556a44b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.485+0000 7f45377fe640 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44b1ea0 0x5556a44c66d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f452c009610 tx=0x7f452c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fec4ea0268c44e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.485+0000 7f451dffb640 1 -- 172.21.15.66:0/2052831312 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c020040 con 0x5556a44b1ea0 2024-04-21T09:08:50.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.485+0000 7f451dffb640 1 -- 172.21.15.66:0/2052831312 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452c015020 con 0x5556a44b1ea0 2024-04-21T09:08:50.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.485+0000 7f451dffb640 1 -- 172.21.15.66:0/2052831312 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c0ce120 con 0x5556a44b1ea0 2024-04-21T09:08:50.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 -- 172.21.15.66:0/2052831312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44b1ea0 msgr2=0x5556a44c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44b1ea0 0x5556a44c66d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f452c009610 tx=0x7f452c087040 comp rx=0 tx=0).stop 2024-04-21T09:08:50.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 -- 172.21.15.66:0/2052831312 shutdown_connections 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44baa80 0x5556a44b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44b1ea0 0x5556a44c66d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 --2- 172.21.15.66:0/2052831312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 -- 172.21.15.66:0/2052831312 >> 172.21.15.66:0/2052831312 conn(0x5556a44abcb0 msgr2=0x5556a44aa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 -- 172.21.15.66:0/2052831312 shutdown_connections 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 -- 172.21.15.66:0/2052831312 wait complete. 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.486+0000 7f454100b740 1 -- start start 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 1 Processor -- start 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1ea0 0x5556a44b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45381ab730 con 0x5556a44baa80 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45381acba0 con 0x5556a44b1ea0 2024-04-21T09:08:50.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45381abe00 con 0x5556a44b1580 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f453c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f454100b740 10 client.0 _ll_register_callbacks cb 0x7f4528005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f453c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51008/0 (socket says 172.21.15.66:51008) 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f453c9be640 1 -- 172.21.15.66:0/1345560694 learned_addr learned my addr 172.21.15.66:0/1345560694 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f45377fe640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1ea0 0x5556a44b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.487+0000 7f4537fff640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f453c9be640 1 -- 172.21.15.66:0/1345560694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 msgr2=0x5556a44c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f453c9be640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f453c9be640 1 -- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1ea0 msgr2=0x5556a44b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f453c9be640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1ea0 0x5556a44b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f453c9be640 1 -- 172.21.15.66:0/1345560694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a44b4f90 con 0x5556a44baa80 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f45377fe640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1ea0 0x5556a44b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f453c9be640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b48d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4530006010 tx=0x7f45300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248fd4c2ab73b086 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:50.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f45177fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f45177fe640 1 -- 172.21.15.66:0/1345560694 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530039040 con 0x5556a44baa80 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f45177fe640 1 -- 172.21.15.66:0/1345560694 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4530032640 con 0x5556a44baa80 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f452c015020 con 0x5556a44baa80 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f45177fe640 1 -- 172.21.15.66:0/1345560694 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f453005b050 con 0x5556a44baa80 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f454100b740 10 client.16035 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.488+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556a4530f00 con 0x5556a44baa80 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f45177fe640 1 -- 172.21.15.66:0/1345560694 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452c015020 con 0x5556a44baa80 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f45177fe640 1 -- 172.21.15.66:0/1345560694 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45300526c0 con 0x5556a44baa80 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f454100b740 10 client.16035 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f454100b740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f454100b740 20 client.16035 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f454100b740 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a4531290 0x5556a4533680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f454100b740 4 client.16035 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:08:50.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556a441da70 con 0x5556a4531290 2024-04-21T09:08:50.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.489+0000 7f4537fff640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a4531290 0x5556a4533680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f4537fff640 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a4531290 0x5556a4533680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60dde39f0b9fd491 server_cookie=c8f9d58af08139d2 in_seq=0 out_seq=0 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f45177fe640 10 client.16035 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f45177fe640 1 -- 172.21.15.66:0/1345560694 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5556a441da70 con 0x5556a4531290 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f45177fe640 10 client.16035 handle_command_reply: tid=0 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 client.16035 shutdown 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 msgr2=0x5556a44b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b48d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4530006010 tx=0x7f45300095c0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 shutdown_connections 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5556a4531290 0x5556a4533680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5556a44baa80 0x5556a44b48d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f45177fe640 0 client.16035 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:50.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556a44b1ea0 0x5556a44b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 --2- 172.21.15.66:0/1345560694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5556a44b1580 0x5556a44c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 >> 172.21.15.66:0/1345560694 conn(0x5556a44abcb0 msgr2=0x5556a44b7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 shutdown_connections 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 1 -- 172.21.15.66:0/1345560694 wait complete. 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.490+0000 7f454100b740 20 client.16035 trim_cache size 0 max 16384 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stdout: "4400eb14-48d0-4ca3-9b02-2c494cc77a65": { 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:08:50.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4400eb14-48d0-4ca3-9b02-2c494cc77a65", 2024-04-21T09:08:50.495 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:08:50.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:08:50.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:08:50.495 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f45200a3db0 msgr2=0x7f45200a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f45200a3db0 0x7f45200a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5556a44b1050 tx=0x7f4528013040 comp rx=0 tx=0).stop 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4538103ae0 msgr2=0x7f4538198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4538103ae0 0x7f4538198c50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c00dde0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 shutdown_connections 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f45200a3db0 0x7f45200a6270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f453810c500 0x7f453819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4538103ae0 0x7f4538198c50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 --2- 172.21.15.66:0/2062386060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45381031c0 0x7f4538198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 >> 172.21.15.66:0/2062386060 conn(0x7f4538078000 msgr2=0x7f45380ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 shutdown_connections 2024-04-21T09:08:50.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:08:50.491+0000 7f453e421640 1 -- 172.21.15.66:0/2062386060 wait complete. 2024-04-21T09:08:50.502 INFO:tasks.cephfs.filesystem:scrub status for tag:4400eb14-48d0-4ca3-9b02-2c494cc77a65 - {'path': '/', 'tag': '4400eb14-48d0-4ca3-9b02-2c494cc77a65', 'options': 'recursive,force'} 2024-04-21T09:08:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:51 smithi186 ceph-mon[31505]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:08:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:51 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:08:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:51 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:51 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:08:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:51 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:51 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:08:51.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:51 smithi083 ceph-mon[31413]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:08:51.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:51 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:08:51.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:51 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:51.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:51 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:08:51.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:51 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:51.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:51 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:08:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:51 smithi066 ceph-mon[27953]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:08:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:51 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:08:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:51 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:51 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:08:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:51 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:08:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:51 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:08:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:53 smithi186 ceph-mon[31505]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T09:08:53.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:53 smithi083 ceph-mon[31413]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T09:08:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:53 smithi066 ceph-mon[27953]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T09:08:55.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:55 smithi083 ceph-mon[31413]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T09:08:55.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:55 smithi066 ceph-mon[27953]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T09:08:55.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:55 smithi186 ceph-mon[31505]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T09:08:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:57 smithi083 ceph-mon[31413]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:08:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:57 smithi066 ceph-mon[27953]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:08:57.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:57 smithi186 ceph-mon[31505]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T09:08:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 -- 172.21.15.66:0/2456437423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b01031e0 msgr2=0x7f19b01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/2456437423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b01031e0 0x7f19b01035c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009900 tx=0x7f19a002bcd0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 -- 172.21.15.66:0/2456437423 shutdown_connections 2024-04-21T09:08:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/2456437423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19b010c5b0 0x7f19b010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/2456437423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19b0103b00 0x7f19b010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/2456437423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b01031e0 0x7f19b01035c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 -- 172.21.15.66:0/2456437423 >> 172.21.15.66:0/2456437423 conn(0x7f19b0077f60 msgr2=0x7f19b0078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 -- 172.21.15.66:0/2456437423 shutdown_connections 2024-04-21T09:08:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.656+0000 7f19b4b4b640 1 -- 172.21.15.66:0/2456437423 wait complete. 2024-04-21T09:08:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 Processor -- start 2024-04-21T09:08:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 -- start start 2024-04-21T09:08:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19b01031e0 0x7f19b0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19b0103b00 0x7f19b0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b010c5b0 0x7f19b019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0112200 con 0x7f19b010c5b0 2024-04-21T09:08:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0112080 con 0x7f19b0103b00 2024-04-21T09:08:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19b4b4b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0112380 con 0x7f19b01031e0 2024-04-21T09:08:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19aed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b010c5b0 0x7f19b019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19b0103b00 0x7f19b0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19aed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b010c5b0 0x7f19b019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36248/0 (socket says 172.21.15.66:36248) 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19aed9d640 1 -- 172.21.15.66:0/1984747752 learned_addr learned my addr 172.21.15.66:0/1984747752 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.657+0000 7f19ae59c640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19b01031e0 0x7f19b0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19aed9d640 1 -- 172.21.15.66:0/1984747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19b01031e0 msgr2=0x7f19b0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19aed9d640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19b01031e0 0x7f19b0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19aed9d640 1 -- 172.21.15.66:0/1984747752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19b0103b00 msgr2=0x7f19b0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19aed9d640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19b0103b00 0x7f19b0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19aed9d640 1 -- 172.21.15.66:0/1984747752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b019d6c0 con 0x7f19b010c5b0 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19a7fff640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19b0103b00 0x7f19b0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19ae59c640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19b01031e0 0x7f19b0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19aed9d640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b010c5b0 0x7f19b019cf90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f19a800eb40 tx=0x7f19a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19a77fe640 1 -- 172.21.15.66:0/1984747752 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8004b20 con 0x7f19b010c5b0 2024-04-21T09:08:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b019d9b0 con 0x7f19b010c5b0 2024-04-21T09:08:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.658+0000 7f19a77fe640 1 -- 172.21.15.66:0/1984747752 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a8004cc0 con 0x7f19b010c5b0 2024-04-21T09:08:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.659+0000 7f19a77fe640 1 -- 172.21.15.66:0/1984747752 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8010720 con 0x7f19b010c5b0 2024-04-21T09:08:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.659+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b01a5240 con 0x7f19b010c5b0 2024-04-21T09:08:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.661+0000 7f19a77fe640 1 -- 172.21.15.66:0/1984747752 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f19a8005560 con 0x7f19b010c5b0 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.661+0000 7f19b7811740 1 Processor -- start 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.661+0000 7f19b7811740 1 -- start start 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19a77fe640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f199c0a3e60 0x7f199c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19b7811740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037f6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19b7811740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19b7811740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9038024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19b7811740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c90375b8d0 con 0x55c9037f6b60 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19b7811740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9036f8e80 con 0x55c9038000e0 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19b7811740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c903757890 con 0x55c9037f78d0 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19a77fe640 1 -- 172.21.15.66:0/1984747752 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f19a80d1370 con 0x7f19b010c5b0 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19a7fff640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f199c0a3e60 0x7f199c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36262/0 (socket says 172.21.15.66:36262) 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19a7fff640 1 -- 172.21.15.66:0/2130030231 learned_addr learned my addr 172.21.15.66:0/2130030231 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19aed9d640 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9038024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19ae59c640 1 -- 172.21.15.66:0/2130030231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 msgr2=0x55c9038024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9038024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19ae59c640 1 -- 172.21.15.66:0/2130030231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 msgr2=0x55c9037f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19ae59c640 1 -- 172.21.15.66:0/2130030231 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c90371a320 con 0x55c9037f78d0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.662+0000 7f19a7fff640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f199c0a3e60 0x7f199c0a6320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f199800bfd0 tx=0x7f199800bf40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.663+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037f6620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f19a0000c00 tx=0x7f19a002c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec7371a16ca4773 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.663+0000 7f19a4ff9640 1 -- 172.21.15.66:0/2130030231 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0043070 con 0x55c9037f78d0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.663+0000 7f19a4ff9640 1 -- 172.21.15.66:0/2130030231 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a002c880 con 0x55c9037f78d0 2024-04-21T09:08:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.663+0000 7f19a4ff9640 1 -- 172.21.15.66:0/2130030231 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a00039f0 con 0x55c9037f78d0 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 -- 172.21.15.66:0/2130030231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 msgr2=0x55c9037f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037f6620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f19a0000c00 tx=0x7f19a002c690 comp rx=0 tx=0).stop 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 -- 172.21.15.66:0/2130030231 shutdown_connections 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9038024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 --2- 172.21.15.66:0/2130030231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037f6620 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 -- 172.21.15.66:0/2130030231 >> 172.21.15.66:0/2130030231 conn(0x55c9037bdc90 msgr2=0x55c9037f1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 -- 172.21.15.66:0/2130030231 shutdown_connections 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 -- 172.21.15.66:0/2130030231 wait complete. 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 -- start start 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.664+0000 7f19b7811740 1 Processor -- start 2024-04-21T09:08:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19b7811740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19b7811740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19b7811740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9037fd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19b7811740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c903802db0 con 0x55c9037f6b60 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19b7811740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9036e3eb0 con 0x55c9038000e0 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19b7811740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c903802f30 con 0x55c9037f78d0 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19b7811740 10 client.0 _ll_register_callbacks cb 0x55c903710b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:33972/0 (socket says 172.21.15.66:33972) 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 -- 172.21.15.66:0/1386922729 learned_addr learned my addr 172.21.15.66:0/1386922729 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19ae59c640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19aed9d640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9037fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 -- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 msgr2=0x55c9037fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9037fd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 -- 172.21.15.66:0/1386922729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 msgr2=0x55c9037ffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037ffa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 -- 172.21.15.66:0/1386922729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c90346e210 con 0x55c9037f78d0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19aed9d640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9037fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f19a7fff640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037fce00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1998010de0 tx=0x7f199801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee208cc77d94d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f1995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.665+0000 7f1995ffb640 1 -- 172.21.15.66:0/1386922729 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998025070 con 0x55c9037f78d0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a002c880 con 0x55c9037f78d0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f1995ffb640 1 -- 172.21.15.66:0/1386922729 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1998005220 con 0x55c9037f78d0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f1995ffb640 1 -- 172.21.15.66:0/1386922729 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1998005420 con 0x55c9037f78d0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 10 client.25564 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c90353a620 con 0x55c9037f78d0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f1995ffb640 1 -- 172.21.15.66:0/1386922729 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a002c880 con 0x55c9037f78d0 2024-04-21T09:08:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f1995ffb640 1 -- 172.21.15.66:0/1386922729 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f199801f4e0 con 0x55c9037f78d0 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 10 client.25564 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 10 client.25564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 20 client.25564 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c903814540 0x55c903816930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 4 client.25564 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.666+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c90376f880 con 0x55c903814540 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.667+0000 7f19ae59c640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c903814540 0x55c903816930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.667+0000 7f19ae59c640 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c903814540 0x55c903816930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43021ebbf45861ad server_cookie=f90e8bc96fc0565a in_seq=0 out_seq=0 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.667+0000 7f1995ffb640 10 client.25564 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f1995ffb640 1 -- 172.21.15.66:0/1386922729 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c90376f880 con 0x55c903814540 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f1995ffb640 10 client.25564 handle_command_reply: tid=0 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 client.25564 shutdown 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 msgr2=0x55c9037fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037fce00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1998010de0 tx=0x7f199801a000 comp rx=0 tx=0).stop 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 shutdown_connections 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c903814540 0x55c903816930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9038000e0 0x55c9037fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037f78d0 0x55c9037fce00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f1995ffb640 0 client.25564 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 --2- 172.21.15.66:0/1386922729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f6b60 0x55c9037ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 >> 172.21.15.66:0/1386922729 conn(0x55c9037bdc90 msgr2=0x55c9037f35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 shutdown_connections 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 1 -- 172.21.15.66:0/1386922729 wait complete. 2024-04-21T09:08:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.668+0000 7f19b7811740 20 client.25564 trim_cache size 0 max 16384 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.692+0000 7f19b7811740 1 Processor -- start 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.692+0000 7f19b7811740 1 -- start start 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.692+0000 7f19b7811740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c90380bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.692+0000 7f19b7811740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9037f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.692+0000 7f19b7811740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9037ff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.692+0000 7f19b7811740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01af8b0 con 0x55c9037f78d0 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19b7811740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01ad180 con 0x55c9037f6b60 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19b7811740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01aafb0 con 0x55c9037bdc90 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9037ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9037ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36268/0 (socket says 172.21.15.66:36268) 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 -- 172.21.15.66:0/2555165445 learned_addr learned my addr 172.21.15.66:0/2555165445 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c90380bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19a7fff640 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9037f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 -- 172.21.15.66:0/2555165445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 msgr2=0x55c90380bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c90380bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 -- 172.21.15.66:0/2555165445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 msgr2=0x55c9037f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9037f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 -- 172.21.15.66:0/2555165445 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9037ecf30 con 0x55c9037f78d0 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f19aed9d640 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9037ff530 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000f80 tx=0x7f19a809c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281ab5af42b1be05 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.693+0000 7f1997fff640 1 -- 172.21.15.66:0/2555165445 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a80bb070 con 0x55c9037f78d0 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.694+0000 7f1997fff640 1 -- 172.21.15.66:0/2555165445 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a80a11e0 con 0x55c9037f78d0 2024-04-21T09:08:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.694+0000 7f1997fff640 1 -- 172.21.15.66:0/2555165445 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8099630 con 0x55c9037f78d0 2024-04-21T09:08:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.694+0000 7f19b7811740 1 -- 172.21.15.66:0/2555165445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 msgr2=0x55c9037ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.694+0000 7f19b7811740 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9037ff530 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000f80 tx=0x7f19a809c790 comp rx=0 tx=0).stop 2024-04-21T09:08:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- 172.21.15.66:0/2555165445 shutdown_connections 2024-04-21T09:08:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9037ff530 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9037f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 --2- 172.21.15.66:0/2555165445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c90380bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- 172.21.15.66:0/2555165445 >> 172.21.15.66:0/2555165445 conn(0x55c9037f6620 msgr2=0x55c9037f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- 172.21.15.66:0/2555165445 shutdown_connections 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- 172.21.15.66:0/2555165445 wait complete. 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- start start 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 Processor -- start 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c903803c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9038041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9038046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01ab640 con 0x55c9037f78d0 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01acab0 con 0x55c9037f6b60 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01abd10 con 0x55c9037bdc90 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.695+0000 7f19b7811740 10 client.0 _ll_register_callbacks cb 0x7f199800bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c903803c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c903803c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:33990/0 (socket says 172.21.15.66:33990) 2024-04-21T09:08:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 -- 172.21.15.66:0/2996232210 learned_addr learned my addr 172.21.15.66:0/2996232210 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19aed9d640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9038046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19a7fff640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9038041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 -- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 msgr2=0x55c9038041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9038041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 -- 172.21.15.66:0/2996232210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 msgr2=0x55c9038046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9038046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 -- 172.21.15.66:0/2996232210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c903879b60 con 0x55c9037bdc90 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19aed9d640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9038046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f19ae59c640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c903803c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f19a0036040 tx=0x7f19a00035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d11a92370a974d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f1995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.696+0000 7f1995ffb640 1 -- 172.21.15.66:0/2996232210 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0057060 con 0x55c9037bdc90 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f1995ffb640 1 -- 172.21.15.66:0/2996232210 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f19a0048650 con 0x55c9037bdc90 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a80a11e0 con 0x55c9037bdc90 2024-04-21T09:08:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f1995ffb640 1 -- 172.21.15.66:0/2996232210 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19a005e050 con 0x55c9037bdc90 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 10 client.25570 fetch_fsmap learned FSMap version 25 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c90387ae30 con 0x55c9037bdc90 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f1995ffb640 1 -- 172.21.15.66:0/2996232210 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a80a11e0 con 0x55c9037bdc90 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f1995ffb640 1 -- 172.21.15.66:0/2996232210 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f19a0051070 con 0x55c9037bdc90 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 10 client.25570 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 10 client.25570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 20 client.25570 populate_metadata read hostname 'smithi066' 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c9038000e0 0x55c90387d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 4 client.25570 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.697+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9036fb3d0 con 0x55c9038000e0 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.698+0000 7f19a7fff640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c9038000e0 0x55c90387d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.698+0000 7f19a7fff640 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c9038000e0 0x55c90387d1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943cff051023aa0b server_cookie=f06ce26a37597964 in_seq=0 out_seq=0 2024-04-21T09:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.698+0000 7f1995ffb640 10 client.25570 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f1995ffb640 1 -- 172.21.15.66:0/2996232210 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9036fb3d0 con 0x55c9038000e0 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f1995ffb640 10 client.25570 handle_command_reply: tid=0 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 client.25570 shutdown 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 msgr2=0x55c903803c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c903803c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f19a0036040 tx=0x7f19a00035f0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 shutdown_connections 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c9038000e0 0x55c90387d1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f1995ffb640 0 client.25570 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9037f78d0 0x55c9038046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9037f6b60 0x55c9038041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 --2- 172.21.15.66:0/2996232210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9037bdc90 0x55c903803c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 >> 172.21.15.66:0/2996232210 conn(0x55c9037f6620 msgr2=0x55c9037f0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 shutdown_connections 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 1 -- 172.21.15.66:0/2996232210 wait complete. 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b7811740 20 client.25570 trim_cache size 0 max 16384 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:08:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:08:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f199c0a3e60 msgr2=0x7f199c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.705+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f199c0a3e60 0x7f199c0a6320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f199800bfd0 tx=0x7f199800bf40 comp rx=0 tx=0).stop 2024-04-21T09:08:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b010c5b0 msgr2=0x7f19b019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:08:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b010c5b0 0x7f19b019cf90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f19a800eb40 tx=0x7f19a800c2d0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 shutdown_connections 2024-04-21T09:08:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f199c0a3e60 0x7f199c0a6320 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b010c5b0 0x7f19b019cf90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19b0103b00 0x7f19b0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 --2- 172.21.15.66:0/1984747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19b01031e0 0x7f19b0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:08:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 >> 172.21.15.66:0/1984747752 conn(0x7f19b0077f60 msgr2=0x7f19b00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:08:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 shutdown_connections 2024-04-21T09:08:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:08:58.706+0000 7f19b4b4b640 1 -- 172.21.15.66:0/1984747752 wait complete. 2024-04-21T09:08:59.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:08:59 smithi083 ceph-mon[31413]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T09:08:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:08:59 smithi066 ceph-mon[27953]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T09:08:59.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:08:59 smithi186 ceph-mon[31505]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T09:09:00.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:09:00 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:09:00.447+0000 7f1c40df2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:09:00.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:09:00 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:09:00.447+0000 7f1c40df2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:09:00.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:09:00 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:09:00.447+0000 7f1c40df2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:09:00.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:09:00 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:09:00.447+0000 7f1c40df2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:09:00.870 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:09:00 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:09:00.447+0000 7f1c40df2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T09:09:01.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:01 smithi083 ceph-mon[31413]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-21T09:09:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:01 smithi066 ceph-mon[27953]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-21T09:09:01.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:01 smithi186 ceph-mon[31505]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-21T09:09:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:03 smithi083 ceph-mon[31413]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-21T09:09:03.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:03 smithi083 ceph-mon[31413]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-04-21T09:09:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:03 smithi066 ceph-mon[27953]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-21T09:09:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:03 smithi066 ceph-mon[27953]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr:Note: switching to '4f6bc5273643aaf5660909e85400647a8951c10b'. 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr:state without impacting any branches by switching back to a branch. 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-21T09:09:03.641 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr: git switch -c 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr:Or undo this operation with: 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr: git switch - 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:03.642 INFO:tasks.workunit.client.0.smithi066.stderr:HEAD is now at 4f6bc527364 use ubuntu for fs/workload tests 2024-04-21T09:09:03.645 DEBUG:teuthology.orchestra.run.smithi066:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-21T09:09:03.680 INFO:tasks.workunit.client.0.smithi066.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-21T09:09:03.682 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T09:09:03.682 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-21T09:09:03.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:03 smithi186 ceph-mon[31505]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-21T09:09:03.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:03 smithi186 ceph-mon[31505]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-04-21T09:09:04.275 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-21T09:09:04.331 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-21T09:09:04.373 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T09:09:04.374 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T09:09:04.374 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-21T09:09:04.414 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T09:09:04.417 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:09:04.417 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-21T09:09:04.473 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-04-21T09:09:04.474 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-04-21T09:09:04.474 DEBUG:teuthology.orchestra.run.smithi066:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4f6bc5273643aaf5660909e85400647a8951c10b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-21T09:09:04.534 INFO:tasks.workunit.client.0.smithi066.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-04-21T09:09:04.550 INFO:tasks.workunit.client.0.smithi066.stderr:--2024-04-21 09:09:04-- http://download.ceph.com/qa/fsync-tester.c 2024-04-21T09:09:04.551 INFO:tasks.workunit.client.0.smithi066.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-21T09:09:04.587 INFO:tasks.workunit.client.0.smithi066.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-21T09:09:04.624 INFO:tasks.workunit.client.0.smithi066.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-21T09:09:04.624 INFO:tasks.workunit.client.0.smithi066.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-04-21T09:09:04.625 INFO:tasks.workunit.client.0.smithi066.stderr:Saving to: ‘fsync-tester.c’ 2024-04-21T09:09:04.629 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:04.629 INFO:tasks.workunit.client.0.smithi066.stderr: 0K .. 100% 532M=0s 2024-04-21T09:09:04.629 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:04.629 INFO:tasks.workunit.client.0.smithi066.stderr:2024-04-21 09:09:04 (532 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-04-21T09:09:04.629 INFO:tasks.workunit.client.0.smithi066.stderr: 2024-04-21T09:09:04.629 INFO:tasks.workunit.client.0.smithi066.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-04-21T09:09:04.698 INFO:tasks.workunit.client.0.smithi066.stderr:+ ./fsync-tester 2024-04-21T09:09:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:05 smithi083 ceph-mon[31413]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-21T09:09:05.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:05 smithi066 ceph-mon[27953]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-21T09:09:05.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:05 smithi186 ceph-mon[31505]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-21T09:09:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:07 smithi083 ceph-mon[31413]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-21T09:09:07.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:07 smithi066 ceph-mon[27953]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-21T09:09:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:07 smithi186 ceph-mon[31505]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-21T09:09:09.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:09 smithi066 ceph-mon[27953]: pgmap v31: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 26 op/s 2024-04-21T09:09:09.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:09 smithi186 ceph-mon[31505]: pgmap v31: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 26 op/s 2024-04-21T09:09:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:09 smithi083 ceph-mon[31413]: pgmap v31: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 26 op/s 2024-04-21T09:09:11.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:11 smithi066 ceph-mon[27953]: pgmap v32: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 372 KiB/s wr, 24 op/s 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 -- 172.21.15.66:0/2961774876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 msgr2=0x7fcf101014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 --2- 172.21.15.66:0/2961774876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf101014f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009900 tx=0x7fcf0002bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 -- 172.21.15.66:0/2961774876 shutdown_connections 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 --2- 172.21.15.66:0/2961774876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf10106190 0x7fcf1010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 --2- 172.21.15.66:0/2961774876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf10101ac0 0x7fcf10105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 --2- 172.21.15.66:0/2961774876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf101014f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 -- 172.21.15.66:0/2961774876 >> 172.21.15.66:0/2961774876 conn(0x7fcf10077fa0 msgr2=0x7fcf100783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 -- 172.21.15.66:0/2961774876 shutdown_connections 2024-04-21T09:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.799+0000 7fcf14d85640 1 -- 172.21.15.66:0/2961774876 wait complete. 2024-04-21T09:09:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 Processor -- start 2024-04-21T09:09:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 -- start start 2024-04-21T09:09:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf10199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf10106190 0x7fcf10199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf10199fd0 0x7fcf101a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101110f0 con 0x7fcf10101110 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10110f70 con 0x7fcf10199fd0 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf14d85640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10111270 con 0x7fcf10106190 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf10199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf10199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59366/0 (socket says 172.21.15.66:59366) 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf0e59c640 1 -- 172.21.15.66:0/1869728275 learned_addr learned my addr 172.21.15.66:0/1869728275 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.800+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf10199fd0 0x7fcf101a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf10106190 0x7fcf10199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0e59c640 1 -- 172.21.15.66:0/1869728275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf10106190 msgr2=0x7fcf10199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf10106190 0x7fcf10199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0e59c640 1 -- 172.21.15.66:0/1869728275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf10199fd0 msgr2=0x7fcf101a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf10199fd0 0x7fcf101a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0e59c640 1 -- 172.21.15.66:0/1869728275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf101a4d70 con 0x7fcf10101110 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf10106190 0x7fcf10199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf10199550 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005940 tx=0x7fcf000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf077fe640 1 -- 172.21.15.66:0/1869728275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00043020 con 0x7fcf10101110 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf101a5000 con 0x7fcf10101110 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf077fe640 1 -- 172.21.15.66:0/1869728275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0002caf0 con 0x7fcf10101110 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf077fe640 1 -- 172.21.15.66:0/1869728275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00003af0 con 0x7fcf10101110 2024-04-21T09:09:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.801+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf101a55a0 con 0x7fcf10101110 2024-04-21T09:09:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.803+0000 7fcf077fe640 1 -- 172.21.15.66:0/1869728275 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7fcf00046020 con 0x7fcf10101110 2024-04-21T09:09:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.803+0000 7fcf17974740 1 Processor -- start 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf17974740 1 -- start start 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf17974740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c6d8d0 0x558d39c6c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf17974740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf17974740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c7a4f0 0x558d39c7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf17974740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558d39bd18d0 con 0x558d39c6cb60 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf17974740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d39b6ee80 con 0x558d39c7a4f0 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf17974740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558d39bcd890 con 0x558d39c6d8d0 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c6cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59370/0 (socket says 172.21.15.66:59370) 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf0ed9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c6d8d0 0x558d39c6c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3287880497 learned_addr learned my addr 172.21.15.66:0/3287880497 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c7a4f0 0x558d39c7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.804+0000 7fcf077fe640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcefc0a3f10 0x7fcefc0a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3287880497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c6d8d0 msgr2=0x558d39c6c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c6d8d0 0x558d39c6c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3287880497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c7a4f0 msgr2=0x558d39c7c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c7a4f0 0x558d39c7c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3287880497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d39b90320 con 0x558d39c6cb60 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c7a4f0 0x558d39c7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c6d8d0 0x558d39c6c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcefc0a3f10 0x7fcefc0a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf077fe640 1 -- 172.21.15.66:0/1869728275 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fcf000f2eb0 con 0x7fcf10101110 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c6cfe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcf000f01c0 tx=0x7fcf0007e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b8897953b3b44a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf04ff9640 1 -- 172.21.15.66:0/3287880497 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000be770 con 0x558d39c6cb60 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcefc0a3f10 0x7fcefc0a63d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009fd0 tx=0x7fcf08004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf04ff9640 1 -- 172.21.15.66:0/3287880497 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf000bf0a0 con 0x558d39c6cb60 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.805+0000 7fcf04ff9640 1 -- 172.21.15.66:0/3287880497 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000be300 con 0x558d39c6cb60 2024-04-21T09:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 -- 172.21.15.66:0/3287880497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 msgr2=0x558d39c6cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c6cfe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcf000f01c0 tx=0x7fcf0007e440 comp rx=0 tx=0).stop 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 -- 172.21.15.66:0/3287880497 shutdown_connections 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c7a4f0 0x558d39c7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c6cfe0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 --2- 172.21.15.66:0/3287880497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c6d8d0 0x558d39c6c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 -- 172.21.15.66:0/3287880497 >> 172.21.15.66:0/3287880497 conn(0x558d39c33c90 msgr2=0x558d39c67850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 -- 172.21.15.66:0/3287880497 shutdown_connections 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 -- 172.21.15.66:0/3287880497 wait complete. 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 -- start start 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.806+0000 7fcf17974740 1 Processor -- start 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf17974740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6cb60 0x558d39c79e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf17974740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6d8d0 0x558d39c77210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf17974740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c77790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf17974740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558d39c7cfe0 con 0x558d39c6d8d0 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf17974740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d39b59eb0 con 0x558d39c6cb60 2024-04-21T09:09:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf17974740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558d39c7d160 con 0x558d39c7a4f0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf17974740 10 client.0 _ll_register_callbacks cb 0x558d39b86b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6d8d0 0x558d39c77210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6d8d0 0x558d39c77210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59386/0 (socket says 172.21.15.66:59386) 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 -- 172.21.15.66:0/2688713287 learned_addr learned my addr 172.21.15.66:0/2688713287 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c77790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6cb60 0x558d39c79e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 -- 172.21.15.66:0/2688713287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 msgr2=0x558d39c77790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c77790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 -- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6cb60 msgr2=0x558d39c79e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6cb60 0x558d39c79e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 -- 172.21.15.66:0/2688713287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d398e4210 con 0x558d39c6d8d0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c77790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6d8d0 0x558d39c77210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009f90 tx=0x7fcef8011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31b51d4e0c8bd8b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.807+0000 7fcef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcef5ffb640 1 -- 172.21.15.66:0/2688713287 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8012920 con 0x558d39c6d8d0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcef5ffb640 1 -- 172.21.15.66:0/2688713287 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fcef8012ac0 con 0x558d39c6d8d0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf000bf0a0 con 0x558d39c6d8d0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcef5ffb640 1 -- 172.21.15.66:0/2688713287 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcef8025050 con 0x558d39c6d8d0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 10 client.16068 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d399b0620 con 0x558d39c6d8d0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcef5ffb640 1 -- 172.21.15.66:0/2688713287 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf000bf0a0 con 0x558d39c6d8d0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcef5ffb640 1 -- 172.21.15.66:0/2688713287 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcef800f040 con 0x558d39c6d8d0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 10 client.16068 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 20 client.16068 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39c8a300 0x558d39c8c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 4 client.16068 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.808+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d39be5880 con 0x558d39c8a300 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.809+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39c8a300 0x558d39c8c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.809+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39c8a300 0x558d39c8c6f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a38b807fa9675d server_cookie=cfab142512c62733 in_seq=0 out_seq=0 2024-04-21T09:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.809+0000 7fcef5ffb640 10 client.16068 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcef5ffb640 1 -- 172.21.15.66:0/2688713287 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d39be5880 con 0x558d39c8a300 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcef5ffb640 10 client.16068 handle_command_reply: tid=0 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 client.16068 shutdown 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6d8d0 msgr2=0x558d39c77210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6d8d0 0x558d39c77210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009f90 tx=0x7fcef8011870 comp rx=0 tx=0).stop 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 shutdown_connections 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39c8a300 0x558d39c8c6f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c77790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcef5ffb640 0 client.16068 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6d8d0 0x558d39c77210 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 --2- 172.21.15.66:0/2688713287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6cb60 0x558d39c79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 >> 172.21.15.66:0/2688713287 conn(0x558d39c33c90 msgr2=0x558d39c695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 shutdown_connections 2024-04-21T09:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 1 -- 172.21.15.66:0/2688713287 wait complete. 2024-04-21T09:09:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.810+0000 7fcf17974740 20 client.16068 trim_cache size 0 max 16384 2024-04-21T09:09:11.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:11 smithi186 ceph-mon[31505]: pgmap v32: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 372 KiB/s wr, 24 op/s 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.842+0000 7fcf17974740 1 Processor -- start 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.842+0000 7fcf17974740 1 -- start start 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf17974740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c79940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf17974740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c79e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf17974740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c91fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf17974740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101af970 con 0x558d39c6cb60 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf17974740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101ad240 con 0x558d39c6d8d0 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf17974740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101ab070 con 0x558d39c7a4f0 2024-04-21T09:09:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c79940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c79940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59392/0 (socket says 172.21.15.66:59392) 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 -- 172.21.15.66:0/4246249340 learned_addr learned my addr 172.21.15.66:0/4246249340 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c79e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c91fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 -- 172.21.15.66:0/4246249340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 msgr2=0x558d39c91fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c91fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 -- 172.21.15.66:0/4246249340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 msgr2=0x558d39c79e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c79e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 -- 172.21.15.66:0/4246249340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d39c62f30 con 0x558d39c6cb60 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c79940 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcf000bcc50 tx=0x7fcf000f6980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93b8161de6b471c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcef7fff640 1 -- 172.21.15.66:0/4246249340 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000be4e0 con 0x558d39c6cb60 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcef7fff640 1 -- 172.21.15.66:0/4246249340 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf000bf0a0 con 0x558d39c6cb60 2024-04-21T09:09:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.843+0000 7fcef7fff640 1 -- 172.21.15.66:0/4246249340 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000c7610 con 0x558d39c6cb60 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 -- 172.21.15.66:0/4246249340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 msgr2=0x558d39c79940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c79940 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcf000bcc50 tx=0x7fcf000f6980 comp rx=0 tx=0).stop 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 -- 172.21.15.66:0/4246249340 shutdown_connections 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c91fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 --2- 172.21.15.66:0/4246249340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c79940 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 -- 172.21.15.66:0/4246249340 >> 172.21.15.66:0/4246249340 conn(0x558d39c33c90 msgr2=0x558d39c72490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 -- 172.21.15.66:0/4246249340 shutdown_connections 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.844+0000 7fcf17974740 1 -- 172.21.15.66:0/4246249340 wait complete. 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 -- start start 2024-04-21T09:09:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 Processor -- start 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c825a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c73b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c74060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101ab700 con 0x558d39c6cb60 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101acb70 con 0x558d39c6d8d0 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101abdd0 con 0x558d39c7a4f0 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf17974740 10 client.0 _ll_register_callbacks cb 0x7fcef80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c825a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59404/0 (socket says 172.21.15.66:59404) 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3444313225 learned_addr learned my addr 172.21.15.66:0/3444313225 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c73b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0ed9d640 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c74060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3444313225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 msgr2=0x558d39c74060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c74060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 msgr2=0x558d39c73b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c73b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.845+0000 7fcf0e59c640 1 -- 172.21.15.66:0/3444313225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d39c746a0 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf0e59c640 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c825a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcf000a9010 tx=0x7fcf000be4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82008031d5faa84 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcef5ffb640 1 -- 172.21.15.66:0/3444313225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000be300 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcef5ffb640 1 -- 172.21.15.66:0/3444313225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fcf000bb010 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf000bf0a0 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcef5ffb640 1 -- 172.21.15.66:0/3444313225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf000be770 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 10 client.16074 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d39cf0660 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcef5ffb640 1 -- 172.21.15.66:0/3444313225 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf000bf0a0 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcef5ffb640 1 -- 172.21.15.66:0/3444313225 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf000053b0 con 0x558d39c6cb60 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 10 client.16074 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 20 client.16074 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39cf09a0 0x558d39cf2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 4 client.16074 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:09:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.846+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d39b713d0 con 0x558d39cf09a0 2024-04-21T09:09:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.847+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39cf09a0 0x558d39cf2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.847+0000 7fcf0dd9b640 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39cf09a0 0x558d39cf2d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d5a3bf828ea8ae server_cookie=40809dbd75e69864 in_seq=0 out_seq=0 2024-04-21T09:09:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.847+0000 7fcef5ffb640 10 client.16074 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.859+0000 7fcef5ffb640 1 -- 172.21.15.66:0/3444313225 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d39b713d0 con 0x558d39cf09a0 2024-04-21T09:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.859+0000 7fcef5ffb640 10 client.16074 handle_command_reply: tid=0 2024-04-21T09:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.859+0000 7fcf17974740 1 client.16074 shutdown 2024-04-21T09:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 msgr2=0x558d39c825a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c825a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcf000a9010 tx=0x7fcf000be4e0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 shutdown_connections 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x558d39cf09a0 0x558d39cf2d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d39c7a4f0 0x558d39c74060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d39c6d8d0 0x558d39c73b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcef5ffb640 0 client.16074 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 --2- 172.21.15.66:0/3444313225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558d39c6cb60 0x558d39c825a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 >> 172.21.15.66:0/3444313225 conn(0x558d39c33c90 msgr2=0x558d39c756b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 shutdown_connections 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 1 -- 172.21.15.66:0/3444313225 wait complete. 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf17974740 20 client.16074 trim_cache size 0 max 16384 2024-04-21T09:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:09:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:09:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:09:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:09:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcefc0a3f10 msgr2=0x7fcefc0a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcefc0a3f10 0x7fcefc0a63d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009fd0 tx=0x7fcf08004a40 comp rx=0 tx=0).stop 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 msgr2=0x7fcf10199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf10199550 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005940 tx=0x7fcf000045e0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 shutdown_connections 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcefc0a3f10 0x7fcefc0a63d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf10199fd0 0x7fcf101a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf10106190 0x7fcf10199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 --2- 172.21.15.66:0/1869728275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf10101110 0x7fcf10199550 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.860+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 >> 172.21.15.66:0/1869728275 conn(0x7fcf10077fa0 msgr2=0x7fcf100ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.861+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 shutdown_connections 2024-04-21T09:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:11.861+0000 7fcf14d85640 1 -- 172.21.15.66:0/1869728275 wait complete. 2024-04-21T09:09:11.863 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:11 smithi083 ceph-mon[31413]: pgmap v32: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 372 KiB/s wr, 24 op/s 2024-04-21T09:09:13.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:13 smithi066 ceph-mon[27953]: pgmap v33: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 372 KiB/s wr, 24 op/s 2024-04-21T09:09:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:13 smithi186 ceph-mon[31505]: pgmap v33: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 372 KiB/s wr, 24 op/s 2024-04-21T09:09:13.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:13 smithi083 ceph-mon[31413]: pgmap v33: 129 pgs: 129 active+clean; 2.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 372 KiB/s wr, 24 op/s 2024-04-21T09:09:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:15 smithi186 ceph-mon[31505]: pgmap v34: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 805 KiB/s wr, 31 op/s 2024-04-21T09:09:15.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:15 smithi083 ceph-mon[31413]: pgmap v34: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 805 KiB/s wr, 31 op/s 2024-04-21T09:09:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:15 smithi066 ceph-mon[27953]: pgmap v34: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 805 KiB/s wr, 31 op/s 2024-04-21T09:09:17.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:17 smithi186 ceph-mon[31505]: pgmap v35: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 781 KiB/s wr, 12 op/s 2024-04-21T09:09:17.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:17 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:09:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:17 smithi083 ceph-mon[31413]: pgmap v35: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 781 KiB/s wr, 12 op/s 2024-04-21T09:09:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:17 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:09:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:17 smithi066 ceph-mon[27953]: pgmap v35: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 781 KiB/s wr, 12 op/s 2024-04-21T09:09:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:17 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:09:19.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:09:19.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:09:19.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:18 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:09:19.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:09:19.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:18 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:09:19.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:18 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:09:19.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:19 smithi186 ceph-mon[31505]: pgmap v36: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-21T09:09:20.011 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:19 smithi083 ceph-mon[31413]: pgmap v36: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-21T09:09:20.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:19 smithi066 ceph-mon[27953]: pgmap v36: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-21T09:09:20.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:09:20.503 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:09:20.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2974696995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 msgr2=0x7ff0e810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2974696995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e810dd20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009ca0 tx=0x7ff0e002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2974696995 shutdown_connections 2024-04-21T09:09:20.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2974696995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e810dd20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2974696995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e8101a30 0x7ff0e8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2974696995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0e8101080 0x7ff0e8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2974696995 >> 172.21.15.66:0/2974696995 conn(0x7ff0e8077f30 msgr2=0x7ff0e8078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:20.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2974696995 shutdown_connections 2024-04-21T09:09:20.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.615+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2974696995 wait complete. 2024-04-21T09:09:20.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 Processor -- start 2024-04-21T09:09:20.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 -- start start 2024-04-21T09:09:20.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e8101080 0x7ff0e8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0e819bec0 0x7ff0e819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81110a0 con 0x7ff0e8106150 2024-04-21T09:09:20.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8110f20 con 0x7ff0e819bec0 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.616+0000 7ff0ed6b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8111220 con 0x7ff0e8101080 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0e819bec0 0x7ff0e819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54656/0 (socket says 172.21.15.66:54656) 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e67fc640 1 -- 172.21.15.66:0/2732261226 learned_addr learned my addr 172.21.15.66:0/2732261226 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e8101080 0x7ff0e8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e77fe640 1 -- 172.21.15.66:0/2732261226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e8101080 msgr2=0x7ff0e8197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e77fe640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e8101080 0x7ff0e8197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e77fe640 1 -- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 msgr2=0x7ff0e8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e77fe640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e77fe640 1 -- 172.21.15.66:0/2732261226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e819c9b0 con 0x7ff0e819bec0 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e8101080 0x7ff0e8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0e77fe640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0e819bec0 0x7ff0e819c370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0e002c700 tx=0x7ff0e00346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0cffff640 1 -- 172.21.15.66:0/2732261226 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0039040 con 0x7ff0e819bec0 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e819cc40 con 0x7ff0e819bec0 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0cffff640 1 -- 172.21.15.66:0/2732261226 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e0044070 con 0x7ff0e819bec0 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.617+0000 7ff0cffff640 1 -- 172.21.15.66:0/2732261226 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00042a0 con 0x7ff0e819bec0 2024-04-21T09:09:20.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.618+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81a43a0 con 0x7ff0e819bec0 2024-04-21T09:09:20.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.620+0000 7ff0f037f740 1 Processor -- start 2024-04-21T09:09:20.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0cffff640 1 -- 172.21.15.66:0/2732261226 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7ff0e0004440 con 0x7ff0e819bec0 2024-04-21T09:09:20.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0f037f740 1 -- start start 2024-04-21T09:09:20.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb5b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb66750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb69080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0f037f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5652bdac01e0 con 0x5652bdb5b930 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0f037f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652bda5d390 con 0x5652bdb5af80 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0f037f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5652bdabc1a0 con 0x5652bdb66c90 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb5b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb66750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb5b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:43202/0 (socket says 172.21.15.66:43202) 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1486369313 learned_addr learned my addr 172.21.15.66:0/1486369313 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb69080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0cffff640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff0d00a3e40 0x7ff0d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1486369313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 msgr2=0x5652bdb69080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb69080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1486369313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 msgr2=0x5652bdb66750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb66750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1486369313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652bd89c7d0 con 0x5652bdb5af80 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.621+0000 7ff0cffff640 1 -- 172.21.15.66:0/2732261226 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff0e00f2f40 con 0x7ff0e819bec0 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.622+0000 7ff0e77fe640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff0d00a3e40 0x7ff0d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.622+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb5b360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00bfd0 tx=0x7ff0dc005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f34ddbef2c12776 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.622+0000 7ff0cd7fa640 1 -- 172.21.15.66:0/1486369313 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc012c70 con 0x5652bdb5af80 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.622+0000 7ff0cd7fa640 1 -- 172.21.15.66:0/1486369313 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0dc012e10 con 0x5652bdb5af80 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.622+0000 7ff0cd7fa640 1 -- 172.21.15.66:0/1486369313 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc005560 con 0x5652bdb5af80 2024-04-21T09:09:20.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- 172.21.15.66:0/1486369313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 msgr2=0x5652bdb5b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb5b360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00bfd0 tx=0x7ff0dc005d90 comp rx=0 tx=0).stop 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0e77fe640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff0d00a3e40 0x7ff0d00a6300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00f0310 tx=0x7ff0e002c0f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- 172.21.15.66:0/1486369313 shutdown_connections 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb66750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1486369313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb5b360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- 172.21.15.66:0/1486369313 >> 172.21.15.66:0/1486369313 conn(0x5652bdb55cb0 msgr2=0x5652bdb56090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- 172.21.15.66:0/1486369313 shutdown_connections 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- 172.21.15.66:0/1486369313 wait complete. 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- start start 2024-04-21T09:09:20.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 Processor -- start 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5af80 0x5652bdb63ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb66c90 0x5652bdb64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5652bdb69a80 con 0x5652bdb5b930 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652bda792f0 con 0x5652bdb66c90 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5652bdb69c00 con 0x5652bdb5af80 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.623+0000 7ff0f037f740 10 client.0 _ll_register_callbacks cb 0x5652bda86580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb64220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54676/0 (socket says 172.21.15.66:54676) 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 -- 172.21.15.66:0/823257677 learned_addr learned my addr 172.21.15.66:0/823257677 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb66c90 0x5652bdb64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5af80 0x5652bdb63ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 -- 172.21.15.66:0/823257677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5af80 msgr2=0x5652bdb63ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5af80 0x5652bdb63ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 -- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb66c90 msgr2=0x5652bdb64760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb66c90 0x5652bdb64760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 -- 172.21.15.66:0/823257677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652bd7d23f0 con 0x5652bdb5b930 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb66c90 0x5652bdb64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb64220 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8013740 tx=0x7ff0d8013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7540ba89842c08c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0caff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0caff5640 1 -- 172.21.15.66:0/823257677 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8004220 con 0x5652bdb5b930 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0dc012e10 con 0x5652bdb5b930 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0caff5640 1 -- 172.21.15.66:0/823257677 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff0d80043c0 con 0x5652bdb5b930 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0caff5640 1 -- 172.21.15.66:0/823257677 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0d80119c0 con 0x5652bdb5b930 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0f037f740 10 client.16086 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.624+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5652bd7920f0 con 0x5652bdb5b930 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0caff5640 1 -- 172.21.15.66:0/823257677 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0dc012e10 con 0x5652bdb5b930 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0caff5640 1 -- 172.21.15.66:0/823257677 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff0d801a070 con 0x5652bdb5b930 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0f037f740 10 client.16086 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0f037f740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:20.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0f037f740 20 client.16086 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0f037f740 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdb76d90 0x5652bdb791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0f037f740 4 client.16086 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652bdacdc00 con 0x5652bdb76d90 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdb76d90 0x5652bdb791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.625+0000 7ff0caff5640 10 client.16086 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdb76d90 0x5652bdb791f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df3b191bc5125b5 server_cookie=eb48102ae31e70e8 in_seq=0 out_seq=0 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0caff5640 1 -- 172.21.15.66:0/823257677 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652bdacdc00 con 0x5652bdb76d90 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0caff5640 10 client.16086 handle_command_reply: tid=0 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 client.16086 shutdown 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 msgr2=0x5652bdb64220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb64220 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8013740 tx=0x7ff0d8013c00 comp rx=0 tx=0).stop 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 shutdown_connections 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdb76d90 0x5652bdb791f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb66c90 0x5652bdb64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb64220 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0caff5640 0 client.16086 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 --2- 172.21.15.66:0/823257677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5af80 0x5652bdb63ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.626+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 >> 172.21.15.66:0/823257677 conn(0x5652bdb55cb0 msgr2=0x5652bdb57a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.627+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 shutdown_connections 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.627+0000 7ff0f037f740 1 -- 172.21.15.66:0/823257677 wait complete. 2024-04-21T09:09:20.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.627+0000 7ff0f037f740 20 client.16086 trim_cache size 0 max 16384 2024-04-21T09:09:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.644+0000 7ff0f037f740 1 Processor -- start 2024-04-21T09:09:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.644+0000 7ff0f037f740 1 -- start start 2024-04-21T09:09:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb66410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5b930 0x5652bdb72980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb66c90 0x5652bdb7eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0f037f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81af960 con 0x5652bdb66c90 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0f037f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81ad230 con 0x5652bdb5af80 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0f037f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81ab060 con 0x5652bdb5b930 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb66c90 0x5652bdb7eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb66c90 0x5652bdb7eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54688/0 (socket says 172.21.15.66:54688) 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1504623876 learned_addr learned my addr 172.21.15.66:0/1504623876 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb66410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5b930 0x5652bdb72980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1504623876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5b930 msgr2=0x5652bdb72980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5b930 0x5652bdb72980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1504623876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 msgr2=0x5652bdb66410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb66410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 -- 172.21.15.66:0/1504623876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652bdb69bd0 con 0x5652bdb66c90 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb66c90 0x5652bdb7eb00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005130 tx=0x7ff0dc005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33e41558b715e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0ccff9640 1 -- 172.21.15.66:0/1504623876 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc005270 con 0x5652bdb66c90 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0ccff9640 1 -- 172.21.15.66:0/1504623876 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0dc016030 con 0x5652bdb66c90 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.645+0000 7ff0ccff9640 1 -- 172.21.15.66:0/1504623876 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc003850 con 0x5652bdb66c90 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 -- 172.21.15.66:0/1504623876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb66c90 msgr2=0x5652bdb7eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb66c90 0x5652bdb7eb00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005130 tx=0x7ff0dc005200 comp rx=0 tx=0).stop 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 -- 172.21.15.66:0/1504623876 shutdown_connections 2024-04-21T09:09:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb66c90 0x5652bdb7eb00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb5b930 0x5652bdb72980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 --2- 172.21.15.66:0/1504623876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb66410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 -- 172.21.15.66:0/1504623876 >> 172.21.15.66:0/1504623876 conn(0x5652bdb55cb0 msgr2=0x5652bdb57970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 -- 172.21.15.66:0/1504623876 shutdown_connections 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.646+0000 7ff0f037f740 1 -- 172.21.15.66:0/1504623876 wait complete. 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 -- start start 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 Processor -- start 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb6f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb60a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81ab6f0 con 0x5652bdb5b930 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81acb60 con 0x5652bdb5af80 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81abdc0 con 0x5652bdb66c90 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0f037f740 10 client.0 _ll_register_callbacks cb 0x7ff0d8009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb6f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb60550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54704/0 (socket says 172.21.15.66:54704) 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e6ffd640 1 -- 172.21.15.66:0/2209428278 learned_addr learned my addr 172.21.15.66:0/2209428278 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb60a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 -- 172.21.15.66:0/2209428278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 msgr2=0x5652bdb60a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb60a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 -- 172.21.15.66:0/2209428278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 msgr2=0x5652bdb60550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb60550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 -- 172.21.15.66:0/2209428278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652bdb61100 con 0x5652bdb5af80 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0e67fc640 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb6f0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005130 tx=0x7ff0dc02f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92722484bdb0ff35 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.647+0000 7ff0caff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0caff5640 1 -- 172.21.15.66:0/2209428278 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc010cd0 con 0x5652bdb5af80 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0dc016030 con 0x5652bdb5af80 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0caff5640 1 -- 172.21.15.66:0/2209428278 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff0dc005270 con 0x5652bdb5af80 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0caff5640 1 -- 172.21.15.66:0/2209428278 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0dc042050 con 0x5652bdb5af80 2024-04-21T09:09:20.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 10 client.25694 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5652bdbdd210 con 0x5652bdb5af80 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0caff5640 1 -- 172.21.15.66:0/2209428278 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0dc016030 con 0x5652bdb5af80 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0caff5640 1 -- 172.21.15.66:0/2209428278 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff0dc039070 con 0x5652bdb5af80 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 10 client.25694 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 20 client.25694 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdbdd750 0x5652bdbdfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 4 client.25694 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652bdac7a70 con 0x5652bdbdd750 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.648+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdbdd750 0x5652bdbdfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0caff5640 10 client.25694 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0e6ffd640 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdbdd750 0x5652bdbdfb70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=139d4f332b005383 server_cookie=f697a006debd71b in_seq=0 out_seq=0 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0caff5640 1 -- 172.21.15.66:0/2209428278 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652bdac7a70 con 0x5652bdbdd750 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0caff5640 10 client.25694 handle_command_reply: tid=0 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 client.25694 shutdown 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 msgr2=0x5652bdb6f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb6f0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005130 tx=0x7ff0dc02f040 comp rx=0 tx=0).stop 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 shutdown_connections 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5652bdbdd750 0x5652bdbdfb70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652bdb66c90 0x5652bdb60a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652bdb5b930 0x5652bdb60550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 --2- 172.21.15.66:0/2209428278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652bdb5af80 0x5652bdb6f0d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0caff5640 0 client.25694 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 >> 172.21.15.66:0/2209428278 conn(0x5652bdb55cb0 msgr2=0x5652bdb570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 shutdown_connections 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 1 -- 172.21.15.66:0/2209428278 wait complete. 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.649+0000 7ff0f037f740 20 client.25694 trim_cache size 0 max 16384 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:09:20.658 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:09:20.659 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:09:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff0d00a3e40 msgr2=0x7ff0d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff0d00a3e40 0x7ff0d00a6300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00f0310 tx=0x7ff0e002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0e819bec0 msgr2=0x7ff0e819c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0e819bec0 0x7ff0e819c370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0e002c700 tx=0x7ff0e00346f0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 shutdown_connections 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff0d00a3e40 0x7ff0d00a6300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0e819bec0 0x7ff0e819c370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e8106150 0x7ff0e8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 --2- 172.21.15.66:0/2732261226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e8101080 0x7ff0e8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 >> 172.21.15.66:0/2732261226 conn(0x7ff0e8077f30 msgr2=0x7ff0e80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 shutdown_connections 2024-04-21T09:09:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.650+0000 7ff0ed6b3640 1 -- 172.21.15.66:0/2732261226 wait complete. 2024-04-21T09:09:20.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:09:20.662 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:09:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 -- 172.21.15.66:0/598531879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8101b60 msgr2=0x7feaa810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 --2- 172.21.15.66:0/598531879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8101b60 0x7feaa810dc90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009ca0 tx=0x7feaa002c140 comp rx=0 tx=0).stop 2024-04-21T09:09:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 -- 172.21.15.66:0/598531879 shutdown_connections 2024-04-21T09:09:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 --2- 172.21.15.66:0/598531879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8101b60 0x7feaa810dc90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 --2- 172.21.15.66:0/598531879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa81011a0 0x7feaa8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 --2- 172.21.15.66:0/598531879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa8105ee0 0x7feaa81062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 -- 172.21.15.66:0/598531879 >> 172.21.15.66:0/598531879 conn(0x7feaa8077c20 msgr2=0x7feaa8078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 -- 172.21.15.66:0/598531879 shutdown_connections 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 -- 172.21.15.66:0/598531879 wait complete. 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 Processor -- start 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.770+0000 7feaae86d640 1 -- start start 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaae86d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa81011a0 0x7feaa8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaae86d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa8101b60 0x7feaa8199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaae86d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8105ee0 0x7feaa81a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaae86d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaa81110d0 con 0x7feaa8105ee0 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaae86d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feaa8110f50 con 0x7feaa81011a0 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaae86d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feaa8111250 con 0x7feaa8101b60 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8105ee0 0x7feaa81a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8105ee0 0x7feaa81a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54720/0 (socket says 172.21.15.66:54720) 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 -- 172.21.15.66:0/1545630792 learned_addr learned my addr 172.21.15.66:0/1545630792 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaa7fff640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa81011a0 0x7feaa8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaa77fe640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa8101b60 0x7feaa8199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 -- 172.21.15.66:0/1545630792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa8101b60 msgr2=0x7feaa8199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa8101b60 0x7feaa8199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 -- 172.21.15.66:0/1545630792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa81011a0 msgr2=0x7feaa8199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa81011a0 0x7feaa8199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaace0a640 1 -- 172.21.15.66:0/1545630792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa81a4af0 con 0x7feaa8105ee0 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.771+0000 7feaa7fff640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa81011a0 0x7feaa8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:20.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.772+0000 7feaace0a640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8105ee0 0x7feaa81a45b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feaa002c750 tx=0x7feaa0035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.772+0000 7feaa57fa640 1 -- 172.21.15.66:0/1545630792 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0039040 con 0x7feaa8105ee0 2024-04-21T09:09:20.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.772+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa81a4cc0 con 0x7feaa8105ee0 2024-04-21T09:09:20.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.772+0000 7feaa57fa640 1 -- 172.21.15.66:0/1545630792 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa0044070 con 0x7feaa8105ee0 2024-04-21T09:09:20.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.772+0000 7feaa57fa640 1 -- 172.21.15.66:0/1545630792 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa00042a0 con 0x7feaa8105ee0 2024-04-21T09:09:20.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.772+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa81a5120 con 0x7feaa8105ee0 2024-04-21T09:09:20.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.773+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa8102570 con 0x7feaa8105ee0 2024-04-21T09:09:20.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.774+0000 7feaa57fa640 1 -- 172.21.15.66:0/1545630792 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7feaa0004440 con 0x7feaa8105ee0 2024-04-21T09:09:20.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.775+0000 7feaa57fa640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea940a3e40 0x7fea940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:20.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.775+0000 7feaa57fa640 1 -- 172.21.15.66:0/1545630792 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7feaa00f3290 con 0x7feaa8105ee0 2024-04-21T09:09:20.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.775+0000 7feaa7fff640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea940a3e40 0x7fea940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:20.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.775+0000 7feaa7fff640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea940a3e40 0x7fea940a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea980095f0 tx=0x7fea98005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:20.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:20.780+0000 7feaa57fa640 1 -- 172.21.15.66:0/1545630792 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feaa00bf480 con 0x7feaa8105ee0 2024-04-21T09:09:21.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.010+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaa8109350 con 0x7feaa8105ee0 2024-04-21T09:09:21.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.010+0000 7feaa57fa640 1 -- 172.21.15.66:0/1545630792 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7feaa00bed80 con 0x7feaa8105ee0 2024-04-21T09:09:21.010 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:09:21.010 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:09:21.011 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:09:21.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea940a3e40 msgr2=0x7fea940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea940a3e40 0x7fea940a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea980095f0 tx=0x7fea98005f90 comp rx=0 tx=0).stop 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8105ee0 msgr2=0x7feaa81a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8105ee0 0x7feaa81a45b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feaa002c750 tx=0x7feaa0035d60 comp rx=0 tx=0).stop 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 shutdown_connections 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea940a3e40 0x7fea940a6300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa8105ee0 0x7feaa81a45b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa8101b60 0x7feaa8199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 --2- 172.21.15.66:0/1545630792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa81011a0 0x7feaa8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 >> 172.21.15.66:0/1545630792 conn(0x7feaa8077c20 msgr2=0x7feaa80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 shutdown_connections 2024-04-21T09:09:21.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.013+0000 7feaae86d640 1 -- 172.21.15.66:0/1545630792 wait complete. 2024-04-21T09:09:21.024 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 -- 172.21.15.66:0/2199825572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678101a30 msgr2=0x7fe678105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 --2- 172.21.15.66:0/2199825572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678101a30 0x7fe678105a70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe668009900 tx=0x7fe66802bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 -- 172.21.15.66:0/2199825572 shutdown_connections 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 --2- 172.21.15.66:0/2199825572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678106150 0x7fe67810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 --2- 172.21.15.66:0/2199825572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678101a30 0x7fe678105a70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 --2- 172.21.15.66:0/2199825572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe678101080 0x7fe678101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 -- 172.21.15.66:0/2199825572 >> 172.21.15.66:0/2199825572 conn(0x7fe678077ef0 msgr2=0x7fe678078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 -- 172.21.15.66:0/2199825572 shutdown_connections 2024-04-21T09:09:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.134+0000 7fe67cb75640 1 -- 172.21.15.66:0/2199825572 wait complete. 2024-04-21T09:09:21.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 Processor -- start 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 -- start start 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe678101080 0x7fe67819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678106150 0x7fe67819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678198610 0x7fe678198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781121f0 con 0x7fe678198610 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe678112070 con 0x7fe678106150 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe67cb75640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe678112370 con 0x7fe678101080 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678198610 0x7fe678198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe675d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678106150 0x7fe67819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678198610 0x7fe678198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54736/0 (socket says 172.21.15.66:54736) 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.135+0000 7fe676d9d640 1 -- 172.21.15.66:0/1272862144 learned_addr learned my addr 172.21.15.66:0/1272862144 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe67659c640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe678101080 0x7fe67819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe675d9b640 1 -- 172.21.15.66:0/1272862144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe678101080 msgr2=0x7fe67819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe678101080 0x7fe67819e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe675d9b640 1 -- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678198610 msgr2=0x7fe678198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678198610 0x7fe678198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe675d9b640 1 -- 172.21.15.66:0/1272862144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe678199130 con 0x7fe678106150 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe676d9d640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678198610 0x7fe678198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe67659c640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe678101080 0x7fe67819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678106150 0x7fe67819e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe668036040 tx=0x7fe6680045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe66f7fe640 1 -- 172.21.15.66:0/1272862144 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668043020 con 0x7fe678106150 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6781993c0 con 0x7fe678106150 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe66f7fe640 1 -- 172.21.15.66:0/1272862144 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe66802cb70 con 0x7fe678106150 2024-04-21T09:09:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe66f7fe640 1 -- 172.21.15.66:0/1272862144 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668003b50 con 0x7fe678106150 2024-04-21T09:09:21.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.136+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6781a5660 con 0x7fe678106150 2024-04-21T09:09:21.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.138+0000 7fe66f7fe640 1 -- 172.21.15.66:0/1272862144 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7fe668046020 con 0x7fe678106150 2024-04-21T09:09:21.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 Processor -- start 2024-04-21T09:09:21.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 -- start start 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b40d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40de80 0x559a1b416430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b416a00 0x559a1b418e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe66f7fe640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe6640a3e40 0x7fe6640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559a1b3721f0 con 0x559a1b40de80 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a1b30f4b0 con 0x559a1b416a00 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559a1b36e1b0 con 0x559a1b40d4d0 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.139+0000 7fe67659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40de80 0x559a1b416430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe675d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b416a00 0x559a1b418e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe67659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40de80 0x559a1b416430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54744/0 (socket says 172.21.15.66:54744) 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe67659c640 1 -- 172.21.15.66:0/1262016257 learned_addr learned my addr 172.21.15.66:0/1262016257 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe676d9d640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe6640a3e40 0x7fe6640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe676d9d640 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b40d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe66f7fe640 1 -- 172.21.15.66:0/1272862144 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe6680f2e90 con 0x7fe678106150 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe675d9b640 1 -- 172.21.15.66:0/1262016257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 msgr2=0x559a1b40d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b40d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe675d9b640 1 -- 172.21.15.66:0/1262016257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40de80 msgr2=0x559a1b416430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40de80 0x559a1b416430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe675d9b640 1 -- 172.21.15.66:0/1262016257 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1b3011b0 con 0x559a1b416a00 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe676d9d640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe6640a3e40 0x7fe6640a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe670009fd0 tx=0x7fe670009a30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b416a00 0x559a1b418e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6680f01c0 tx=0x7fe668003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e81aeb3ade50a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe66cff9640 1 -- 172.21.15.66:0/1262016257 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680be480 con 0x559a1b416a00 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe66cff9640 1 -- 172.21.15.66:0/1262016257 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe668083040 con 0x559a1b416a00 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.140+0000 7fe66cff9640 1 -- 172.21.15.66:0/1262016257 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680bc170 con 0x559a1b416a00 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1262016257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b416a00 msgr2=0x559a1b418e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b416a00 0x559a1b418e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6680f01c0 tx=0x7fe668003680 comp rx=0 tx=0).stop 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1262016257 shutdown_connections 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b416a00 0x559a1b418e50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40de80 0x559a1b416430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1262016257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b40d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1262016257 >> 172.21.15.66:0/1262016257 conn(0x559a1b407ba0 msgr2=0x559a1b407f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1262016257 shutdown_connections 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1262016257 wait complete. 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.141+0000 7fe67f3fd740 1 -- start start 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 1 Processor -- start 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40d4d0 0x559a1b416310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40de80 0x559a1b413710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b413d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559a1b30f4b0 con 0x559a1b416a00 2024-04-21T09:09:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a1b32b500 con 0x559a1b40d4d0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559a1b419730 con 0x559a1b40de80 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67f3fd740 10 client.0 _ll_register_callbacks cb 0x559a1b2f9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40d4d0 0x559a1b416310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40d4d0 0x559a1b416310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:43272/0 (socket says 172.21.15.66:43272) 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b413d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 -- 172.21.15.66:0/1471900239 learned_addr learned my addr 172.21.15.66:0/1471900239 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40de80 0x559a1b413710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 -- 172.21.15.66:0/1471900239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40de80 msgr2=0x559a1b413710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40de80 0x559a1b413710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 -- 172.21.15.66:0/1471900239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 msgr2=0x559a1b413d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b413d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 -- 172.21.15.66:0/1471900239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1b0843f0 con 0x559a1b40d4d0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe67659c640 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40d4d0 0x559a1b416310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe66000f4f0 tx=0x7fe66000f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d735970fa1ce0638 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.142+0000 7fe65dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:09:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe65dffb640 1 -- 172.21.15.66:0/1471900239 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660013850 con 0x559a1b40d4d0 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe65dffb640 1 -- 172.21.15.66:0/1471900239 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe6600139f0 con 0x559a1b40d4d0 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe668083040 con 0x559a1b40d4d0 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe65dffb640 1 -- 172.21.15.66:0/1471900239 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe660013bf0 con 0x559a1b40d4d0 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 10 client.25712 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a1b14e7d0 con 0x559a1b40d4d0 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe65dffb640 1 -- 172.21.15.66:0/1471900239 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe668083040 con 0x559a1b40d4d0 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe65dffb640 1 -- 172.21.15.66:0/1471900239 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe6600113e0 con 0x559a1b40d4d0 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 10 client.25712 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 10 client.25712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 20 client.25712 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b428dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 4 client.25712 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a1b311930 con 0x559a1b426960 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.143+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b428dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.144+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b428dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a5fa9f5b3a176d2 server_cookie=4d1d16872cf0edb8 in_seq=0 out_seq=0 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.144+0000 7fe65dffb640 10 client.25712 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.144+0000 7fe65dffb640 1 -- 172.21.15.66:0/1471900239 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a1b311930 con 0x559a1b426960 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.144+0000 7fe65dffb640 10 client.25712 handle_command_reply: tid=0 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.144+0000 7fe67f3fd740 1 client.25712 shutdown 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40d4d0 msgr2=0x559a1b416310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40d4d0 0x559a1b416310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe66000f4f0 tx=0x7fe66000f9b0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 shutdown_connections 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b428dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b413d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe65dffb640 0 client.25712 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40de80 0x559a1b413710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1471900239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40d4d0 0x559a1b416310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 >> 172.21.15.66:0/1471900239 conn(0x559a1b407ba0 msgr2=0x559a1b4098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 shutdown_connections 2024-04-21T09:09:21.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1471900239 wait complete. 2024-04-21T09:09:21.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.145+0000 7fe67f3fd740 20 client.25712 trim_cache size 0 max 16384 2024-04-21T09:09:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.158+0000 7fe67f3fd740 1 Processor -- start 2024-04-21T09:09:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.158+0000 7fe67f3fd740 1 -- start start 2024-04-21T09:09:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b422470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b415e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b42e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781b1c20 con 0x559a1b416a00 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781af4f0 con 0x559a1b40de80 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ad320 con 0x559a1b40d4d0 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b42e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe675d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b415e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b42e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54762/0 (socket says 172.21.15.66:54762) 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe676d9d640 1 -- 172.21.15.66:0/1707055112 learned_addr learned my addr 172.21.15.66:0/1707055112 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67659c640 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b422470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe675d9b640 1 -- 172.21.15.66:0/1707055112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 msgr2=0x559a1b422470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b422470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe675d9b640 1 -- 172.21.15.66:0/1707055112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 msgr2=0x559a1b42e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b42e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe675d9b640 1 -- 172.21.15.66:0/1707055112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1b411ca0 con 0x559a1b40de80 2024-04-21T09:09:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe67659c640 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b422470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe676d9d640 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b42e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.159+0000 7fe675d9b640 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b415e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe668005740 tx=0x7fe668040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbb3dbfb4ba2c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe65ffff640 1 -- 172.21.15.66:0/1707055112 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680bc390 con 0x559a1b40de80 2024-04-21T09:09:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe65ffff640 1 -- 172.21.15.66:0/1707055112 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6680bf040 con 0x559a1b40de80 2024-04-21T09:09:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe65ffff640 1 -- 172.21.15.66:0/1707055112 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680bc820 con 0x559a1b40de80 2024-04-21T09:09:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1707055112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 msgr2=0x559a1b415e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b415e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe668005740 tx=0x7fe668040000 comp rx=0 tx=0).stop 2024-04-21T09:09:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1707055112 shutdown_connections 2024-04-21T09:09:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b416a00 0x559a1b42e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b415e40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/1707055112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b40d4d0 0x559a1b422470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1707055112 >> 172.21.15.66:0/1707055112 conn(0x559a1b407ba0 msgr2=0x559a1b407f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1707055112 shutdown_connections 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.160+0000 7fe67f3fd740 1 -- 172.21.15.66:0/1707055112 wait complete. 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 -- start start 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 Processor -- start 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40d4d0 0x559a1b41eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b4101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b416a00 0x559a1b4106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ad9b0 con 0x559a1b40d4d0 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781aee20 con 0x559a1b40de80 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ae080 con 0x559a1b416a00 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67f3fd740 10 client.0 _ll_register_callbacks cb 0x7fe66000ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40d4d0 0x559a1b41eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40d4d0 0x559a1b41eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54764/0 (socket says 172.21.15.66:54764) 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe675d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b4101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67659c640 1 -- 172.21.15.66:0/2950227215 learned_addr learned my addr 172.21.15.66:0/2950227215 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe675d9b640 1 -- 172.21.15.66:0/2950227215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b416a00 msgr2=0x559a1b4106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe675d9b640 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b416a00 0x559a1b4106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe675d9b640 1 -- 172.21.15.66:0/2950227215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40d4d0 msgr2=0x559a1b41eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe675d9b640 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40d4d0 0x559a1b41eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe675d9b640 1 -- 172.21.15.66:0/2950227215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1b410ef0 con 0x559a1b40de80 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.161+0000 7fe67659c640 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40d4d0 0x559a1b41eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe675d9b640 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b4101b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe668005d00 tx=0x7fe6680df0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b73e3dfdd1c2e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe65dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe65dffb640 1 -- 172.21.15.66:0/2950227215 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668088030 con 0x559a1b40de80 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe65dffb640 1 -- 172.21.15.66:0/2950227215 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe6680bc350 con 0x559a1b40de80 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6680bf040 con 0x559a1b40de80 2024-04-21T09:09:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe65dffb640 1 -- 172.21.15.66:0/2950227215 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6680bca80 con 0x559a1b40de80 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 10 client.25718 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a1b410ef0 con 0x559a1b40de80 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe65dffb640 1 -- 172.21.15.66:0/2950227215 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6680bf040 con 0x559a1b40de80 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe65dffb640 1 -- 172.21.15.66:0/2950227215 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe6680bafb0 con 0x559a1b40de80 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 10 client.25718 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 10 client.25718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 20 client.25718 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b48f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 4 client.25718 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.162+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a1b37fc10 con 0x559a1b426960 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67659c640 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b48f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe65dffb640 10 client.25718 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67659c640 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b48f080 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc1fbc639e9775a9 server_cookie=9eff9ebb44204356 in_seq=0 out_seq=0 2024-04-21T09:09:21.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe65dffb640 1 -- 172.21.15.66:0/2950227215 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a1b37fc10 con 0x559a1b426960 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe65dffb640 10 client.25718 handle_command_reply: tid=0 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 client.25718 shutdown 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 msgr2=0x559a1b4101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b4101b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe668005d00 tx=0x7fe6680df0e0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 shutdown_connections 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x559a1b426960 0x559a1b48f080 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a1b416a00 0x559a1b4106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a1b40de80 0x559a1b4101b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 --2- 172.21.15.66:0/2950227215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a1b40d4d0 0x559a1b41eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe65dffb640 0 client.25718 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 >> 172.21.15.66:0/2950227215 conn(0x559a1b407ba0 msgr2=0x559a1b406370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 shutdown_connections 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 1 -- 172.21.15.66:0/2950227215 wait complete. 2024-04-21T09:09:21.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.163+0000 7fe67f3fd740 20 client.25718 trim_cache size 0 max 16384 2024-04-21T09:09:21.170 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:09:21.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe6640a3e40 msgr2=0x7fe6640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe6640a3e40 0x7fe6640a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe670009fd0 tx=0x7fe670009a30 comp rx=0 tx=0).stop 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678106150 msgr2=0x7fe67819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678106150 0x7fe67819e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe668036040 tx=0x7fe6680045e0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 shutdown_connections 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe6640a3e40 0x7fe6640a6300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe678198610 0x7fe678198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678106150 0x7fe67819e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 --2- 172.21.15.66:0/1272862144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe678101080 0x7fe67819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 >> 172.21.15.66:0/1272862144 conn(0x7fe678077ef0 msgr2=0x7fe6780ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 shutdown_connections 2024-04-21T09:09:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.164+0000 7fe67cb75640 1 -- 172.21.15.66:0/1272862144 wait complete. 2024-04-21T09:09:21.175 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:09:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 -- 172.21.15.66:0/199811871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644107260 msgr2=0x7fb64410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 --2- 172.21.15.66:0/199811871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644107260 0x7fb64410ee30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009ca0 tx=0x7fb63c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 -- 172.21.15.66:0/199811871 shutdown_connections 2024-04-21T09:09:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 --2- 172.21.15.66:0/199811871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644107260 0x7fb64410ee30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 --2- 172.21.15.66:0/199811871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb644102b40 0x7fb644106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 --2- 172.21.15.66:0/199811871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb644102190 0x7fb644102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 -- 172.21.15.66:0/199811871 >> 172.21.15.66:0/199811871 conn(0x7fb6440793a0 msgr2=0x7fb6440797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 -- 172.21.15.66:0/199811871 shutdown_connections 2024-04-21T09:09:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.284+0000 7fb64a975640 1 -- 172.21.15.66:0/199811871 wait complete. 2024-04-21T09:09:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 Processor -- start 2024-04-21T09:09:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 -- start start 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644102190 0x7fb644198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb644107260 0x7fb644198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb64419d010 0x7fb64419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441121b0 con 0x7fb644102190 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb644112030 con 0x7fb644107260 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb64a975640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb644112330 con 0x7fb64419d010 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.285+0000 7fb643fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644102190 0x7fb644198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644102190 0x7fb644198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54778/0 (socket says 172.21.15.66:54778) 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 -- 172.21.15.66:0/2724732911 learned_addr learned my addr 172.21.15.66:0/2724732911 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb648f12640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb64419d010 0x7fb64419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb6437fe640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb644107260 0x7fb644198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 -- 172.21.15.66:0/2724732911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb64419d010 msgr2=0x7fb64419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb64419d010 0x7fb64419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 -- 172.21.15.66:0/2724732911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb644107260 msgr2=0x7fb644198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb644107260 0x7fb644198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 -- 172.21.15.66:0/2724732911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64419db00 con 0x7fb644102190 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb643fff640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644102190 0x7fb644198750 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9f0 tx=0x7fb63800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb6417fa640 1 -- 172.21.15.66:0/2724732911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63800cd80 con 0x7fb644102190 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64419ddf0 con 0x7fb644102190 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb6417fa640 1 -- 172.21.15.66:0/2724732911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6380108e0 con 0x7fb644102190 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb6417fa640 1 -- 172.21.15.66:0/2724732911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638018980 con 0x7fb644102190 2024-04-21T09:09:21.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.286+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6441a55e0 con 0x7fb644102190 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb6417fa640 1 -- 172.21.15.66:0/2724732911 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7fb638018b20 con 0x7fb644102190 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 Processor -- start 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 -- start start 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a6798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a682430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a684e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c83a5de1f0 con 0x55c83a682a00 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c83a57b4b0 con 0x55c83a6794d0 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.289+0000 7fb64d567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c83a5da1b0 con 0x55c83a679e80 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a684e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6417fa640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb62c0a3e40 0x7fb62c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a684e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54786/0 (socket says 172.21.15.66:54786) 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 -- 172.21.15.66:0/970380119 learned_addr learned my addr 172.21.15.66:0/970380119 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb643fff640 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a682430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb648f12640 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a6798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb648f12640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb62c0a3e40 0x7fb62c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 -- 172.21.15.66:0/970380119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 msgr2=0x55c83a682430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a682430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 -- 172.21.15.66:0/970380119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 msgr2=0x55c83a6798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a6798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 -- 172.21.15.66:0/970380119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83a56d1b0 con 0x55c83a682a00 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6417fa640 1 -- 172.21.15.66:0/2724732911 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb6380d1170 con 0x7fb644102190 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb643fff640 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a682430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb6437fe640 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a684e50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb63400bbd0 tx=0x7fb6340107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667d9ef7e8ad5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb648f12640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb62c0a3e40 0x7fb62c0a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb63c004af0 tx=0x7fb63c005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.290+0000 7fb62a7fc640 1 -- 172.21.15.66:0/970380119 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63401c480 con 0x55c83a682a00 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb62a7fc640 1 -- 172.21.15.66:0/970380119 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6340118f0 con 0x55c83a682a00 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb62a7fc640 1 -- 172.21.15.66:0/970380119 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63401a820 con 0x55c83a682a00 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 -- 172.21.15.66:0/970380119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 msgr2=0x55c83a684e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a684e50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb63400bbd0 tx=0x7fb6340107a0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 -- 172.21.15.66:0/970380119 shutdown_connections 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a684e50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a682430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 --2- 172.21.15.66:0/970380119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a6798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 -- 172.21.15.66:0/970380119 >> 172.21.15.66:0/970380119 conn(0x55c83a673ba0 msgr2=0x55c83a673f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 -- 172.21.15.66:0/970380119 shutdown_connections 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.291+0000 7fb64d567740 1 -- 172.21.15.66:0/970380119 wait complete. 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 -- start start 2024-04-21T09:09:21.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 Processor -- start 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a682320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a67f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c83a685970 con 0x55c83a682a00 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c83a597500 con 0x55c83a6794d0 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c83a685af0 con 0x55c83a679e80 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb64d567740 10 client.0 _ll_register_callbacks cb 0x55c83a565b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb648f12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb643fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a682320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb648f12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54800/0 (socket says 172.21.15.66:54800) 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.292+0000 7fb648f12640 1 -- 172.21.15.66:0/1959785994 learned_addr learned my addr 172.21.15.66:0/1959785994 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb6437fe640 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a67f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb643fff640 1 -- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 msgr2=0x55c83a67f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb643fff640 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a67f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb643fff640 1 -- 172.21.15.66:0/1959785994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 msgr2=0x55c83a67fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb643fff640 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb643fff640 1 -- 172.21.15.66:0/1959785994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83a2f03f0 con 0x55c83a6794d0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb648f12640 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb643fff640 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a682320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6380ce3d0 tx=0x7fb63809cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c631a236b2e5a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb6237fe640 1 -- 172.21.15.66:0/1959785994 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638099270 con 0x55c83a6794d0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb6237fe640 1 -- 172.21.15.66:0/1959785994 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb638010430 con 0x55c83a6794d0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6340118f0 con 0x55c83a6794d0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb6237fe640 1 -- 172.21.15.66:0/1959785994 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb638099410 con 0x55c83a6794d0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb64d567740 10 client.25730 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.293+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c83a3ba7d0 con 0x55c83a6794d0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb6237fe640 1 -- 172.21.15.66:0/1959785994 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6340118f0 con 0x55c83a6794d0 2024-04-21T09:09:21.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb6237fe640 1 -- 172.21.15.66:0/1959785994 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb638098950 con 0x55c83a6794d0 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb64d567740 10 client.25730 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb64d567740 10 client.25730 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb64d567740 20 client.25730 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb64d567740 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a695020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb64d567740 4 client.25730 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83a57d930 con 0x55c83a692bb0 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.294+0000 7fb6437fe640 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a695020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.295+0000 7fb6237fe640 10 client.25730 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.295+0000 7fb6437fe640 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a695020 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7e0e4923bfbd3a7 server_cookie=7e71b55d3caf0f79 in_seq=0 out_seq=0 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.295+0000 7fb6237fe640 1 -- 172.21.15.66:0/1959785994 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c83a57d930 con 0x55c83a692bb0 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.295+0000 7fb6237fe640 10 client.25730 handle_command_reply: tid=0 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.295+0000 7fb64d567740 1 client.25730 shutdown 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 msgr2=0x55c83a682320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a682320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6380ce3d0 tx=0x7fb63809cc00 comp rx=0 tx=0).stop 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 shutdown_connections 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a695020 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a679e80 0x55c83a67f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb6237fe640 0 client.25730 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 --2- 172.21.15.66:0/1959785994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a6794d0 0x55c83a682320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 >> 172.21.15.66:0/1959785994 conn(0x55c83a673ba0 msgr2=0x55c83a6758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 shutdown_connections 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 1 -- 172.21.15.66:0/1959785994 wait complete. 2024-04-21T09:09:21.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.296+0000 7fb64d567740 20 client.25730 trim_cache size 0 max 16384 2024-04-21T09:09:21.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.309+0000 7fb64d567740 1 Processor -- start 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.309+0000 7fb64d567740 1 -- start start 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb64d567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb64d567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a681de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb64d567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a682320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb64d567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441b1c70 con 0x55c83a682a00 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb64d567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441af540 con 0x55c83a679e80 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb64d567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441ad370 con 0x55c83a6794d0 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a682320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a682320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54814/0 (socket says 172.21.15.66:54814) 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 -- 172.21.15.66:0/2352790864 learned_addr learned my addr 172.21.15.66:0/2352790864 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb6437fe640 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a681de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb643fff640 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 -- 172.21.15.66:0/2352790864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 msgr2=0x55c83a68e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 -- 172.21.15.66:0/2352790864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 msgr2=0x55c83a681de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a681de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 -- 172.21.15.66:0/2352790864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83a679c20 con 0x55c83a682a00 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb643fff640 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.310+0000 7fb648f12640 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a682320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb63c037040 tx=0x7fb63c03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388a451f9957d6e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.311+0000 7fb629ffb640 1 -- 172.21.15.66:0/2352790864 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c054070 con 0x55c83a682a00 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.311+0000 7fb629ffb640 1 -- 172.21.15.66:0/2352790864 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c003cf0 con 0x55c83a682a00 2024-04-21T09:09:21.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.311+0000 7fb629ffb640 1 -- 172.21.15.66:0/2352790864 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c039040 con 0x55c83a682a00 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 -- 172.21.15.66:0/2352790864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 msgr2=0x55c83a682320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a682320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb63c037040 tx=0x7fb63c03a000 comp rx=0 tx=0).stop 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 -- 172.21.15.66:0/2352790864 shutdown_connections 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a682320 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a681de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 --2- 172.21.15.66:0/2352790864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 -- 172.21.15.66:0/2352790864 >> 172.21.15.66:0/2352790864 conn(0x55c83a673ba0 msgr2=0x55c83a67f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 -- 172.21.15.66:0/2352790864 shutdown_connections 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 -- 172.21.15.66:0/2352790864 wait complete. 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.312+0000 7fb64d567740 1 -- start start 2024-04-21T09:09:21.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 1 Processor -- start 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a67bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441ada00 con 0x55c83a682a00 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441aee70 con 0x55c83a679e80 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441ae0d0 con 0x55c83a6794d0 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb64d567740 10 client.0 _ll_register_callbacks cb 0x7fb63809c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb648f12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb6437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a67bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb648f12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54828/0 (socket says 172.21.15.66:54828) 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb648f12640 1 -- 172.21.15.66:0/728082773 learned_addr learned my addr 172.21.15.66:0/728082773 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb643fff640 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb6437fe640 1 -- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 msgr2=0x55c83a68af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb6437fe640 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb6437fe640 1 -- 172.21.15.66:0/728082773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 msgr2=0x55c83a67c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb6437fe640 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb6437fe640 1 -- 172.21.15.66:0/728082773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83a67c960 con 0x55c83a679e80 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.313+0000 7fb648f12640 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb6437fe640 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a67bee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb634010ca0 tx=0x7fb634010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b097674b05a1ff02 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb6237fe640 1 -- 172.21.15.66:0/728082773 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63402e020 con 0x55c83a679e80 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb6237fe640 1 -- 172.21.15.66:0/728082773 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb634005460 con 0x55c83a679e80 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63c003cf0 con 0x55c83a679e80 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb6237fe640 1 -- 172.21.15.66:0/728082773 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb634036050 con 0x55c83a679e80 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 10 client.25736 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c83a6f8e40 con 0x55c83a679e80 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb6237fe640 1 -- 172.21.15.66:0/728082773 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c003cf0 con 0x55c83a679e80 2024-04-21T09:09:21.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb6237fe640 1 -- 172.21.15.66:0/728082773 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb634004ab0 con 0x55c83a679e80 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 10 client.25736 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 10 client.25736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 20 client.25736 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a6fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 4 client.25736 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.314+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c83a5ebc10 con 0x55c83a692bb0 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.315+0000 7fb643fff640 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a6fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.315+0000 7fb643fff640 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a6fb3c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbcd85d0f4e56f3e server_cookie=39233fe88de5309d in_seq=0 out_seq=0 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.315+0000 7fb6237fe640 10 client.25736 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.315+0000 7fb6237fe640 1 -- 172.21.15.66:0/728082773 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c83a5ebc10 con 0x55c83a692bb0 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb6237fe640 10 client.25736 handle_command_reply: tid=0 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 client.25736 shutdown 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 msgr2=0x55c83a67bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a67bee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb634010ca0 tx=0x7fb634010fd0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 shutdown_connections 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c83a692bb0 0x55c83a6fb3c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c83a682a00 0x55c83a67c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83a679e80 0x55c83a67bee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 --2- 172.21.15.66:0/728082773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c83a6794d0 0x55c83a68af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb6237fe640 0 client.25736 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 >> 172.21.15.66:0/728082773 conn(0x55c83a673ba0 msgr2=0x55c83a672370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 shutdown_connections 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 1 -- 172.21.15.66:0/728082773 wait complete. 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64d567740 20 client.25736 trim_cache size 0 max 16384 2024-04-21T09:09:21.321 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:09:21.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb62c0a3e40 msgr2=0x7fb62c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64a975640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb62c0a3e40 0x7fb62c0a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb63c004af0 tx=0x7fb63c005c50 comp rx=0 tx=0).stop 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644102190 msgr2=0x7fb644198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.316+0000 7fb64a975640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644102190 0x7fb644198750 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9f0 tx=0x7fb63800eeb0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 shutdown_connections 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb62c0a3e40 0x7fb62c0a6300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb64419d010 0x7fb64419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb644107260 0x7fb644198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 --2- 172.21.15.66:0/2724732911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb644102190 0x7fb644198750 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 >> 172.21.15.66:0/2724732911 conn(0x7fb6440793a0 msgr2=0x7fb6440fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 shutdown_connections 2024-04-21T09:09:21.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.317+0000 7fb64a975640 1 -- 172.21.15.66:0/2724732911 wait complete. 2024-04-21T09:09:21.328 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:09:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 -- 172.21.15.66:0/3165175768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006ac40 msgr2=0x7f555010ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 --2- 172.21.15.66:0/3165175768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006ac40 0x7f555010ed80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5548009ca0 tx=0x7f554802c140 comp rx=0 tx=0).stop 2024-04-21T09:09:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 -- 172.21.15.66:0/3165175768 shutdown_connections 2024-04-21T09:09:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 --2- 172.21.15.66:0/3165175768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006ac40 0x7f555010ed80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 --2- 172.21.15.66:0/3165175768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555006a2a0 0x7f555006a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 --2- 172.21.15.66:0/3165175768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550106f10 0x7f55501072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 -- 172.21.15.66:0/3165175768 >> 172.21.15.66:0/3165175768 conn(0x7f55500769d0 msgr2=0x7f5550076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.437+0000 7f5554def640 1 -- 172.21.15.66:0/3165175768 shutdown_connections 2024-04-21T09:09:21.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.438+0000 7f5554def640 1 -- 172.21.15.66:0/3165175768 wait complete. 2024-04-21T09:09:21.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.438+0000 7f5554def640 1 Processor -- start 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.438+0000 7f5554def640 1 -- start start 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.438+0000 7f5554def640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006a2a0 0x7f55501a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.438+0000 7f5554def640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555006ac40 0x7f55501a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.438+0000 7f5554def640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550106f10 0x7f555019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f5554def640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5550112280 con 0x7f555006a2a0 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f5554def640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5550112100 con 0x7f555006ac40 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f5554def640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5550112400 con 0x7f5550106f10 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006a2a0 0x7f55501a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006a2a0 0x7f55501a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54842/0 (socket says 172.21.15.66:54842) 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 -- 172.21.15.66:0/918046083 learned_addr learned my addr 172.21.15.66:0/918046083 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554dd9b640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555006ac40 0x7f55501a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554ed9d640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550106f10 0x7f555019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 -- 172.21.15.66:0/918046083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550106f10 msgr2=0x7f555019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550106f10 0x7f555019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 -- 172.21.15.66:0/918046083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555006ac40 msgr2=0x7f55501a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555006ac40 0x7f55501a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 -- 172.21.15.66:0/918046083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555019b080 con 0x7f555006a2a0 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f554e59c640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006a2a0 0x7f55501a0670 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f554000e9f0 tx=0x7f554000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f553f7fe640 1 -- 172.21.15.66:0/918046083 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000cd80 con 0x7f555006a2a0 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555019b370 con 0x7f555006a2a0 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f553f7fe640 1 -- 172.21.15.66:0/918046083 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55400054d0 con 0x7f555006a2a0 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f553f7fe640 1 -- 172.21.15.66:0/918046083 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540010690 con 0x7f555006a2a0 2024-04-21T09:09:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.439+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55501a7660 con 0x7f555006a2a0 2024-04-21T09:09:21.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.442+0000 7f553f7fe640 1 -- 172.21.15.66:0/918046083 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7f5540004e90 con 0x7f555006a2a0 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.442+0000 7f55579d6740 1 Processor -- start 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.442+0000 7f55579d6740 1 -- start start 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.442+0000 7f553f7fe640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55380a3e40 0x7f55380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f55579d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff64d0 0x55a777ff68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f55579d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff6e80 0x55a777fff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f55579d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a778001e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f55579d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a777f5b1f0 con 0x55a777fffa00 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f55579d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a777ef84b0 con 0x55a777ff64d0 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f55579d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a777f571b0 con 0x55a777ff6e80 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff64d0 0x55a777ff68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554ed9d640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55380a3e40 0x7f55380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff64d0 0x55a777ff68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:43360/0 (socket says 172.21.15.66:43360) 2024-04-21T09:09:21.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 -- 172.21.15.66:0/2455851621 learned_addr learned my addr 172.21.15.66:0/2455851621 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554dd9b640 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff6e80 0x55a777fff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f553f7fe640 1 -- 172.21.15.66:0/918046083 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f55400d11c0 con 0x7f555006a2a0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554ed9d640 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a778001e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 -- 172.21.15.66:0/2455851621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff6e80 msgr2=0x55a777fff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff6e80 0x55a777fff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 -- 172.21.15.66:0/2455851621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 msgr2=0x55a778001e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a778001e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 -- 172.21.15.66:0/2455851621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a777eea1b0 con 0x55a777ff64d0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554ed9d640 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a778001e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554e59c640 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff64d0 0x55a777ff68b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5540004a40 tx=0x7f5540098e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d21f7c94dd9dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f554ed9d640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55380a3e40 0x7f55380a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5548004af0 tx=0x7f554802e040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.443+0000 7f553cff9640 1 -- 172.21.15.66:0/2455851621 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554009c500 con 0x55a777ff64d0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f553cff9640 1 -- 172.21.15.66:0/2455851621 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f554009d2a0 con 0x55a777ff64d0 2024-04-21T09:09:21.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f553cff9640 1 -- 172.21.15.66:0/2455851621 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540005180 con 0x55a777ff64d0 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 -- 172.21.15.66:0/2455851621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff64d0 msgr2=0x55a777ff68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff64d0 0x55a777ff68b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5540004a40 tx=0x7f5540098e40 comp rx=0 tx=0).stop 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 -- 172.21.15.66:0/2455851621 shutdown_connections 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a778001e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff6e80 0x55a777fff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 --2- 172.21.15.66:0/2455851621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff64d0 0x55a777ff68b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 -- 172.21.15.66:0/2455851621 >> 172.21.15.66:0/2455851621 conn(0x55a777ff0ba0 msgr2=0x55a777ff0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 -- 172.21.15.66:0/2455851621 shutdown_connections 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.444+0000 7f55579d6740 1 -- 172.21.15.66:0/2455851621 wait complete. 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 -- start start 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 Processor -- start 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a777fff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777ffcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a778002970 con 0x55a777fffa00 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a777f14500 con 0x55a777ff6e80 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a778002af0 con 0x55a777ff64d0 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f55579d6740 10 client.0 _ll_register_callbacks cb 0x55a777ee2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777ffcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:43370/0 (socket says 172.21.15.66:43370) 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 -- 172.21.15.66:0/79974626 learned_addr learned my addr 172.21.15.66:0/79974626 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554e59c640 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a777fff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 -- 172.21.15.66:0/79974626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 msgr2=0x55a777fff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a777fff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 -- 172.21.15.66:0/79974626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 msgr2=0x55a777ffcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777ffcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.445+0000 7f554dd9b640 1 -- 172.21.15.66:0/79974626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a777c6d3f0 con 0x55a777ff6e80 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f554dd9b640 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffc6b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5544005d00 tx=0x7f55440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f66cda1cef17b24a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f5535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f5535ffb640 1 -- 172.21.15.66:0/79974626 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544004020 con 0x55a777ff6e80 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f5535ffb640 1 -- 172.21.15.66:0/79974626 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f55440041c0 con 0x55a777ff6e80 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554009d2a0 con 0x55a777ff6e80 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f5535ffb640 1 -- 172.21.15.66:0/79974626 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5544024050 con 0x55a777ff6e80 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f55579d6740 10 client.25748 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a777d377d0 con 0x55a777ff6e80 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f5535ffb640 1 -- 172.21.15.66:0/79974626 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f554009d2a0 con 0x55a777ff6e80 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f5535ffb640 1 -- 172.21.15.66:0/79974626 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5544004b80 con 0x55a777ff6e80 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f55579d6740 10 client.25748 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f55579d6740 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f55579d6740 20 client.25748 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.446+0000 7f55579d6740 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a77800fd30 0x55a778012150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.447+0000 7f55579d6740 4 client.25748 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.447+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a777efa930 con 0x55a77800fd30 2024-04-21T09:09:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.447+0000 7f554e59c640 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a77800fd30 0x55a778012150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.447+0000 7f5535ffb640 10 client.25748 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.447+0000 7f554e59c640 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a77800fd30 0x55a778012150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5dc7273404ca590 server_cookie=6b6a1bde58f0a03e in_seq=0 out_seq=0 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f5535ffb640 1 -- 172.21.15.66:0/79974626 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a777efa930 con 0x55a77800fd30 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f5535ffb640 10 client.25748 handle_command_reply: tid=0 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 client.25748 shutdown 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 msgr2=0x55a777ffc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffc6b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5544005d00 tx=0x7f55440118d0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 shutdown_connections 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a77800fd30 0x55a778012150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777ffcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f5535ffb640 0 client.25748 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffc6b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 --2- 172.21.15.66:0/79974626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a777fff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 >> 172.21.15.66:0/79974626 conn(0x55a777ff0ba0 msgr2=0x55a777ff28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 shutdown_connections 2024-04-21T09:09:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 1 -- 172.21.15.66:0/79974626 wait complete. 2024-04-21T09:09:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.448+0000 7f55579d6740 20 client.25748 trim_cache size 0 max 16384 2024-04-21T09:09:21.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 Processor -- start 2024-04-21T09:09:21.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 -- start start 2024-04-21T09:09:21.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a77800b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777fff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55501b3cd0 con 0x55a777fffa00 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55501b15a0 con 0x55a777ff6e80 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f55579d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f55501af3d0 con 0x55a777ff64d0 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f554ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777fff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f554ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777fff320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54864/0 (socket says 172.21.15.66:54864) 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f554ed9d640 1 -- 172.21.15.66:0/575806243 learned_addr learned my addr 172.21.15.66:0/575806243 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f554dd9b640 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.462+0000 7f554e59c640 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a77800b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f554ed9d640 1 -- 172.21.15.66:0/575806243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 msgr2=0x55a77800b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f554ed9d640 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a77800b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f554ed9d640 1 -- 172.21.15.66:0/575806243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 msgr2=0x55a777ffede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f554ed9d640 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f554ed9d640 1 -- 172.21.15.66:0/575806243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a777ff14e0 con 0x55a777fffa00 2024-04-21T09:09:21.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f554ed9d640 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777fff320 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5548009ca0 tx=0x7f5548005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0783ab968d05644 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f5537fff640 1 -- 172.21.15.66:0/575806243 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548052070 con 0x55a777fffa00 2024-04-21T09:09:21.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f5537fff640 1 -- 172.21.15.66:0/575806243 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5548003cf0 con 0x55a777fffa00 2024-04-21T09:09:21.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.463+0000 7f5537fff640 1 -- 172.21.15.66:0/575806243 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548039040 con 0x55a777fffa00 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 -- 172.21.15.66:0/575806243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 msgr2=0x55a777fff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777fff320 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5548009ca0 tx=0x7f5548005c50 comp rx=0 tx=0).stop 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 -- 172.21.15.66:0/575806243 shutdown_connections 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777fffa00 0x55a777fff320 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a777ffede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 --2- 172.21.15.66:0/575806243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777ff64d0 0x55a77800b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 -- 172.21.15.66:0/575806243 >> 172.21.15.66:0/575806243 conn(0x55a777ff0ba0 msgr2=0x55a777ff7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 -- 172.21.15.66:0/575806243 shutdown_connections 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 -- 172.21.15.66:0/575806243 wait complete. 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 -- start start 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 Processor -- start 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.464+0000 7f55579d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777ff64d0 0x55a778007d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f55579d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a7780082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f55579d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777fffa00 0x55a778004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f55579d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55501afa60 con 0x55a777ff64d0 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f55579d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55501b0ed0 con 0x55a777ff6e80 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f55579d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f55501b0130 con 0x55a777fffa00 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f55579d6740 10 client.0 _ll_register_callbacks cb 0x7f554400ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777ff64d0 0x55a778007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777ff64d0 0x55a778007d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54870/0 (socket says 172.21.15.66:54870) 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 -- 172.21.15.66:0/2020905399 learned_addr learned my addr 172.21.15.66:0/2020905399 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554dd9b640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a7780082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554ed9d640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777fffa00 0x55a778004d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 -- 172.21.15.66:0/2020905399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777fffa00 msgr2=0x55a778004d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777fffa00 0x55a778004d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 -- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 msgr2=0x55a7780082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a7780082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 -- 172.21.15.66:0/2020905399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7780053e0 con 0x55a777ff64d0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554dd9b640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a7780082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f554e59c640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777ff64d0 0x55a778007d70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5540098fb0 tx=0x7f55400bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35608c75d5f6550 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f5535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f5535ffb640 1 -- 172.21.15.66:0/2020905399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554009c500 con 0x55a777ff64d0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f5535ffb640 1 -- 172.21.15.66:0/2020905399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f5540099210 con 0x55a777ff64d0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.465+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5548003cf0 con 0x55a777ff64d0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f5535ffb640 1 -- 172.21.15.66:0/2020905399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f554009d2a0 con 0x55a777ff64d0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 10 client.16152 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a778076c00 con 0x55a777ff64d0 2024-04-21T09:09:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f5535ffb640 1 -- 172.21.15.66:0/2020905399 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5548003cf0 con 0x55a777ff64d0 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f5535ffb640 1 -- 172.21.15.66:0/2020905399 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f554009ce00 con 0x55a777ff64d0 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 10 client.16152 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 20 client.16152 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a778076ee0 0x55a7780792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 4 client.16152 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a777f68c10 con 0x55a778076ee0 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.466+0000 7f554dd9b640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a778076ee0 0x55a7780792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f554dd9b640 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a778076ee0 0x55a7780792d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ae3cb4617146fd server_cookie=76ad855f996a2e15 in_seq=0 out_seq=0 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f5535ffb640 10 client.16152 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f5535ffb640 1 -- 172.21.15.66:0/2020905399 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a777f68c10 con 0x55a778076ee0 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f5535ffb640 10 client.16152 handle_command_reply: tid=0 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 client.16152 shutdown 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777ff64d0 msgr2=0x55a778007d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777ff64d0 0x55a778007d70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5540098fb0 tx=0x7f55400bd040 comp rx=0 tx=0).stop 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 shutdown_connections 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a778076ee0 0x55a7780792d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a777fffa00 0x55a778004d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f5535ffb640 0 client.16152 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a777ff6e80 0x55a7780082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 --2- 172.21.15.66:0/2020905399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a777ff64d0 0x55a778007d70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 >> 172.21.15.66:0/2020905399 conn(0x55a777ff0ba0 msgr2=0x55a777fef370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 shutdown_connections 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 1 -- 172.21.15.66:0/2020905399 wait complete. 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.467+0000 7f55579d6740 20 client.16152 trim_cache size 0 max 16384 2024-04-21T09:09:21.472 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:09:21.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55380a3e40 msgr2=0x7f55380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55380a3e40 0x7f55380a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5548004af0 tx=0x7f554802e040 comp rx=0 tx=0).stop 2024-04-21T09:09:21.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006a2a0 msgr2=0x7f55501a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:21.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006a2a0 0x7f55501a0670 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f554000e9f0 tx=0x7f554000edf0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 shutdown_connections 2024-04-21T09:09:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55380a3e40 0x7f55380a6300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550106f10 0x7f555019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555006ac40 0x7f55501a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 --2- 172.21.15.66:0/918046083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f555006a2a0 0x7f55501a0670 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 >> 172.21.15.66:0/918046083 conn(0x7f55500769d0 msgr2=0x7f555010d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 shutdown_connections 2024-04-21T09:09:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:21.468+0000 7f5554def640 1 -- 172.21.15.66:0/918046083 wait complete. 2024-04-21T09:09:22.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:21 smithi186 ceph-mon[31505]: pgmap v37: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 863 KiB/s wr, 11 op/s 2024-04-21T09:09:22.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:21 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1545630792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:09:22.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:21 smithi083 ceph-mon[31413]: pgmap v37: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 863 KiB/s wr, 11 op/s 2024-04-21T09:09:22.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:21 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1545630792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:09:22.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:21 smithi066 ceph-mon[27953]: pgmap v37: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 863 KiB/s wr, 11 op/s 2024-04-21T09:09:22.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:21 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1545630792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:09:22.479 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:09:22.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:09:22.480 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:09:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.591+0000 7f7c64d8e640 1 -- 172.21.15.66:0/2088369157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c60107270 msgr2=0x7f7c6010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.591+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/2088369157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c60107270 0x7f7c6010ee40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009ca0 tx=0x7f7c5802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 -- 172.21.15.66:0/2088369157 shutdown_connections 2024-04-21T09:09:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/2088369157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c60107270 0x7f7c6010ee40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/2088369157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c60102b50 0x7f7c60106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/2088369157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c601021a0 0x7f7c60102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 -- 172.21.15.66:0/2088369157 >> 172.21.15.66:0/2088369157 conn(0x7f7c60079350 msgr2=0x7f7c60079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 -- 172.21.15.66:0/2088369157 shutdown_connections 2024-04-21T09:09:22.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 -- 172.21.15.66:0/2088369157 wait complete. 2024-04-21T09:09:22.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 Processor -- start 2024-04-21T09:09:22.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.592+0000 7f7c64d8e640 1 -- start start 2024-04-21T09:09:22.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c64d8e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c601021a0 0x7f7c60198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c64d8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c60107270 0x7f7c60198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c64d8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6019cfe0 0x7f7c6019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c64d8e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601121c0 con 0x7f7c6019cfe0 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c64d8e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60112040 con 0x7f7c60107270 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c64d8e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60112340 con 0x7f7c601021a0 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6019cfe0 0x7f7c6019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6019cfe0 0x7f7c6019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54880/0 (socket says 172.21.15.66:54880) 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 -- 172.21.15.66:0/3157528896 learned_addr learned my addr 172.21.15.66:0/3157528896 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c601021a0 0x7f7c60198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c60107270 0x7f7c60198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 -- 172.21.15.66:0/3157528896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c601021a0 msgr2=0x7f7c60198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c601021a0 0x7f7c60198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 -- 172.21.15.66:0/3157528896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c60107270 msgr2=0x7f7c60198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c60107270 0x7f7c60198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 -- 172.21.15.66:0/3157528896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6019dad0 con 0x7f7c6019cfe0 2024-04-21T09:09:22.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c60107270 0x7f7c60198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.593+0000 7f7c5ed9d640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6019cfe0 0x7f7c6019d490 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005e30 tx=0x7f7c58034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.594+0000 7f7c4f7fe640 1 -- 172.21.15.66:0/3157528896 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58039040 con 0x7f7c6019cfe0 2024-04-21T09:09:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.594+0000 7f7c4f7fe640 1 -- 172.21.15.66:0/3157528896 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c58044070 con 0x7f7c6019cfe0 2024-04-21T09:09:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.594+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6019dd60 con 0x7f7c6019cfe0 2024-04-21T09:09:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.594+0000 7f7c4f7fe640 1 -- 172.21.15.66:0/3157528896 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58003e60 con 0x7f7c6019cfe0 2024-04-21T09:09:22.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.594+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c601a5520 con 0x7f7c6019cfe0 2024-04-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.596+0000 7f7c67976740 1 Processor -- start 2024-04-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.597+0000 7f7c4f7fe640 1 -- 172.21.15.66:0/3157528896 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7f7c580324c0 con 0x7f7c6019cfe0 2024-04-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.597+0000 7f7c67976740 1 -- start start 2024-04-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.597+0000 7f7c67976740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c67976740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c67976740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c67976740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56084d911f00 con 0x56084d9ae010 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c67976740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56084d8af590 con 0x56084d9ad660 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c67976740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56084d90dec0 con 0x56084d9bb220 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54884/0 (socket says 172.21.15.66:54884) 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 -- 172.21.15.66:0/363929597 learned_addr learned my addr 172.21.15.66:0/363929597 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c4f7fe640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7c480a3e40 0x7f7c480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 -- 172.21.15.66:0/363929597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 msgr2=0x56084d9bd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 -- 172.21.15.66:0/363929597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 msgr2=0x56084d9ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 -- 172.21.15.66:0/363929597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084d6ec7e0 con 0x56084d9ae010 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c4f7fe640 1 -- 172.21.15.66:0/3157528896 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7c580f30a0 con 0x7f7c6019cfe0 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5ed9d640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7c480a3e40 0x7f7c480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.598+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bac50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000dcb0 tx=0x7f7c5000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a3f53ff5e86d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c4cff9640 1 -- 172.21.15.66:0/363929597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50011440 con 0x56084d9ae010 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c4cff9640 1 -- 172.21.15.66:0/363929597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c50011a90 con 0x56084d9ae010 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c5ed9d640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7c480a3e40 0x7f7c480a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c580f0470 tx=0x7f7c58003bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c4cff9640 1 -- 172.21.15.66:0/363929597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5001b9a0 con 0x56084d9ae010 2024-04-21T09:09:22.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c67976740 1 -- 172.21.15.66:0/363929597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 msgr2=0x56084d9bac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c67976740 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bac50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000dcb0 tx=0x7f7c5000f730 comp rx=0 tx=0).stop 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c67976740 1 -- 172.21.15.66:0/363929597 shutdown_connections 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c67976740 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c67976740 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bac50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c67976740 1 --2- 172.21.15.66:0/363929597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.599+0000 7f7c67976740 1 -- 172.21.15.66:0/363929597 >> 172.21.15.66:0/363929597 conn(0x56084d9a7fb0 msgr2=0x56084d9a8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 -- 172.21.15.66:0/363929597 shutdown_connections 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 -- 172.21.15.66:0/363929597 wait complete. 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 -- start start 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 Processor -- start 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ad660 0x56084d9b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ae010 0x56084d9b8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9bb220 0x56084d9b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56084d9be150 con 0x56084d9bb220 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56084d8cb580 con 0x56084d9ae010 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56084d9be2d0 con 0x56084d9ad660 2024-04-21T09:09:22.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c67976740 10 client.0 _ll_register_callbacks cb 0x56084d8d1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c5dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9bb220 0x56084d9b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ae010 0x56084d9b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ae010 0x56084d9b8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:43418/0 (socket says 172.21.15.66:43418) 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c5e59c640 1 -- 172.21.15.66:0/4002612449 learned_addr learned my addr 172.21.15.66:0/4002612449 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.600+0000 7f7c5dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9bb220 0x56084d9b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54900/0 (socket says 172.21.15.66:54900) 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ad660 0x56084d9b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c5e59c640 1 -- 172.21.15.66:0/4002612449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ad660 msgr2=0x56084d9b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ad660 0x56084d9b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c5e59c640 1 -- 172.21.15.66:0/4002612449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9bb220 msgr2=0x56084d9b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9bb220 0x56084d9b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c5e59c640 1 -- 172.21.15.66:0/4002612449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084d74cf70 con 0x56084d9ae010 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ae010 0x56084d9b8700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000dfd0 tx=0x7f7c5001b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d468def31080c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c45ffb640 1 -- 172.21.15.66:0/4002612449 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5002a060 con 0x56084d9ae010 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c50011a90 con 0x56084d9ae010 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c45ffb640 1 -- 172.21.15.66:0/4002612449 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7c50004040 con 0x56084d9ae010 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c45ffb640 1 -- 172.21.15.66:0/4002612449 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c50031050 con 0x56084d9ae010 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c67976740 10 client.25766 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.601+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56084d5e4170 con 0x56084d9ae010 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c45ffb640 1 -- 172.21.15.66:0/4002612449 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c50011a90 con 0x56084d9ae010 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c45ffb640 1 -- 172.21.15.66:0/4002612449 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c5001b9d0 con 0x56084d9ae010 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c67976740 10 client.25766 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c67976740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c67976740 20 client.25766 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c67976740 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084d9cb720 0x56084d9cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c67976740 4 client.25766 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084d91f920 con 0x56084d9cb720 2024-04-21T09:09:22.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084d9cb720 0x56084d9cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c45ffb640 10 client.25766 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.602+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084d9cb720 0x56084d9cdb10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f9423eb2f5747b server_cookie=5d73112f650facaf in_seq=0 out_seq=0 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c45ffb640 1 -- 172.21.15.66:0/4002612449 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56084d91f920 con 0x56084d9cb720 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c45ffb640 10 client.25766 handle_command_reply: tid=0 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 client.25766 shutdown 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ae010 msgr2=0x56084d9b8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ae010 0x56084d9b8700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000dfd0 tx=0x7f7c5001b540 comp rx=0 tx=0).stop 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 shutdown_connections 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084d9cb720 0x56084d9cdb10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9bb220 0x56084d9b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ae010 0x56084d9b8700 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 --2- 172.21.15.66:0/4002612449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ad660 0x56084d9b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c45ffb640 0 client.25766 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 >> 172.21.15.66:0/4002612449 conn(0x56084d9a7fb0 msgr2=0x56084d9a9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 shutdown_connections 2024-04-21T09:09:22.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.603+0000 7f7c67976740 1 -- 172.21.15.66:0/4002612449 wait complete. 2024-04-21T09:09:22.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.604+0000 7f7c67976740 20 client.25766 trim_cache size 0 max 16384 2024-04-21T09:09:22.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.621+0000 7f7c67976740 1 Processor -- start 2024-04-21T09:09:22.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.621+0000 7f7c67976740 1 -- start start 2024-04-21T09:09:22.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c67976740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ad660 0x56084d9ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c67976740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ae010 0x56084d9c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c67976740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9bb220 0x56084d9d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c67976740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601af8e0 con 0x56084d9ad660 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c67976740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601ad1b0 con 0x56084d9bb220 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c67976740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601aafe0 con 0x56084d9ae010 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ad660 0x56084d9ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ad660 0x56084d9ba8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54902/0 (socket says 172.21.15.66:54902) 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 -- 172.21.15.66:0/3767154734 learned_addr learned my addr 172.21.15.66:0/3767154734 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ae010 0x56084d9c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9bb220 0x56084d9d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 -- 172.21.15.66:0/3767154734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ae010 msgr2=0x56084d9c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ae010 0x56084d9c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 -- 172.21.15.66:0/3767154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9bb220 msgr2=0x56084d9d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9bb220 0x56084d9d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 -- 172.21.15.66:0/3767154734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084d9b3da0 con 0x56084d9ad660 2024-04-21T09:09:22.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.622+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ad660 0x56084d9ba8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7c540058c0 tx=0x7f7c540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff998f48f6b99e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c47fff640 1 -- 172.21.15.66:0/3767154734 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54027020 con 0x56084d9ad660 2024-04-21T09:09:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c47fff640 1 -- 172.21.15.66:0/3767154734 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c54012dd0 con 0x56084d9ad660 2024-04-21T09:09:22.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c47fff640 1 -- 172.21.15.66:0/3767154734 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c540218d0 con 0x56084d9ad660 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c67976740 1 -- 172.21.15.66:0/3767154734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ad660 msgr2=0x56084d9ba8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c67976740 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ad660 0x56084d9ba8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7c540058c0 tx=0x7f7c540046c0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c67976740 1 -- 172.21.15.66:0/3767154734 shutdown_connections 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c67976740 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9bb220 0x56084d9d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c67976740 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9ae010 0x56084d9c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c67976740 1 --2- 172.21.15.66:0/3767154734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ad660 0x56084d9ba8f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.623+0000 7f7c67976740 1 -- 172.21.15.66:0/3767154734 >> 172.21.15.66:0/3767154734 conn(0x56084d9a7fb0 msgr2=0x56084d9a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 -- 172.21.15.66:0/3767154734 shutdown_connections 2024-04-21T09:09:22.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 -- 172.21.15.66:0/3767154734 wait complete. 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 -- start start 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 Processor -- start 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601ab670 con 0x56084d9ae010 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601acae0 con 0x56084d9ad660 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601abd40 con 0x56084d9bb220 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c67976740 10 client.0 _ll_register_callbacks cb 0x7f7c50015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54918/0 (socket says 172.21.15.66:54918) 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c5e59c640 1 -- 172.21.15.66:0/3319680360 learned_addr learned my addr 172.21.15.66:0/3319680360 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.624+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c5e59c640 1 -- 172.21.15.66:0/3319680360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 msgr2=0x56084d9bfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c5e59c640 1 -- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 msgr2=0x56084d9c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9c3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c5e59c640 1 -- 172.21.15.66:0/3319680360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084d9c0420 con 0x56084d9ae010 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c5e59c640 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bf9a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006010 tx=0x7f7c5000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c54f81dab8877fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c45ffb640 1 -- 172.21.15.66:0/3319680360 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5002a040 con 0x56084d9ae010 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c45ffb640 1 -- 172.21.15.66:0/3319680360 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7c5001b6f0 con 0x56084d9ae010 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c54012dd0 con 0x56084d9ae010 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c45ffb640 1 -- 172.21.15.66:0/3319680360 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c5000db00 con 0x56084d9ae010 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c67976740 10 client.16170 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:22.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56084da32650 con 0x56084d9ae010 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.625+0000 7f7c45ffb640 1 -- 172.21.15.66:0/3319680360 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c54012dd0 con 0x56084d9ae010 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c45ffb640 1 -- 172.21.15.66:0/3319680360 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c50005030 con 0x56084d9ae010 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c67976740 10 client.16170 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c67976740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c67976740 20 client.16170 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c67976740 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084da32d60 0x56084da35180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c67976740 4 client.16170 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56084d919790 con 0x56084da32d60 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084da32d60 0x56084da35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c45ffb640 10 client.16170 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.626+0000 7f7c5dd9b640 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084da32d60 0x56084da35180 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29889a91e896121b server_cookie=6715ba8b6abf00cf in_seq=0 out_seq=0 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c45ffb640 1 -- 172.21.15.66:0/3319680360 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56084d919790 con 0x56084da32d60 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c45ffb640 10 client.16170 handle_command_reply: tid=0 2024-04-21T09:09:22.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 client.16170 shutdown 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 msgr2=0x56084d9bf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bf9a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006010 tx=0x7f7c5000d520 comp rx=0 tx=0).stop 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 shutdown_connections 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56084da32d60 0x56084da35180 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c45ffb640 0 client.16170 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56084d9bb220 0x56084d9bfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56084d9ae010 0x56084d9bf9a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 --2- 172.21.15.66:0/3319680360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56084d9ad660 0x56084d9c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 >> 172.21.15.66:0/3319680360 conn(0x56084d9a7fb0 msgr2=0x56084d9b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 shutdown_connections 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 1 -- 172.21.15.66:0/3319680360 wait complete. 2024-04-21T09:09:22.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.628+0000 7f7c67976740 20 client.16170 trim_cache size 0 max 16384 2024-04-21T09:09:22.633 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:09:22.633 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:09:22.633 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "c6406f9a-7251-45e7-9280-588fdb4f6179", 2024-04-21T09:09:22.633 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:09:22.633 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7c480a3e40 msgr2=0x7f7c480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7c480a3e40 0x7f7c480a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c580f0470 tx=0x7f7c58003bc0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6019cfe0 msgr2=0x7f7c6019d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6019cfe0 0x7f7c6019d490 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005e30 tx=0x7f7c58034760 comp rx=0 tx=0).stop 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 shutdown_connections 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7c480a3e40 0x7f7c480a6300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6019cfe0 0x7f7c6019d490 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c60107270 0x7f7c60198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 --2- 172.21.15.66:0/3157528896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c601021a0 0x7f7c60198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 >> 172.21.15.66:0/3157528896 conn(0x7f7c60079350 msgr2=0x7f7c60100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 shutdown_connections 2024-04-21T09:09:22.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.629+0000 7f7c64d8e640 1 -- 172.21.15.66:0/3157528896 wait complete. 2024-04-21T09:09:22.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:09:22.640 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 -- 172.21.15.66:0/2739565923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101a30 msgr2=0x7f0598105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 --2- 172.21.15.66:0/2739565923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101a30 0x7f0598105a70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f058c009900 tx=0x7f058c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 -- 172.21.15.66:0/2739565923 shutdown_connections 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 --2- 172.21.15.66:0/2739565923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598106150 0x7f059810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 --2- 172.21.15.66:0/2739565923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101a30 0x7f0598105a70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 --2- 172.21.15.66:0/2739565923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0598101080 0x7f0598101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 -- 172.21.15.66:0/2739565923 >> 172.21.15.66:0/2739565923 conn(0x7f0598078000 msgr2=0x7f0598078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 -- 172.21.15.66:0/2739565923 shutdown_connections 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 -- 172.21.15.66:0/2739565923 wait complete. 2024-04-21T09:09:22.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 Processor -- start 2024-04-21T09:09:22.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.750+0000 7f059e76d640 1 -- start start 2024-04-21T09:09:22.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f059e76d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101080 0x7f05981995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f059e76d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598106150 0x7f0598199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f059e76d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f059819a080 0x7f05981a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f059e76d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05981110f0 con 0x7f0598101080 2024-04-21T09:09:22.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f059e76d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0598110f70 con 0x7f0598106150 2024-04-21T09:09:22.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f059e76d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0598111270 con 0x7f059819a080 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101080 0x7f05981995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101080 0x7f05981995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54930/0 (socket says 172.21.15.66:54930) 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 -- 172.21.15.66:0/884200514 learned_addr learned my addr 172.21.15.66:0/884200514 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f05977fe640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598106150 0x7f0598199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f059cd0a640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f059819a080 0x7f05981a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 -- 172.21.15.66:0/884200514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f059819a080 msgr2=0x7f05981a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f059819a080 0x7f05981a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 -- 172.21.15.66:0/884200514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598106150 msgr2=0x7f0598199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598106150 0x7f0598199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.751+0000 7f0597fff640 1 -- 172.21.15.66:0/884200514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981a4e00 con 0x7f0598101080 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.752+0000 7f0597fff640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101080 0x7f05981995b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f058800b4a0 tx=0x7f058800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.752+0000 7f05957fa640 1 -- 172.21.15.66:0/884200514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588014070 con 0x7f0598101080 2024-04-21T09:09:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.752+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05981a50f0 con 0x7f0598101080 2024-04-21T09:09:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.752+0000 7f05957fa640 1 -- 172.21.15.66:0/884200514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0588010ac0 con 0x7f0598101080 2024-04-21T09:09:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.752+0000 7f05957fa640 1 -- 172.21.15.66:0/884200514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588017c30 con 0x7f0598101080 2024-04-21T09:09:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.752+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a5610 con 0x7f0598101080 2024-04-21T09:09:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.754+0000 7f05957fa640 1 -- 172.21.15.66:0/884200514 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7f0588017070 con 0x7f0598101080 2024-04-21T09:09:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.755+0000 7f05957fa640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f05800a3e40 0x7f05800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.755+0000 7f05957fa640 1 -- 172.21.15.66:0/884200514 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f05880d0010 con 0x7f0598101080 2024-04-21T09:09:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.755+0000 7f05977fe640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f05800a3e40 0x7f05800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.755+0000 7f05977fe640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f05800a3e40 0x7f05800a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f058c009230 tx=0x7f058c005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 Processor -- start 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 -- start start 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aed2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aedb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aedde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55632ae371e0 con 0x55632aed2ea0 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55632add4390 con 0x55632aed2580 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05a1433740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55632ae331a0 con 0x55632aedba80 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aedb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aedb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54944/0 (socket says 172.21.15.66:54944) 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f05977fe640 1 -- 172.21.15.66:0/2316959803 learned_addr learned my addr 172.21.15.66:0/2316959803 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.756+0000 7f0597fff640 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aed2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f059cd0a640 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aedde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f05977fe640 1 -- 172.21.15.66:0/2316959803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 msgr2=0x55632aedde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f05977fe640 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aedde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f05977fe640 1 -- 172.21.15.66:0/2316959803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 msgr2=0x55632aed2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f05977fe640 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aed2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f05977fe640 1 -- 172.21.15.66:0/2316959803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632ac137d0 con 0x55632aed2ea0 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f05977fe640 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aedb540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f058c035000 tx=0x7f058c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4833afecd1f83085 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f057e7fc640 1 -- 172.21.15.66:0/2316959803 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0044c0 con 0x55632aed2ea0 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f057e7fc640 1 -- 172.21.15.66:0/2316959803 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f058c050070 con 0x55632aed2ea0 2024-04-21T09:09:22.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.757+0000 7f057e7fc640 1 -- 172.21.15.66:0/2316959803 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0035f0 con 0x55632aed2ea0 2024-04-21T09:09:22.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 -- 172.21.15.66:0/2316959803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 msgr2=0x55632aedb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aedb540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f058c035000 tx=0x7f058c03e000 comp rx=0 tx=0).stop 2024-04-21T09:09:22.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 -- 172.21.15.66:0/2316959803 shutdown_connections 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aedde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aedb540 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 --2- 172.21.15.66:0/2316959803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aed2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 -- 172.21.15.66:0/2316959803 >> 172.21.15.66:0/2316959803 conn(0x55632aecccb0 msgr2=0x55632aecd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 -- 172.21.15.66:0/2316959803 shutdown_connections 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 -- 172.21.15.66:0/2316959803 wait complete. 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.758+0000 7f05a1433740 1 -- start start 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 1 Processor -- start 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aed8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aed9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aed9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55632aede810 con 0x55632aedba80 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55632adf02f0 con 0x55632aed2ea0 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55632aede990 con 0x55632aed2580 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05a1433740 10 client.0 _ll_register_callbacks cb 0x55632adfd580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:22.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aed9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aed9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54950/0 (socket says 172.21.15.66:54950) 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 -- 172.21.15.66:0/124343074 learned_addr learned my addr 172.21.15.66:0/124343074 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f0597fff640 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aed8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f05977fe640 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aed9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 -- 172.21.15.66:0/124343074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 msgr2=0x55632aed8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aed8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 -- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 msgr2=0x55632aed9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aed9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.759+0000 7f059cd0a640 1 -- 172.21.15.66:0/124343074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632ab493f0 con 0x55632aedba80 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f059cd0a640 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aed9560 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f05900126a0 tx=0x7f0590012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d3e6c33e59243e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05777fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05777fe640 1 -- 172.21.15.66:0/124343074 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590004d20 con 0x55632aedba80 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05777fe640 1 -- 172.21.15.66:0/124343074 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f0590010b00 con 0x55632aedba80 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f058c050070 con 0x55632aedba80 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05777fe640 1 -- 172.21.15.66:0/124343074 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0590025050 con 0x55632aedba80 2024-04-21T09:09:22.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 10 client.16182 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55632ab090f0 con 0x55632aedba80 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05777fe640 1 -- 172.21.15.66:0/124343074 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f058c050070 con 0x55632aedba80 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05777fe640 1 -- 172.21.15.66:0/124343074 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0590019030 con 0x55632aedba80 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 10 client.16182 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 20 client.16182 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632aeebb40 0x55632aeedf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 4 client.16182 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.760+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55632ae44c00 con 0x55632aeebb40 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.761+0000 7f0597fff640 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632aeebb40 0x55632aeedf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.761+0000 7f05777fe640 10 client.16182 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.761+0000 7f0597fff640 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632aeebb40 0x55632aeedf70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15eafd4d958d3fb server_cookie=6428c28acf56ca84 in_seq=0 out_seq=0 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05777fe640 1 -- 172.21.15.66:0/124343074 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55632ae44c00 con 0x55632aeebb40 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05777fe640 10 client.16182 handle_command_reply: tid=0 2024-04-21T09:09:22.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 client.16182 shutdown 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 msgr2=0x55632aed9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aed9560 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f05900126a0 tx=0x7f0590012b60 comp rx=0 tx=0).stop 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 shutdown_connections 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632aeebb40 0x55632aeedf70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aed9560 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aed9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 --2- 172.21.15.66:0/124343074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aed8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05777fe640 0 client.16182 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 >> 172.21.15.66:0/124343074 conn(0x55632aecccb0 msgr2=0x55632aecea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 shutdown_connections 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 1 -- 172.21.15.66:0/124343074 wait complete. 2024-04-21T09:09:22.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.762+0000 7f05a1433740 20 client.16182 trim_cache size 0 max 16384 2024-04-21T09:09:22.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 Processor -- start 2024-04-21T09:09:22.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 -- start start 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aedb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aee76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aed7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b1ac0 con 0x55632aed2ea0 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05981af390 con 0x55632aed2580 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05a1433740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f05981ad1c0 con 0x55632aedba80 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aee76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aee76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54952/0 (socket says 172.21.15.66:54952) 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f05977fe640 1 -- 172.21.15.66:0/1781796191 learned_addr learned my addr 172.21.15.66:0/1781796191 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f0597fff640 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aedb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.780+0000 7f059cd0a640 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aed7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f05977fe640 1 -- 172.21.15.66:0/1781796191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 msgr2=0x55632aed7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f05977fe640 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aed7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f05977fe640 1 -- 172.21.15.66:0/1781796191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 msgr2=0x55632aedb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f05977fe640 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aedb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f05977fe640 1 -- 172.21.15.66:0/1781796191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632aed2cd0 con 0x55632aed2ea0 2024-04-21T09:09:22.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f0597fff640 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aedb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:22.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f05977fe640 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aee76d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f058c004bc0 tx=0x7f058c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2433be36f177e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f057dffb640 1 -- 172.21.15.66:0/1781796191 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c05a070 con 0x55632aed2ea0 2024-04-21T09:09:22.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f057dffb640 1 -- 172.21.15.66:0/1781796191 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f058c050040 con 0x55632aed2ea0 2024-04-21T09:09:22.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.781+0000 7f057dffb640 1 -- 172.21.15.66:0/1781796191 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c041460 con 0x55632aed2ea0 2024-04-21T09:09:22.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 -- 172.21.15.66:0/1781796191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 msgr2=0x55632aee76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aee76d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f058c004bc0 tx=0x7f058c004b50 comp rx=0 tx=0).stop 2024-04-21T09:09:22.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 -- 172.21.15.66:0/1781796191 shutdown_connections 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aedba80 0x55632aed7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aed2ea0 0x55632aee76d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 --2- 172.21.15.66:0/1781796191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2580 0x55632aedb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 -- 172.21.15.66:0/1781796191 >> 172.21.15.66:0/1781796191 conn(0x55632aecccb0 msgr2=0x55632aed8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 -- 172.21.15.66:0/1781796191 shutdown_connections 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 -- 172.21.15.66:0/1781796191 wait complete. 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 -- start start 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.782+0000 7f05a1433740 1 Processor -- start 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05a1433740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aedf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05a1433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aedfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05a1433740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aee0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05a1433740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05981ad850 con 0x55632aedba80 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05a1433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05981aecc0 con 0x55632aed2ea0 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05a1433740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f05981adf20 con 0x55632aed2580 2024-04-21T09:09:22.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05a1433740 10 client.0 _ll_register_callbacks cb 0x7f0590009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aee0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aee0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54964/0 (socket says 172.21.15.66:54964) 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 -- 172.21.15.66:0/3548412076 learned_addr learned my addr 172.21.15.66:0/3548412076 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05977fe640 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aedfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f0597fff640 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aedf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 -- 172.21.15.66:0/3548412076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 msgr2=0x55632aedf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aedf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 -- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 msgr2=0x55632aedfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aedfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 -- 172.21.15.66:0/3548412076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632af51160 con 0x55632aedba80 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f059cd0a640 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aee0120 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0590012670 tx=0x7f0590012300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c66bf8869b6e7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.783+0000 7f05777fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05777fe640 1 -- 172.21.15.66:0/3548412076 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590010d00 con 0x55632aedba80 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05777fe640 1 -- 172.21.15.66:0/3548412076 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f0590003970 con 0x55632aedba80 2024-04-21T09:09:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f058c050040 con 0x55632aedba80 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05777fe640 1 -- 172.21.15.66:0/3548412076 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f059003a050 con 0x55632aedba80 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 10 client.16188 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55632af52450 con 0x55632aedba80 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05777fe640 1 -- 172.21.15.66:0/3548412076 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f058c050040 con 0x55632aedba80 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05777fe640 1 -- 172.21.15.66:0/3548412076 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0590019040 con 0x55632aedba80 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 10 client.16188 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 20 client.16188 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632af52b00 0x55632af54ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 4 client.16188 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.784+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55632ae3ea70 con 0x55632af52b00 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f0597fff640 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632af52b00 0x55632af54ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05777fe640 10 client.16188 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f0597fff640 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632af52b00 0x55632af54ef0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342d5823c9369f3a server_cookie=7bcf0f56beb910e in_seq=0 out_seq=0 2024-04-21T09:09:22.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05777fe640 1 -- 172.21.15.66:0/3548412076 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55632ae3ea70 con 0x55632af52b00 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05777fe640 10 client.16188 handle_command_reply: tid=0 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 client.16188 shutdown 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 msgr2=0x55632aee0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aee0120 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0590012670 tx=0x7f0590012300 comp rx=0 tx=0).stop 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 shutdown_connections 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55632af52b00 0x55632af54ef0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55632aedba80 0x55632aee0120 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55632aed2ea0 0x55632aedfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05777fe640 0 client.16188 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 --2- 172.21.15.66:0/3548412076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55632aed2580 0x55632aedf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.785+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 >> 172.21.15.66:0/3548412076 conn(0x55632aecccb0 msgr2=0x55632aed9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.786+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 shutdown_connections 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.786+0000 7f05a1433740 1 -- 172.21.15.66:0/3548412076 wait complete. 2024-04-21T09:09:22.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.786+0000 7f05a1433740 20 client.16188 trim_cache size 0 max 16384 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: "c6406f9a-7251-45e7-9280-588fdb4f6179": { 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "c6406f9a-7251-45e7-9280-588fdb4f6179", 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:09:22.791 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:09:22.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.786+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f05800a3e40 msgr2=0x7f05800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.786+0000 7f059e76d640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f05800a3e40 0x7f05800a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f058c009230 tx=0x7f058c005fd0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.786+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101080 msgr2=0x7f05981995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:22.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.786+0000 7f059e76d640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101080 0x7f05981995b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f058800b4a0 tx=0x7f058800b960 comp rx=0 tx=0).stop 2024-04-21T09:09:22.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 shutdown_connections 2024-04-21T09:09:22.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f05800a3e40 0x7f05800a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f059819a080 0x7f05981a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598106150 0x7f0598199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 --2- 172.21.15.66:0/884200514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0598101080 0x7f05981995b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:22.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 >> 172.21.15.66:0/884200514 conn(0x7f0598078000 msgr2=0x7f05980ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:22.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 shutdown_connections 2024-04-21T09:09:22.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:22.787+0000 7f059e76d640 1 -- 172.21.15.66:0/884200514 wait complete. 2024-04-21T09:09:22.803 INFO:tasks.cephfs.filesystem:scrub status for tag:c6406f9a-7251-45e7-9280-588fdb4f6179 - {'path': '/', 'tag': 'c6406f9a-7251-45e7-9280-588fdb4f6179', 'options': 'recursive,force'} 2024-04-21T09:09:24.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:23 smithi186 ceph-mon[31505]: pgmap v38: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 863 KiB/s wr, 11 op/s 2024-04-21T09:09:24.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:23 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:09:24.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:23 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:24.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:23 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:09:24.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:23 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:24.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:23 smithi083 ceph-mon[31413]: pgmap v38: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 863 KiB/s wr, 11 op/s 2024-04-21T09:09:24.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:23 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:09:24.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:23 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:24.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:23 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:09:24.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:23 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:24.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:23 smithi066 ceph-mon[27953]: pgmap v38: 129 pgs: 129 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 863 KiB/s wr, 11 op/s 2024-04-21T09:09:24.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:23 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:09:24.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:23 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:24.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:23 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:09:24.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:23 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 -- 172.21.15.66:0/1327780759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe344074320 msgr2=0x7fe344074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/1327780759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe344074320 0x7fe344074700 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe340009900 tx=0x7fe34002bcd0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 -- 172.21.15.66:0/1327780759 shutdown_connections 2024-04-21T09:09:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/1327780759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe34410e7d0 0x7fe344110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/1327780759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe344074c40 0x7fe34410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/1327780759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe344074320 0x7fe344074700 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 -- 172.21.15.66:0/1327780759 >> 172.21.15.66:0/1327780759 conn(0x7fe3440fda00 msgr2=0x7fe3440ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 -- 172.21.15.66:0/1327780759 shutdown_connections 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.953+0000 7fe34b9e6640 1 -- 172.21.15.66:0/1327780759 wait complete. 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 Processor -- start 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 -- start start 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe344074320 0x7fe3441a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344074c40 0x7fe3441a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34410e7d0 0x7fe34419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441143f0 con 0x7fe34410e7d0 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe344114270 con 0x7fe344074c40 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe34b9e6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe344114570 con 0x7fe344074320 2024-04-21T09:09:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe349f83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34410e7d0 0x7fe34419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe348f81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344074c40 0x7fe3441a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe349f83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34410e7d0 0x7fe34419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33388/0 (socket says 172.21.15.66:33388) 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe349f83640 1 -- 172.21.15.66:0/3012322386 learned_addr learned my addr 172.21.15.66:0/3012322386 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.954+0000 7fe349782640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe344074320 0x7fe3441a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe348f81640 1 -- 172.21.15.66:0/3012322386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe344074320 msgr2=0x7fe3441a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe348f81640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe344074320 0x7fe3441a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe348f81640 1 -- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34410e7d0 msgr2=0x7fe34419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe348f81640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34410e7d0 0x7fe34419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe348f81640 1 -- 172.21.15.66:0/3012322386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34419aff0 con 0x7fe344074c40 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe349f83640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34410e7d0 0x7fe34419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe348f81640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344074c40 0x7fe3441a0bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe33400d6d0 tx=0x7fe33400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe33a7fc640 1 -- 172.21.15.66:0/3012322386 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334017070 con 0x7fe344074c40 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe33a7fc640 1 -- 172.21.15.66:0/3012322386 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe334009820 con 0x7fe344074c40 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34419b2e0 con 0x7fe344074c40 2024-04-21T09:09:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe33a7fc640 1 -- 172.21.15.66:0/3012322386 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334004c40 con 0x7fe344074c40 2024-04-21T09:09:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.955+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441a75f0 con 0x7fe344074c40 2024-04-21T09:09:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.958+0000 7fe33a7fc640 1 -- 172.21.15.66:0/3012322386 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7fe334005220 con 0x7fe344074c40 2024-04-21T09:09:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 Processor -- start 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 -- start start 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe33a7fc640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3300a3ee0 0x7fe3300a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f00ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f00f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f01ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f88ef6b8d0 con 0x55f88f0100e0 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f88ef08e80 con 0x55f88f0188f0 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe349f83640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3300a3ee0 0x7fe3300a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f88ef67890 con 0x55f88f00f370 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe33a7fc640 1 -- 172.21.15.66:0/3012322386 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe3340d0960 con 0x7fe344074c40 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe349782640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f00ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe349782640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f00ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33398/0 (socket says 172.21.15.66:33398) 2024-04-21T09:09:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.959+0000 7fe349782640 1 -- 172.21.15.66:0/833939965 learned_addr learned my addr 172.21.15.66:0/833939965 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe348f81640 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f00f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349f83640 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f01ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349f83640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3300a3ee0 0x7fe3300a63a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0049d0 tx=0x7fe33c011a80 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349782640 1 -- 172.21.15.66:0/833939965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 msgr2=0x55f88f00f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349782640 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f00f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349782640 1 -- 172.21.15.66:0/833939965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 msgr2=0x55f88f01ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349782640 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f01ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349782640 1 -- 172.21.15.66:0/833939965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88ef2a320 con 0x55f88f0100e0 2024-04-21T09:09:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.960+0000 7fe349782640 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f00ee30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe340009900 tx=0x7fe340005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395b4f2e4dea6d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe32f7fe640 1 -- 172.21.15.66:0/833939965 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340004240 con 0x55f88f0100e0 2024-04-21T09:09:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe32f7fe640 1 -- 172.21.15.66:0/833939965 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3400043e0 con 0x55f88f0100e0 2024-04-21T09:09:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe32f7fe640 1 -- 172.21.15.66:0/833939965 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3400046c0 con 0x55f88f0100e0 2024-04-21T09:09:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe34e5d4740 1 -- 172.21.15.66:0/833939965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 msgr2=0x55f88f00ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f00ee30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe340009900 tx=0x7fe340005e70 comp rx=0 tx=0).stop 2024-04-21T09:09:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe34e5d4740 1 -- 172.21.15.66:0/833939965 shutdown_connections 2024-04-21T09:09:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f01ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f00f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.961+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/833939965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f00ee30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 -- 172.21.15.66:0/833939965 >> 172.21.15.66:0/833939965 conn(0x55f88efcdc90 msgr2=0x55f88f00a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 -- 172.21.15.66:0/833939965 shutdown_connections 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 -- 172.21.15.66:0/833939965 wait complete. 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 -- start start 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 Processor -- start 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f018210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f0155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0188f0 0x55f88f015ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f88f01b5c0 con 0x55f88f0100e0 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f88eef3eb0 con 0x55f88f00f370 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f88f01b740 con 0x55f88f0188f0 2024-04-21T09:09:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.962+0000 7fe34e5d4740 10 client.0 _ll_register_callbacks cb 0x55f88ef20b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f0155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f0155a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33404/0 (socket says 172.21.15.66:33404) 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 -- 172.21.15.66:0/3079339271 learned_addr learned my addr 172.21.15.66:0/3079339271 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe349782640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f018210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe349f83640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0188f0 0x55f88f015ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 -- 172.21.15.66:0/3079339271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0188f0 msgr2=0x55f88f015ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0188f0 0x55f88f015ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 -- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 msgr2=0x55f88f018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f018210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 -- 172.21.15.66:0/3079339271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88ec7e210 con 0x55f88f0100e0 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe349782640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f018210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe348f81640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f0155a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe33409c040 tx=0x7fe33409cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f7ac55ccc82a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe32cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe32cff9640 1 -- 172.21.15.66:0/3079339271 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334020040 con 0x55f88f0100e0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe32cff9640 1 -- 172.21.15.66:0/3079339271 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe3340099d0 con 0x55f88f0100e0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.963+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3400043e0 con 0x55f88f0100e0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe32cff9640 1 -- 172.21.15.66:0/3079339271 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe3340bb170 con 0x55f88f0100e0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 10 client.16200 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f88ed4a620 con 0x55f88f0100e0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe32cff9640 1 -- 172.21.15.66:0/3079339271 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3400043e0 con 0x55f88f0100e0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe32cff9640 1 -- 172.21.15.66:0/3079339271 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe334098090 con 0x55f88f0100e0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 10 client.16200 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 20 client.16200 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f0287f0 0x55f88f02abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 4 client.16200 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88ef7f880 con 0x55f88f0287f0 2024-04-21T09:09:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.964+0000 7fe349782640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f0287f0 0x55f88f02abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe349782640 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f0287f0 0x55f88f02abe0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a705bbe436d6dc0 server_cookie=86c057cde089dfaa in_seq=0 out_seq=0 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe32cff9640 10 client.16200 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe32cff9640 1 -- 172.21.15.66:0/3079339271 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f88ef7f880 con 0x55f88f0287f0 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe32cff9640 10 client.16200 handle_command_reply: tid=0 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 client.16200 shutdown 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 msgr2=0x55f88f0155a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f0155a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe33409c040 tx=0x7fe33409cb80 comp rx=0 tx=0).stop 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 shutdown_connections 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f0287f0 0x55f88f02abe0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0188f0 0x55f88f015ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f0155a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe32cff9640 0 client.16200 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3079339271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f018210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.965+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 >> 172.21.15.66:0/3079339271 conn(0x55f88efcdc90 msgr2=0x55f88f00bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.966+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 shutdown_connections 2024-04-21T09:09:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.966+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3079339271 wait complete. 2024-04-21T09:09:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.966+0000 7fe34e5d4740 20 client.16200 trim_cache size 0 max 16384 2024-04-21T09:09:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 Processor -- start 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 -- start start 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f0244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f017cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f018210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441af9e0 con 0x55f88f0100e0 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441ad2b0 con 0x55f88f0188f0 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441ab100 con 0x55f88f00f370 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f017cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f017cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33406/0 (socket says 172.21.15.66:33406) 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe349f83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f018210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 -- 172.21.15.66:0/1599784646 learned_addr learned my addr 172.21.15.66:0/1599784646 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe349782640 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f0244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 -- 172.21.15.66:0/1599784646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 msgr2=0x55f88f0244f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f0244f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 -- 172.21.15.66:0/1599784646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 msgr2=0x55f88f018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f018210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe348f81640 1 -- 172.21.15.66:0/1599784646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88effcf30 con 0x55f88f0100e0 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe349782640 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f0244f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.981+0000 7fe349f83640 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f018210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.982+0000 7fe348f81640 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f017cd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe334009610 tx=0x7fe3340a5130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb3503f57e924c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.982+0000 7fe32effd640 1 -- 172.21.15.66:0/1599784646 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33409c110 con 0x55f88f0100e0 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.982+0000 7fe32effd640 1 -- 172.21.15.66:0/1599784646 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33409c340 con 0x55f88f0100e0 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.982+0000 7fe32effd640 1 -- 172.21.15.66:0/1599784646 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3340989b0 con 0x55f88f0100e0 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 -- 172.21.15.66:0/1599784646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 msgr2=0x55f88f017cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f017cd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe334009610 tx=0x7fe3340a5130 comp rx=0 tx=0).stop 2024-04-21T09:09:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 -- 172.21.15.66:0/1599784646 shutdown_connections 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f0188f0 0x55f88f018210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0100e0 0x55f88f017cd0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/1599784646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f00f370 0x55f88f0244f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 -- 172.21.15.66:0/1599784646 >> 172.21.15.66:0/1599784646 conn(0x55f88efcdc90 msgr2=0x55f88f015560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 -- 172.21.15.66:0/1599784646 shutdown_connections 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 -- 172.21.15.66:0/1599784646 wait complete. 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 -- start start 2024-04-21T09:09:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 Processor -- start 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.983+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f020bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0100e0 0x55f88f011c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe34e5d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0188f0 0x55f88f012140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441ab770 con 0x55f88f0188f0 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441acbe0 con 0x55f88f00f370 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe34e5d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441abe40 con 0x55f88f0100e0 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe34e5d4740 10 client.0 _ll_register_callbacks cb 0x7fe33409bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0188f0 0x55f88f012140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0188f0 0x55f88f012140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33414/0 (socket says 172.21.15.66:33414) 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 -- 172.21.15.66:0/3327348410 learned_addr learned my addr 172.21.15.66:0/3327348410 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349782640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f020bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe348f81640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0100e0 0x55f88f011c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 -- 172.21.15.66:0/3327348410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0100e0 msgr2=0x55f88f011c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0100e0 0x55f88f011c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 -- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 msgr2=0x55f88f020bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f020bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 -- 172.21.15.66:0/3327348410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88f012680 con 0x55f88f0188f0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe348f81640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0100e0 0x55f88f011c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349782640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f020bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe349f83640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0188f0 0x55f88f012140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00fe60 tx=0x7fe33c016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53e91d528b991cb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe32cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe32cff9640 1 -- 172.21.15.66:0/3327348410 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c02f060 con 0x55f88f0188f0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe32cff9640 1 -- 172.21.15.66:0/3327348410 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe33c020840 con 0x55f88f0188f0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.984+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33409c340 con 0x55f88f0188f0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe32cff9640 1 -- 172.21.15.66:0/3327348410 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe33c036050 con 0x55f88f0188f0 2024-04-21T09:09:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 10 client.16206 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f88f08ea30 con 0x55f88f0188f0 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe32cff9640 1 -- 172.21.15.66:0/3327348410 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33409c340 con 0x55f88f0188f0 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe32cff9640 1 -- 172.21.15.66:0/3327348410 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe33c00f040 con 0x55f88f0188f0 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 10 client.16206 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 20 client.16206 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f08f210 0x55f88f091630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 4 client.16206 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f88ef0b3d0 con 0x55f88f08f210 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.985+0000 7fe349782640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f08f210 0x55f88f091630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.986+0000 7fe32cff9640 10 client.16206 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.986+0000 7fe349782640 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f08f210 0x55f88f091630 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7c43a6328ad509 server_cookie=473e849895edad42 in_seq=0 out_seq=0 2024-04-21T09:09:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe32cff9640 1 -- 172.21.15.66:0/3327348410 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f88ef0b3d0 con 0x55f88f08f210 2024-04-21T09:09:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe32cff9640 10 client.16206 handle_command_reply: tid=0 2024-04-21T09:09:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 client.16206 shutdown 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0188f0 msgr2=0x55f88f012140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0188f0 0x55f88f012140 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00fe60 tx=0x7fe33c016070 comp rx=0 tx=0).stop 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 shutdown_connections 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f88f08f210 0x55f88f091630 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88f0188f0 0x55f88f012140 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f88f0100e0 0x55f88f011c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe32cff9640 0 client.16206 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 --2- 172.21.15.66:0/3327348410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f88f00f370 0x55f88f020bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 >> 172.21.15.66:0/3327348410 conn(0x55f88efcdc90 msgr2=0x55f88f0106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 shutdown_connections 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.995+0000 7fe34e5d4740 1 -- 172.21.15.66:0/3327348410 wait complete. 2024-04-21T09:09:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34e5d4740 20 client.16206 trim_cache size 0 max 16384 2024-04-21T09:09:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:09:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:09:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:09:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3300a3ee0 msgr2=0x7fe3300a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3300a3ee0 0x7fe3300a63a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0049d0 tx=0x7fe33c011a80 comp rx=0 tx=0).stop 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344074c40 msgr2=0x7fe3441a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344074c40 0x7fe3441a0bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe33400d6d0 tx=0x7fe33400db90 comp rx=0 tx=0).stop 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 shutdown_connections 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3300a3ee0 0x7fe3300a63a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34410e7d0 0x7fe34419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344074c40 0x7fe3441a0bb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 --2- 172.21.15.66:0/3012322386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe344074320 0x7fe3441a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 >> 172.21.15.66:0/3012322386 conn(0x7fe3440fda00 msgr2=0x7fe3440fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 shutdown_connections 2024-04-21T09:09:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:24.996+0000 7fe34b9e6640 1 -- 172.21.15.66:0/3012322386 wait complete. 2024-04-21T09:09:26.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:25 smithi186 ceph-mon[31505]: pgmap v39: 129 pgs: 129 active+clean; 2.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-21T09:09:26.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:25 smithi083 ceph-mon[31413]: pgmap v39: 129 pgs: 129 active+clean; 2.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-21T09:09:26.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:25 smithi066 ceph-mon[27953]: pgmap v39: 129 pgs: 129 active+clean; 2.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-21T09:09:27.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:26 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:09:27.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:26 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:09:27.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:26 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:09:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:27 smithi186 ceph-mon[31505]: pgmap v40: 129 pgs: 129 active+clean; 2.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 9 op/s 2024-04-21T09:09:28.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:27 smithi083 ceph-mon[31413]: pgmap v40: 129 pgs: 129 active+clean; 2.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 9 op/s 2024-04-21T09:09:28.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:27 smithi066 ceph-mon[27953]: pgmap v40: 129 pgs: 129 active+clean; 2.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 9 op/s 2024-04-21T09:09:29.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:28 smithi066 ceph-mon[27953]: pgmap v41: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:29.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:28 smithi186 ceph-mon[31505]: pgmap v41: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:29.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:28 smithi083 ceph-mon[31413]: pgmap v41: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:31 smithi186 ceph-mon[31505]: pgmap v42: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:09:31.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:31 smithi083 ceph-mon[31413]: pgmap v42: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:09:31.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:31 smithi066 ceph-mon[27953]: pgmap v42: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:09:33.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:33 smithi186 ceph-mon[31505]: pgmap v43: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:09:33.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:33 smithi083 ceph-mon[31413]: pgmap v43: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:09:33.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:33 smithi066 ceph-mon[27953]: pgmap v43: 129 pgs: 129 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:09:35.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:35 smithi186 ceph-mon[31505]: pgmap v44: 129 pgs: 129 active+clean; 2.7 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:35 smithi083 ceph-mon[31413]: pgmap v44: 129 pgs: 129 active+clean; 2.7 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:35.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:35 smithi066 ceph-mon[27953]: pgmap v44: 129 pgs: 129 active+clean; 2.7 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:37 smithi186 ceph-mon[31505]: pgmap v45: 129 pgs: 129 active+clean; 2.7 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 863 KiB/s wr, 9 op/s 2024-04-21T09:09:37.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:37 smithi083 ceph-mon[31413]: pgmap v45: 129 pgs: 129 active+clean; 2.7 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 863 KiB/s wr, 9 op/s 2024-04-21T09:09:37.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:37 smithi066 ceph-mon[27953]: pgmap v45: 129 pgs: 129 active+clean; 2.7 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 863 KiB/s wr, 9 op/s 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 -- 172.21.15.66:0/1916785586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc101ad0 msgr2=0x7f59bc105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 --2- 172.21.15.66:0/1916785586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc101ad0 0x7f59bc105ac0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009900 tx=0x7f59b002bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 -- 172.21.15.66:0/1916785586 shutdown_connections 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 --2- 172.21.15.66:0/1916785586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc1061a0 0x7f59bc10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 --2- 172.21.15.66:0/1916785586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc101ad0 0x7f59bc105ac0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 --2- 172.21.15.66:0/1916785586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc101120 0x7f59bc101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 -- 172.21.15.66:0/1916785586 >> 172.21.15.66:0/1916785586 conn(0x7f59bc077fa0 msgr2=0x7f59bc0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 -- 172.21.15.66:0/1916785586 shutdown_connections 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.090+0000 7f59c110e640 1 -- 172.21.15.66:0/1916785586 wait complete. 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 Processor -- start 2024-04-21T09:09:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 -- start start 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc101120 0x7f59bc102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc1061a0 0x7f59bc1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc1035e0 0x7f59bc1a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1110c0 con 0x7f59bc1035e0 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc110f40 con 0x7f59bc101120 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59c110e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc111240 con 0x7f59bc1061a0 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59bb59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc1035e0 0x7f59bc1a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59bb59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc1035e0 0x7f59bc1a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35936/0 (socket says 172.21.15.66:35936) 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59bb59e640 1 -- 172.21.15.66:0/1668841605 learned_addr learned my addr 172.21.15.66:0/1668841605 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.091+0000 7f59bad9d640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc101120 0x7f59bc102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59ba59c640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc1061a0 0x7f59bc1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59bb59e640 1 -- 172.21.15.66:0/1668841605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc1061a0 msgr2=0x7f59bc1030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59bb59e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc1061a0 0x7f59bc1030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59bb59e640 1 -- 172.21.15.66:0/1668841605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc101120 msgr2=0x7f59bc102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59bb59e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc101120 0x7f59bc102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59bb59e640 1 -- 172.21.15.66:0/1668841605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc1a6fa0 con 0x7f59bc1035e0 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59bb59e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc1035e0 0x7f59bc1a68a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009fd0 tx=0x7f59b400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59a3fff640 1 -- 172.21.15.66:0/1668841605 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4017070 con 0x7f59bc1035e0 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59a3fff640 1 -- 172.21.15.66:0/1668841605 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b4003e20 con 0x7f59bc1035e0 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc1a7290 con 0x7f59bc1035e0 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59a3fff640 1 -- 172.21.15.66:0/1668841605 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4010460 con 0x7f59bc1035e0 2024-04-21T09:09:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.092+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc1a77b0 con 0x7f59bc1035e0 2024-04-21T09:09:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.094+0000 7f59a3fff640 1 -- 172.21.15.66:0/1668841605 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7f59b4003970 con 0x7f59bc1035e0 2024-04-21T09:09:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.095+0000 7f59a3fff640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59a40a3f90 0x7f59a40a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.095+0000 7f59bad9d640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59a40a3f90 0x7f59a40a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.095+0000 7f59a3fff640 1 -- 172.21.15.66:0/1668841605 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f59b40d0a90 con 0x7f59bc1035e0 2024-04-21T09:09:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.095+0000 7f59bad9d640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59a40a3f90 0x7f59a40a6450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005d00 tx=0x7f59ac00b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 Processor -- start 2024-04-21T09:09:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 -- start start 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55735f5008d0 con 0x55735f59c8d0 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55735f49de80 con 0x55735f59bb60 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55735f4fc890 con 0x55735f5a50e0 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.096+0000 7f59bad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35940/0 (socket says 172.21.15.66:35940) 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 -- 172.21.15.66:0/4283067828 learned_addr learned my addr 172.21.15.66:0/4283067828 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59ba59c640 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bb59e640 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 -- 172.21.15.66:0/4283067828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 msgr2=0x55735f5a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 -- 172.21.15.66:0/4283067828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 msgr2=0x55735f59bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 -- 172.21.15.66:0/4283067828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735f4bf320 con 0x55735f59c8d0 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59bad9d640 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59b620 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f59ac011010 tx=0x7f59ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f4d5c88e2bbb94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59a17fa640 1 -- 172.21.15.66:0/4283067828 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac01e620 con 0x55735f59c8d0 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59a17fa640 1 -- 172.21.15.66:0/4283067828 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ac01ec70 con 0x55735f59c8d0 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.097+0000 7f59a17fa640 1 -- 172.21.15.66:0/4283067828 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac026be0 con 0x55735f59c8d0 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 -- 172.21.15.66:0/4283067828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 msgr2=0x55735f59b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59b620 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f59ac011010 tx=0x7f59ac013000 comp rx=0 tx=0).stop 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 -- 172.21.15.66:0/4283067828 shutdown_connections 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/4283067828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59b620 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 -- 172.21.15.66:0/4283067828 >> 172.21.15.66:0/4283067828 conn(0x55735f562c90 msgr2=0x55735f596850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 -- 172.21.15.66:0/4283067828 shutdown_connections 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 -- 172.21.15.66:0/4283067828 wait complete. 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.098+0000 7f59c3dd9740 1 -- start start 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 1 Processor -- start 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59c8d0 0x55735f5a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55735f5a7e10 con 0x55735f59bb60 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55735f488eb0 con 0x55735f59c8d0 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55735f5a7f90 con 0x55735f5a50e0 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59c3dd9740 10 client.0 _ll_register_callbacks cb 0x55735f4b5b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35954/0 (socket says 172.21.15.66:35954) 2024-04-21T09:09:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 -- 172.21.15.66:0/1821547532 learned_addr learned my addr 172.21.15.66:0/1821547532 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59ba59c640 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59c8d0 0x55735f5a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bb59e640 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 -- 172.21.15.66:0/1821547532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 msgr2=0x55735f5a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 -- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59c8d0 msgr2=0x55735f5a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59c8d0 0x55735f5a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.099+0000 7f59bad9d640 1 -- 172.21.15.66:0/1821547532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735f213210 con 0x55735f59bb60 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59bad9d640 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5a4af0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f59ac026610 tx=0x7f59ac026640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1868cf61306b3160 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f599eff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f599eff5640 1 -- 172.21.15.66:0/1821547532 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac037060 con 0x55735f59bb60 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f599eff5640 1 -- 172.21.15.66:0/1821547532 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f59ac027a30 con 0x55735f59bb60 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac01ec70 con 0x55735f59bb60 2024-04-21T09:09:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f599eff5640 1 -- 172.21.15.66:0/1821547532 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f59ac03e050 con 0x55735f59bb60 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 10 client.16218 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55735f2df620 con 0x55735f59bb60 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f599eff5640 1 -- 172.21.15.66:0/1821547532 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ac01ec70 con 0x55735f59bb60 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f599eff5640 1 -- 172.21.15.66:0/1821547532 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f59ac032570 con 0x55735f59bb60 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 10 client.16218 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 20 client.16218 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f5b5080 0x55735f5b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 4 client.16218 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.100+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55735f514880 con 0x55735f5b5080 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.101+0000 7f59ba59c640 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f5b5080 0x55735f5b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.101+0000 7f59ba59c640 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f5b5080 0x55735f5b7470 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9288b278ced9e3d4 server_cookie=155cd4002d51e136 in_seq=0 out_seq=0 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.101+0000 7f599eff5640 10 client.16218 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f599eff5640 1 -- 172.21.15.66:0/1821547532 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55735f514880 con 0x55735f5b5080 2024-04-21T09:09:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f599eff5640 10 client.16218 handle_command_reply: tid=0 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 client.16218 shutdown 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 msgr2=0x55735f5a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5a4af0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f59ac026610 tx=0x7f59ac026640 comp rx=0 tx=0).stop 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 shutdown_connections 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f5b5080 0x55735f5b7470 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f5a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59c8d0 0x55735f5a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f599eff5640 0 client.16218 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/1821547532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5a4af0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 >> 172.21.15.66:0/1821547532 conn(0x55735f562c90 msgr2=0x55735f5985a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 shutdown_connections 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 1 -- 172.21.15.66:0/1821547532 wait complete. 2024-04-21T09:09:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.102+0000 7f59c3dd9740 20 client.16218 trim_cache size 0 max 16384 2024-04-21T09:09:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.134+0000 7f59c3dd9740 1 Processor -- start 2024-04-21T09:09:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.134+0000 7f59c3dd9740 1 -- start start 2024-04-21T09:09:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f59c8d0 0x55735f5a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f5a50e0 0x55735f5a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b3c50 con 0x55735f59bb60 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b1520 con 0x55735f5a50e0 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1af350 con 0x55735f59c8d0 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35956/0 (socket says 172.21.15.66:35956) 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 -- 172.21.15.66:0/3266731390 learned_addr learned my addr 172.21.15.66:0/3266731390 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bb59e640 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f5a50e0 0x55735f5a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59ba59c640 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f59c8d0 0x55735f5a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 -- 172.21.15.66:0/3266731390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f59c8d0 msgr2=0x55735f5a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f59c8d0 0x55735f5a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 -- 172.21.15.66:0/3266731390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f5a50e0 msgr2=0x55735f5a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f5a50e0 0x55735f5a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 -- 172.21.15.66:0/3266731390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735f59b990 con 0x55735f59bb60 2024-04-21T09:09:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59bad9d640 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5b0d10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f59ac011010 tx=0x7f59ac032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66844df9d626e805 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59a0ff9640 1 -- 172.21.15.66:0/3266731390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac037070 con 0x55735f59bb60 2024-04-21T09:09:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.135+0000 7f59a0ff9640 1 -- 172.21.15.66:0/3266731390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ac0266b0 con 0x55735f59bb60 2024-04-21T09:09:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.136+0000 7f59a0ff9640 1 -- 172.21.15.66:0/3266731390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac01e640 con 0x55735f59bb60 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- 172.21.15.66:0/3266731390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 msgr2=0x55735f5b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5b0d10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f59ac011010 tx=0x7f59ac032cf0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- 172.21.15.66:0/3266731390 shutdown_connections 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f5a50e0 0x55735f5a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f59c8d0 0x55735f5a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/3266731390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59bb60 0x55735f5b0d10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- 172.21.15.66:0/3266731390 >> 172.21.15.66:0/3266731390 conn(0x55735f562c90 msgr2=0x55735f59d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- 172.21.15.66:0/3266731390 shutdown_connections 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- 172.21.15.66:0/3266731390 wait complete. 2024-04-21T09:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- start start 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 Processor -- start 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f59f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1af9e0 con 0x55735f59c8d0 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b0e50 con 0x55735f59bb60 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b00b0 con 0x55735f5a50e0 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.137+0000 7f59c3dd9740 10 client.0 _ll_register_callbacks cb 0x7f59ac00bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35962/0 (socket says 172.21.15.66:35962) 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 -- 172.21.15.66:0/2001340550 learned_addr learned my addr 172.21.15.66:0/2001340550 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59bad9d640 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59bb59e640 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f59f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 -- 172.21.15.66:0/2001340550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 msgr2=0x55735f59f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f59f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 -- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 msgr2=0x55735f59eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 -- 172.21.15.66:0/2001340550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735f5b77e0 con 0x55735f59c8d0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59ba59c640 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59f090 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f59b00092a0 tx=0x7f59b00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4c9d722cb77a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f599eff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f599eff5640 1 -- 172.21.15.66:0/2001340550 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b002c7e0 con 0x55735f59c8d0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f599eff5640 1 -- 172.21.15.66:0/2001340550 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f59b002c980 con 0x55735f59c8d0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.138+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac0266b0 con 0x55735f59c8d0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f599eff5640 1 -- 172.21.15.66:0/2001340550 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f59b0058050 con 0x55735f59c8d0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 10 client.16224 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55735f61b430 con 0x55735f59c8d0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f599eff5640 1 -- 172.21.15.66:0/2001340550 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ac0266b0 con 0x55735f59c8d0 2024-04-21T09:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f599eff5640 1 -- 172.21.15.66:0/2001340550 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f59b00052b0 con 0x55735f59c8d0 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 10 client.16224 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 20 client.16224 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f61bc30 0x55735f61e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 4 client.16224 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55735f4a03d0 con 0x55735f61bc30 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.139+0000 7f59bad9d640 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f61bc30 0x55735f61e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.140+0000 7f599eff5640 10 client.16224 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.140+0000 7f59bad9d640 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f61bc30 0x55735f61e050 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e07911f6d74683e server_cookie=f6fba7622b699974 in_seq=0 out_seq=0 2024-04-21T09:09:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f599eff5640 1 -- 172.21.15.66:0/2001340550 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55735f4a03d0 con 0x55735f61bc30 2024-04-21T09:09:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f599eff5640 10 client.16224 handle_command_reply: tid=0 2024-04-21T09:09:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 client.16224 shutdown 2024-04-21T09:09:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 msgr2=0x55735f59f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59f090 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f59b00092a0 tx=0x7f59b00043c0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 shutdown_connections 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55735f61bc30 0x55735f61e050 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55735f5a50e0 0x55735f59f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55735f59c8d0 0x55735f59f090 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f599eff5640 0 client.16224 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 --2- 172.21.15.66:0/2001340550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55735f59bb60 0x55735f59eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 >> 172.21.15.66:0/2001340550 conn(0x55735f562c90 msgr2=0x55735f595040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 shutdown_connections 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 1 -- 172.21.15.66:0/2001340550 wait complete. 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.149+0000 7f59c3dd9740 20 client.16224 trim_cache size 0 max 16384 2024-04-21T09:09:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59a40a3f90 msgr2=0x7f59a40a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59a40a3f90 0x7f59a40a6450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005d00 tx=0x7f59ac00b410 comp rx=0 tx=0).stop 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc1035e0 msgr2=0x7f59bc1a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc1035e0 0x7f59bc1a68a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009fd0 tx=0x7f59b400b260 comp rx=0 tx=0).stop 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 shutdown_connections 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59a40a3f90 0x7f59a40a6450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59bc1035e0 0x7f59bc1a68a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc1061a0 0x7f59bc1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 --2- 172.21.15.66:0/1668841605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc101120 0x7f59bc102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 >> 172.21.15.66:0/1668841605 conn(0x7f59bc077fa0 msgr2=0x7f59bc104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 shutdown_connections 2024-04-21T09:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:38.150+0000 7f59c110e640 1 -- 172.21.15.66:0/1668841605 wait complete. 2024-04-21T09:09:39.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:39 smithi186 ceph-mon[31505]: pgmap v46: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-04-21T09:09:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:39 smithi083 ceph-mon[31413]: pgmap v46: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-04-21T09:09:39.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:39 smithi066 ceph-mon[27953]: pgmap v46: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-04-21T09:09:41.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:40 smithi186 ceph-mon[31505]: pgmap v47: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-04-21T09:09:41.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:40 smithi083 ceph-mon[31413]: pgmap v47: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-04-21T09:09:41.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:40 smithi066 ceph-mon[27953]: pgmap v47: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-04-21T09:09:43.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:43 smithi186 ceph-mon[31505]: pgmap v48: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-04-21T09:09:43.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:43 smithi083 ceph-mon[31413]: pgmap v48: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-04-21T09:09:43.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:43 smithi066 ceph-mon[27953]: pgmap v48: 129 pgs: 129 active+clean; 183 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-04-21T09:09:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:45 smithi186 ceph-mon[31505]: pgmap v49: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-21T09:09:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:45 smithi186 ceph-mon[31505]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-21T09:09:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:45 smithi083 ceph-mon[31413]: pgmap v49: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-21T09:09:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:45 smithi083 ceph-mon[31413]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-21T09:09:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:45 smithi066 ceph-mon[27953]: pgmap v49: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-21T09:09:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:45 smithi066 ceph-mon[27953]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-21T09:09:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:47 smithi186 ceph-mon[31505]: pgmap v50: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 18 op/s 2024-04-21T09:09:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:47 smithi083 ceph-mon[31413]: pgmap v50: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 18 op/s 2024-04-21T09:09:47.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:47 smithi066 ceph-mon[27953]: pgmap v50: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 18 op/s 2024-04-21T09:09:49.390 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:49 smithi186 ceph-mon[31505]: pgmap v51: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-21T09:09:49.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:49 smithi083 ceph-mon[31413]: pgmap v51: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-21T09:09:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:49 smithi066 ceph-mon[27953]: pgmap v51: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.243+0000 7fdfd8532640 1 -- 172.21.15.66:0/525829328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd0101a70 msgr2=0x7fdfd0105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.243+0000 7fdfd8532640 1 --2- 172.21.15.66:0/525829328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd0101a70 0x7fdfd0105ab0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009900 tx=0x7fdfc002bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.243+0000 7fdfd8532640 1 -- 172.21.15.66:0/525829328 shutdown_connections 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.243+0000 7fdfd8532640 1 --2- 172.21.15.66:0/525829328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd0106190 0x7fdfd010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.243+0000 7fdfd8532640 1 --2- 172.21.15.66:0/525829328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd0101a70 0x7fdfd0105ab0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.243+0000 7fdfd8532640 1 --2- 172.21.15.66:0/525829328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd01010c0 0x7fdfd01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.243+0000 7fdfd8532640 1 -- 172.21.15.66:0/525829328 >> 172.21.15.66:0/525829328 conn(0x7fdfd0077f70 msgr2=0x7fdfd0078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 -- 172.21.15.66:0/525829328 shutdown_connections 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 -- 172.21.15.66:0/525829328 wait complete. 2024-04-21T09:09:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 Processor -- start 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 -- start start 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd01010c0 0x7fdfd01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd0106190 0x7fdfd0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd019cfa0 0x7fdfd019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01110d0 con 0x7fdfd01010c0 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0110f50 con 0x7fdfd019cfa0 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd8532640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0111250 con 0x7fdfd0106190 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.244+0000 7fdfd62ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd01010c0 0x7fdfd01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd01010c0 0x7fdfd01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37362/0 (socket says 172.21.15.66:37362) 2024-04-21T09:09:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 -- 172.21.15.66:0/482246017 learned_addr learned my addr 172.21.15.66:0/482246017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd019cfa0 0x7fdfd019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd0106190 0x7fdfd0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 -- 172.21.15.66:0/482246017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd0106190 msgr2=0x7fdfd0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd0106190 0x7fdfd0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 -- 172.21.15.66:0/482246017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd019cfa0 msgr2=0x7fdfd019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd019cfa0 0x7fdfd019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 -- 172.21.15.66:0/482246017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd019da90 con 0x7fdfd01010c0 2024-04-21T09:09:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd0106190 0x7fdfd0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd01010c0 0x7fdfd01986e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfc77fe640 1 -- 172.21.15.66:0/482246017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc017030 con 0x7fdfd01010c0 2024-04-21T09:09:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd019dd80 con 0x7fdfd01010c0 2024-04-21T09:09:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfc77fe640 1 -- 172.21.15.66:0/482246017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc004020 con 0x7fdfd01010c0 2024-04-21T09:09:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfc77fe640 1 -- 172.21.15.66:0/482246017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0047a0 con 0x7fdfd01010c0 2024-04-21T09:09:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.245+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd01a5570 con 0x7fdfd01010c0 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.248+0000 7fdfdb203740 1 Processor -- start 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.248+0000 7fdfc77fe640 1 -- 172.21.15.66:0/482246017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdfcc009a30 con 0x7fdfd01010c0 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.248+0000 7fdfdb203740 1 -- start start 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec809498d0 0x55ec80948620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec80948b60 0x55ec80948fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec809544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfdb203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ec808ad8d0 con 0x55ec80948b60 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfdb203740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec8084ae80 con 0x55ec809498d0 2024-04-21T09:09:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfdb203740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ec808a9890 con 0x55ec809520e0 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd62ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec80948b60 0x55ec80948fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd62ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec80948b60 0x55ec80948fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37372/0 (socket says 172.21.15.66:37372) 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd62ce640 1 -- 172.21.15.66:0/4147169687 learned_addr learned my addr 172.21.15.66:0/4147169687 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec809544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec809498d0 0x55ec80948620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfc77fe640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfbc0a3dc0 0x7fdfbc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd5acd640 1 -- 172.21.15.66:0/4147169687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec809498d0 msgr2=0x55ec80948620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec809498d0 0x55ec80948620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd5acd640 1 -- 172.21.15.66:0/4147169687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec80948b60 msgr2=0x55ec80948fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec80948b60 0x55ec80948fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd5acd640 1 -- 172.21.15.66:0/4147169687 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8086c320 con 0x55ec809520e0 2024-04-21T09:09:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec80948b60 0x55ec80948fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfbc0a3dc0 0x7fdfbc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.249+0000 7fdfc77fe640 1 -- 172.21.15.66:0/482246017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdfcc0d0680 con 0x7fdfd01010c0 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.250+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec809544d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0100cf0 tx=0x7fdfc002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9e9cce1c19cbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.250+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfbc0a3dc0 0x7fdfbc0a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8005fd0 tx=0x7fdfc8004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.250+0000 7fdfc4bfe640 1 -- 172.21.15.66:0/4147169687 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0043020 con 0x55ec809520e0 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.250+0000 7fdfc4bfe640 1 -- 172.21.15.66:0/4147169687 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc002c7e0 con 0x55ec809520e0 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.250+0000 7fdfc4bfe640 1 -- 172.21.15.66:0/4147169687 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0005560 con 0x55ec809520e0 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.250+0000 7fdfdb203740 1 -- 172.21.15.66:0/4147169687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 msgr2=0x55ec809544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec809544d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0100cf0 tx=0x7fdfc002c650 comp rx=0 tx=0).stop 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- 172.21.15.66:0/4147169687 shutdown_connections 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec809544d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec80948b60 0x55ec80948fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 --2- 172.21.15.66:0/4147169687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec809498d0 0x55ec80948620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- 172.21.15.66:0/4147169687 >> 172.21.15.66:0/4147169687 conn(0x55ec8090fc90 msgr2=0x55ec80943850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- 172.21.15.66:0/4147169687 shutdown_connections 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- 172.21.15.66:0/4147169687 wait complete. 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- start start 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 Processor -- start 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec809519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809498d0 0x55ec8094ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809520e0 0x55ec8094f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ec80954db0 con 0x55ec809520e0 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec80835eb0 con 0x55ec80948b60 2024-04-21T09:09:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ec80954f30 con 0x55ec809498d0 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfdb203740 10 client.0 _ll_register_callbacks cb 0x55ec80862b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.251+0000 7fdfd6acf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809520e0 0x55ec8094f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809498d0 0x55ec8094ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd62ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec809519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809498d0 0x55ec8094ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:48350/0 (socket says 172.21.15.66:48350) 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 -- 172.21.15.66:0/2924960577 learned_addr learned my addr 172.21.15.66:0/2924960577 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd6acf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809520e0 0x55ec8094f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37378/0 (socket says 172.21.15.66:37378) 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 -- 172.21.15.66:0/2924960577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 msgr2=0x55ec809519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec809519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 -- 172.21.15.66:0/2924960577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809520e0 msgr2=0x55ec8094f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809520e0 0x55ec8094f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 -- 172.21.15.66:0/2924960577 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec805c0210 con 0x55ec809498d0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809520e0 0x55ec8094f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809498d0 0x55ec8094ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0036040 tx=0x7fdfc002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e970309d6352679d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/2924960577 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc004c070 con 0x55ec809498d0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc002c7e0 con 0x55ec809498d0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.252+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/2924960577 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdfc00035f0 con 0x55ec809498d0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/2924960577 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfc0058050 con 0x55ec809498d0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 10 client.25720 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec8068c620 con 0x55ec809498d0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/2924960577 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc002c7e0 con 0x55ec809498d0 2024-04-21T09:09:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/2924960577 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdfc00048c0 con 0x55ec809498d0 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 10 client.25720 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 20 client.25720 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec80961ee0 0x55ec809642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 4 client.25720 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec808c1880 con 0x55ec80961ee0 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.253+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec80961ee0 0x55ec809642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfb9ffb640 10 client.25720 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec80961ee0 0x55ec809642d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b152731e72c0a565 server_cookie=55c1ee0306dbb46e in_seq=0 out_seq=0 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/2924960577 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec808c1880 con 0x55ec80961ee0 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfb9ffb640 10 client.25720 handle_command_reply: tid=0 2024-04-21T09:09:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfdb203740 1 client.25720 shutdown 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809498d0 msgr2=0x55ec8094ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfdb203740 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809498d0 0x55ec8094ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0036040 tx=0x7fdfc002bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 shutdown_connections 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.254+0000 7fdfdb203740 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec80961ee0 0x55ec809642d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfdb203740 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809520e0 0x55ec8094f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfb9ffb640 0 client.25720 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfdb203740 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809498d0 0x55ec8094ede0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfdb203740 1 --2- 172.21.15.66:0/2924960577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec809519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 >> 172.21.15.66:0/2924960577 conn(0x55ec8090fc90 msgr2=0x55ec809455a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 shutdown_connections 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfdb203740 1 -- 172.21.15.66:0/2924960577 wait complete. 2024-04-21T09:09:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.255+0000 7fdfdb203740 20 client.25720 trim_cache size 0 max 16384 2024-04-21T09:09:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 Processor -- start 2024-04-21T09:09:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 -- start start 2024-04-21T09:09:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8095dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec80969980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec80969ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b1bc0 con 0x55ec809498d0 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01af490 con 0x55ec80948b60 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfdb203740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ad2c0 con 0x55ec809520e0 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec80969980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec80969980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37384/0 (socket says 172.21.15.66:37384) 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 -- 172.21.15.66:0/1938357485 learned_addr learned my addr 172.21.15.66:0/1938357485 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8095dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec80969ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 -- 172.21.15.66:0/1938357485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 msgr2=0x55ec80969ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec80969ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 -- 172.21.15.66:0/1938357485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 msgr2=0x55ec8095dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8095dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.270+0000 7fdfd5acd640 1 -- 172.21.15.66:0/1938357485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8094f6b0 con 0x55ec809498d0 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.271+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec80969980 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0004ce0 tx=0x7fdfc002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ffd83b339c5076 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.271+0000 7fdfbbfff640 1 -- 172.21.15.66:0/1938357485 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0055030 con 0x55ec809498d0 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.271+0000 7fdfbbfff640 1 -- 172.21.15.66:0/1938357485 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc002c7e0 con 0x55ec809498d0 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.271+0000 7fdfbbfff640 1 -- 172.21.15.66:0/1938357485 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc004c040 con 0x55ec809498d0 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.271+0000 7fdfdb203740 1 -- 172.21.15.66:0/1938357485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 msgr2=0x55ec80969980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.271+0000 7fdfdb203740 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec80969980 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0004ce0 tx=0x7fdfc002c650 comp rx=0 tx=0).stop 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- 172.21.15.66:0/1938357485 shutdown_connections 2024-04-21T09:09:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec80969ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec80969980 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 --2- 172.21.15.66:0/1938357485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8095dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- 172.21.15.66:0/1938357485 >> 172.21.15.66:0/1938357485 conn(0x55ec8090fc90 msgr2=0x55ec8094d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- 172.21.15.66:0/1938357485 shutdown_connections 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- 172.21.15.66:0/1938357485 wait complete. 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- start start 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 Processor -- start 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8094bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec8094c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec8094c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ad950 con 0x55ec809498d0 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01aedc0 con 0x55ec80948b60 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ae020 con 0x55ec809520e0 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfdb203740 10 client.0 _ll_register_callbacks cb 0x7fdfcc0d6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfd5acd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec8094c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfd5acd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec8094c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37400/0 (socket says 172.21.15.66:37400) 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.272+0000 7fdfd5acd640 1 -- 172.21.15.66:0/3111590753 learned_addr learned my addr 172.21.15.66:0/3111590753 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec8094c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8094bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd5acd640 1 -- 172.21.15.66:0/3111590753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 msgr2=0x55ec8094c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec8094c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd5acd640 1 -- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 msgr2=0x55ec8094bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8094bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd5acd640 1 -- 172.21.15.66:0/3111590753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec809cb650 con 0x55ec809498d0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd6acf640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec8094c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfd5acd640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec8094c0e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00057c0 tx=0x7fdfc0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdac5e4008451e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/3111590753 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0055070 con 0x55ec809498d0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/3111590753 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdfc0048900 con 0x55ec809498d0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc002c7e0 con 0x55ec809498d0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/3111590753 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfc004c040 con 0x55ec809498d0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfdb203740 10 client.16242 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec809cc960 con 0x55ec809498d0 2024-04-21T09:09:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.273+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/3111590753 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc002c7e0 con 0x55ec809498d0 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/3111590753 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdfc005b070 con 0x55ec809498d0 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfdb203740 10 client.16242 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfdb203740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfdb203740 20 client.16242 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfdb203740 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec809ccee0 0x55ec809cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfdb203740 4 client.16242 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec8084d3d0 con 0x55ec809ccee0 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec809ccee0 0x55ec809cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfb9ffb640 10 client.16242 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.274+0000 7fdfd62ce640 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec809ccee0 0x55ec809cf300 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e17dc13e88540e5 server_cookie=75c96aa41dcc8078 in_seq=0 out_seq=0 2024-04-21T09:09:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfb9ffb640 1 -- 172.21.15.66:0/3111590753 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec8084d3d0 con 0x55ec809ccee0 2024-04-21T09:09:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfb9ffb640 10 client.16242 handle_command_reply: tid=0 2024-04-21T09:09:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 client.16242 shutdown 2024-04-21T09:09:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 msgr2=0x55ec8094c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec8094c0e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00057c0 tx=0x7fdfc0004d10 comp rx=0 tx=0).stop 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 shutdown_connections 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ec809ccee0 0x55ec809cf300 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec809520e0 0x55ec8094c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec809498d0 0x55ec8094c0e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfb9ffb640 0 client.16242 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 --2- 172.21.15.66:0/3111590753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec80948b60 0x55ec8094bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 >> 172.21.15.66:0/3111590753 conn(0x55ec8090fc90 msgr2=0x55ec80942040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 shutdown_connections 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 1 -- 172.21.15.66:0/3111590753 wait complete. 2024-04-21T09:09:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.286+0000 7fdfdb203740 20 client.16242 trim_cache size 0 max 16384 2024-04-21T09:09:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:09:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:09:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:09:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:09:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfbc0a3dc0 msgr2=0x7fdfbc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfbc0a3dc0 0x7fdfbc0a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8005fd0 tx=0x7fdfc8004a40 comp rx=0 tx=0).stop 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd01010c0 msgr2=0x7fdfd01986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd01010c0 0x7fdfd01986e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00de10 comp rx=0 tx=0).stop 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 shutdown_connections 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdfbc0a3dc0 0x7fdfbc0a6280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfd019cfa0 0x7fdfd019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfd0106190 0x7fdfd0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 --2- 172.21.15.66:0/482246017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfd01010c0 0x7fdfd01986e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 >> 172.21.15.66:0/482246017 conn(0x7fdfd0077f70 msgr2=0x7fdfd006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 shutdown_connections 2024-04-21T09:09:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:09:51.287+0000 7fdfd8532640 1 -- 172.21.15.66:0/482246017 wait complete. 2024-04-21T09:09:51.412 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:51 smithi083 ceph-mon[31413]: pgmap v52: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-21T09:09:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:51 smithi186 ceph-mon[31505]: pgmap v52: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-21T09:09:51.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:51 smithi066 ceph-mon[27953]: pgmap v52: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-21T09:09:52.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:09:52.804 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:09:52.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1316198546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf74102b50 msgr2=0x7fdf74106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1316198546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf74102b50 0x7fdf74106b90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009900 tx=0x7fdf6402bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1316198546 shutdown_connections 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1316198546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf74107270 0x7fdf7410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1316198546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf74102b50 0x7fdf74106b90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1316198546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf741021a0 0x7fdf74102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1316198546 >> 172.21.15.66:0/1316198546 conn(0x7fdf740793e0 msgr2=0x7fdf740797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1316198546 shutdown_connections 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.915+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1316198546 wait complete. 2024-04-21T09:09:52.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.916+0000 7fdf7bceb640 1 Processor -- start 2024-04-21T09:09:52.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.916+0000 7fdf7bceb640 1 -- start start 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7bceb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf741021a0 0x7fdf74198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7bceb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf74107270 0x7fdf74198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7bceb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7419cfd0 0x7fdf7419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7bceb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74112190 con 0x7fdf7419cfd0 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7bceb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74112010 con 0x7fdf74107270 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7bceb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74112310 con 0x7fdf741021a0 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7419cfd0 0x7fdf7419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7419cfd0 0x7fdf7419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37422/0 (socket says 172.21.15.66:37422) 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 -- 172.21.15.66:0/1957975987 learned_addr learned my addr 172.21.15.66:0/1957975987 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf79a87640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf741021a0 0x7fdf74198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf79286640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf74107270 0x7fdf74198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 -- 172.21.15.66:0/1957975987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf741021a0 msgr2=0x7fdf74198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf741021a0 0x7fdf74198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 -- 172.21.15.66:0/1957975987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf74107270 msgr2=0x7fdf74198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf74107270 0x7fdf74198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 -- 172.21.15.66:0/1957975987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7419dac0 con 0x7fdf7419cfd0 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf79a87640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf741021a0 0x7fdf74198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf7a288640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7419cfd0 0x7fdf7419d480 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00eb10 tx=0x7fdf6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.917+0000 7fdf6affd640 1 -- 172.21.15.66:0/1957975987 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c004b20 con 0x7fdf7419cfd0 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.918+0000 7fdf6affd640 1 -- 172.21.15.66:0/1957975987 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf6c004cc0 con 0x7fdf7419cfd0 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.918+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7419ddb0 con 0x7fdf7419cfd0 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.918+0000 7fdf6affd640 1 -- 172.21.15.66:0/1957975987 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c010670 con 0x7fdf7419cfd0 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.918+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a5520 con 0x7fdf7419cfd0 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.920+0000 7fdf6affd640 1 -- 172.21.15.66:0/1957975987 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdf6c005020 con 0x7fdf7419cfd0 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.920+0000 7fdf7e9bc740 1 Processor -- start 2024-04-21T09:09:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf7e9bc740 1 -- start start 2024-04-21T09:09:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733e580 0x5654d733e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733eea0 0x5654d7347540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d7349e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5654d72a31e0 con 0x5654d733eea0 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654d7240390 con 0x5654d7347a80 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5654d729f1a0 con 0x5654d733e580 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733eea0 0x5654d7347540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733eea0 0x5654d7347540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37424/0 (socket says 172.21.15.66:37424) 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 -- 172.21.15.66:0/1914144558 learned_addr learned my addr 172.21.15.66:0/1914144558 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79286640 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733e580 0x5654d733e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79286640 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d7349e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 -- 172.21.15.66:0/1914144558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733e580 msgr2=0x5654d733e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733e580 0x5654d733e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 -- 172.21.15.66:0/1914144558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 msgr2=0x5654d7349e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d7349e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf79a87640 1 -- 172.21.15.66:0/1914144558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d707f7d0 con 0x5654d733eea0 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.921+0000 7fdf6affd640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdf600a3d20 0x7fdf600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.922+0000 7fdf79a87640 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733eea0 0x5654d7347540 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdf700115f0 tx=0x7fdf70011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb54164e2dbc7221 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.922+0000 7fdf79286640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdf600a3d20 0x7fdf600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.922+0000 7fdf6affd640 1 -- 172.21.15.66:0/1957975987 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdf6c0d1070 con 0x7fdf7419cfd0 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.922+0000 7fdf5ffff640 1 -- 172.21.15.66:0/1914144558 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7001e070 con 0x5654d733eea0 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.922+0000 7fdf5ffff640 1 -- 172.21.15.66:0/1914144558 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf7000faf0 con 0x5654d733eea0 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.922+0000 7fdf5ffff640 1 -- 172.21.15.66:0/1914144558 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70019ac0 con 0x5654d733eea0 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/1914144558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733eea0 msgr2=0x5654d7347540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf79286640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdf600a3d20 0x7fdf600a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009230 tx=0x7fdf64005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733eea0 0x5654d7347540 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdf700115f0 tx=0x7fdf70011ab0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/1914144558 shutdown_connections 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d7349e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733eea0 0x5654d7347540 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/1914144558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733e580 0x5654d733e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/1914144558 >> 172.21.15.66:0/1914144558 conn(0x5654d7338cb0 msgr2=0x5654d7339090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/1914144558 shutdown_connections 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/1914144558 wait complete. 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 -- start start 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.923+0000 7fdf7e9bc740 1 Processor -- start 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7344ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733eea0 0x5654d7345000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d7347a80 0x5654d7345540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5654d734a870 con 0x5654d733e580 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654d725c2f0 con 0x5654d733eea0 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5654d734a9f0 con 0x5654d7347a80 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7344ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7e9bc740 10 client.0 _ll_register_callbacks cb 0x5654d7269580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7344ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37436/0 (socket says 172.21.15.66:37436) 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 -- 172.21.15.66:0/2070708584 learned_addr learned my addr 172.21.15.66:0/2070708584 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7a288640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d7347a80 0x5654d7345540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79286640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733eea0 0x5654d7345000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 -- 172.21.15.66:0/2070708584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d7347a80 msgr2=0x5654d7345540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d7347a80 0x5654d7345540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 -- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733eea0 msgr2=0x5654d7345000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733eea0 0x5654d7345000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 -- 172.21.15.66:0/2070708584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d6fb53f0 con 0x5654d733e580 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79286640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733eea0 0x5654d7345000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf7a288640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d7347a80 0x5654d7345540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf79a87640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7344ac0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdf7001c040 tx=0x7fdf7000f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b4d404c6c54344 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.924+0000 7fdf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/2070708584 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70029060 con 0x5654d733e580 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/2070708584 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdf70004040 con 0x5654d733e580 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf7000faf0 con 0x5654d733e580 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/2070708584 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf70030050 con 0x5654d733e580 2024-04-21T09:09:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 10 client.16254 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654d6f750f0 con 0x5654d733e580 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/2070708584 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf7000faf0 con 0x5654d733e580 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/2070708584 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdf700035f0 con 0x5654d733e580 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 10 client.16254 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 20 client.16254 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d7357a10 0x5654d7359e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 4 client.16254 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.925+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d72b0c00 con 0x5654d7357a10 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.926+0000 7fdf79286640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d7357a10 0x5654d7359e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.926+0000 7fdf5d7fa640 10 client.16254 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.926+0000 7fdf79286640 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d7357a10 0x5654d7359e00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14c32413dc0ce7d server_cookie=b168da1f6fefeec6 in_seq=0 out_seq=0 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/2070708584 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654d72b0c00 con 0x5654d7357a10 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf5d7fa640 10 client.16254 handle_command_reply: tid=0 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 client.16254 shutdown 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 msgr2=0x5654d7344ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7344ac0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdf7001c040 tx=0x7fdf7000f8f0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 shutdown_connections 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d7357a10 0x5654d7359e00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d7347a80 0x5654d7345540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733eea0 0x5654d7345000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/2070708584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7344ac0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf5d7fa640 0 client.16254 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 >> 172.21.15.66:0/2070708584 conn(0x5654d7338cb0 msgr2=0x5654d733aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 shutdown_connections 2024-04-21T09:09:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/2070708584 wait complete. 2024-04-21T09:09:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.927+0000 7fdf7e9bc740 20 client.16254 trim_cache size 0 max 16384 2024-04-21T09:09:52.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 Processor -- start 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 -- start start 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7353730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d73443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d735f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741b1b60 con 0x5654d733e580 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741af430 con 0x5654d7347a80 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741ad260 con 0x5654d733eea0 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.945+0000 7fdf79a87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7353730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7353730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37442/0 (socket says 172.21.15.66:37442) 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 -- 172.21.15.66:0/115908483 learned_addr learned my addr 172.21.15.66:0/115908483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79286640 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d73443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf7a288640 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d735f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 -- 172.21.15.66:0/115908483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 msgr2=0x5654d73443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d73443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 -- 172.21.15.66:0/115908483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 msgr2=0x5654d735f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d735f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 -- 172.21.15.66:0/115908483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d7343010 con 0x5654d733e580 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf7a288640 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d735f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf79a87640 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7353730 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009250 tx=0x7fdf700114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23626553144b6a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf5f7fe640 1 -- 172.21.15.66:0/115908483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70016070 con 0x5654d733e580 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf5f7fe640 1 -- 172.21.15.66:0/115908483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf70019410 con 0x5654d733e580 2024-04-21T09:09:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.946+0000 7fdf5f7fe640 1 -- 172.21.15.66:0/115908483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70029070 con 0x5654d733e580 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/115908483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 msgr2=0x5654d7353730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7353730 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009250 tx=0x7fdf700114c0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/115908483 shutdown_connections 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d7347a80 0x5654d735f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d73443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/115908483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d733e580 0x5654d7353730 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/115908483 >> 172.21.15.66:0/115908483 conn(0x5654d7338cb0 msgr2=0x5654d7337d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/115908483 shutdown_connections 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/115908483 wait complete. 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 -- start start 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.947+0000 7fdf7e9bc740 1 Processor -- start 2024-04-21T09:09:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733e580 0x5654d734ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d734b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7e9bc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d7347a80 0x5654d734b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741ad8f0 con 0x5654d7347a80 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741aed60 con 0x5654d733e580 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7e9bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741adfc0 con 0x5654d733eea0 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7e9bc740 10 client.0 _ll_register_callbacks cb 0x7fdf7001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d7347a80 0x5654d734b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d7347a80 0x5654d734b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37444/0 (socket says 172.21.15.66:37444) 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 -- 172.21.15.66:0/3955460087 learned_addr learned my addr 172.21.15.66:0/3955460087 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf79a87640 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733e580 0x5654d734ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf79286640 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d734b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 -- 172.21.15.66:0/3955460087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 msgr2=0x5654d734b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d734b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 -- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733e580 msgr2=0x5654d734ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733e580 0x5654d734ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.948+0000 7fdf7a288640 1 -- 172.21.15.66:0/3955460087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d734c010 con 0x5654d7347a80 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7a288640 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d7347a80 0x5654d734b830 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c09cc30 tx=0x7fdf6c09c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35bf851c8aa30d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/3955460087 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0b5070 con 0x5654d7347a80 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/3955460087 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdf6c0bd0b0 con 0x5654d7347a80 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf70019410 con 0x5654d7347a80 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/3955460087 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf6c0bbe60 con 0x5654d7347a80 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 10 client.16260 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654d73bdea0 con 0x5654d7347a80 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/3955460087 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf70019410 con 0x5654d7347a80 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/3955460087 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdf6c09cf00 con 0x5654d7347a80 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 10 client.16260 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 20 client.16260 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d73be5f0 0x5654d73c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 4 client.16260 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654d72aaa70 con 0x5654d73be5f0 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.949+0000 7fdf79a87640 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d73be5f0 0x5654d73c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.950+0000 7fdf5d7fa640 10 client.16260 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.950+0000 7fdf79a87640 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d73be5f0 0x5654d73c0a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e01cdfab900667 server_cookie=d9de7925d250e6f4 in_seq=0 out_seq=0 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.950+0000 7fdf5d7fa640 1 -- 172.21.15.66:0/3955460087 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5654d72aaa70 con 0x5654d73be5f0 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.950+0000 7fdf5d7fa640 10 client.16260 handle_command_reply: tid=0 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.950+0000 7fdf7e9bc740 1 client.16260 shutdown 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d7347a80 msgr2=0x5654d734b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d7347a80 0x5654d734b830 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c09cc30 tx=0x7fdf6c09c8a0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 shutdown_connections 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5654d73be5f0 0x5654d73c0a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654d7347a80 0x5654d734b830 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5654d733eea0 0x5654d734b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf5d7fa640 0 client.16260 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 --2- 172.21.15.66:0/3955460087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654d733e580 0x5654d734ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 >> 172.21.15.66:0/3955460087 conn(0x5654d7338cb0 msgr2=0x5654d73449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 shutdown_connections 2024-04-21T09:09:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 1 -- 172.21.15.66:0/3955460087 wait complete. 2024-04-21T09:09:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7e9bc740 20 client.16260 trim_cache size 0 max 16384 2024-04-21T09:09:52.962 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:09:52.962 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:09:52.962 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:09:52.962 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdf600a3d20 msgr2=0x7fdf600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdf600a3d20 0x7fdf600a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009230 tx=0x7fdf64005fd0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7419cfd0 msgr2=0x7fdf7419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7419cfd0 0x7fdf7419d480 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00eb10 tx=0x7fdf6c00c2d0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 shutdown_connections 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdf600a3d20 0x7fdf600a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7419cfd0 0x7fdf7419d480 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf74107270 0x7fdf74198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 --2- 172.21.15.66:0/1957975987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf741021a0 0x7fdf74198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.951+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 >> 172.21.15.66:0/1957975987 conn(0x7fdf740793e0 msgr2=0x7fdf74100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.952+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 shutdown_connections 2024-04-21T09:09:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:52.952+0000 7fdf7bceb640 1 -- 172.21.15.66:0/1957975987 wait complete. 2024-04-21T09:09:52.965 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:09:52.965 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:09:53.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 -- 172.21.15.66:0/2369898152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658107260 msgr2=0x7f065810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 --2- 172.21.15.66:0/2369898152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658107260 0x7f065810ee30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0650009ca0 tx=0x7f065002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 -- 172.21.15.66:0/2369898152 shutdown_connections 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 --2- 172.21.15.66:0/2369898152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658107260 0x7f065810ee30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 --2- 172.21.15.66:0/2369898152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0658102b40 0x7f0658106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 --2- 172.21.15.66:0/2369898152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0658102190 0x7f0658102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 -- 172.21.15.66:0/2369898152 >> 172.21.15.66:0/2369898152 conn(0x7f06580793e0 msgr2=0x7f06580797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 -- 172.21.15.66:0/2369898152 shutdown_connections 2024-04-21T09:09:53.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.072+0000 7f065cf23640 1 -- 172.21.15.66:0/2369898152 wait complete. 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 Processor -- start 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 -- start start 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0658102190 0x7f0658074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0658107260 0x7f0658074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658074cd0 0x7f06581a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06581121b0 con 0x7f0658074cd0 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0658112030 con 0x7f0658107260 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f065cf23640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0658112330 con 0x7f0658102190 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.073+0000 7f0656d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658074cd0 0x7f06581a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658074cd0 0x7f06581a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37470/0 (socket says 172.21.15.66:37470) 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 -- 172.21.15.66:0/2957716348 learned_addr learned my addr 172.21.15.66:0/2957716348 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0655d9b640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0658107260 0x7f0658074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f065659c640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0658102190 0x7f0658074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 -- 172.21.15.66:0/2957716348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0658102190 msgr2=0x7f0658074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0658102190 0x7f0658074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 -- 172.21.15.66:0/2957716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0658107260 msgr2=0x7f0658074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0658107260 0x7f0658074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 -- 172.21.15.66:0/2957716348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06581a6fc0 con 0x7f0658074cd0 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f0656d9d640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658074cd0 0x7f06581a68c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0650005e30 tx=0x7f0650034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f06477fe640 1 -- 172.21.15.66:0/2957716348 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650039040 con 0x7f0658074cd0 2024-04-21T09:09:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f06477fe640 1 -- 172.21.15.66:0/2957716348 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0650044070 con 0x7f0658074cd0 2024-04-21T09:09:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06581a7250 con 0x7f0658074cd0 2024-04-21T09:09:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f06477fe640 1 -- 172.21.15.66:0/2957716348 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650003e60 con 0x7f0658074cd0 2024-04-21T09:09:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.074+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581a7740 con 0x7f0658074cd0 2024-04-21T09:09:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.076+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0658069230 con 0x7f0658074cd0 2024-04-21T09:09:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.077+0000 7f06477fe640 1 -- 172.21.15.66:0/2957716348 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f06500324c0 con 0x7f0658074cd0 2024-04-21T09:09:53.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.077+0000 7f06477fe640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f06400a3d50 0x7f06400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.077+0000 7f065659c640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f06400a3d50 0x7f06400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.077+0000 7f06477fe640 1 -- 172.21.15.66:0/2957716348 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f06500f2cf0 con 0x7f0658074cd0 2024-04-21T09:09:53.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.078+0000 7f065659c640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f06400a3d50 0x7f06400a6210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0648005fd0 tx=0x7f064800b590 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.083+0000 7f06477fe640 1 -- 172.21.15.66:0/2957716348 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06500bf130 con 0x7f0658074cd0 2024-04-21T09:09:53.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.314+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f065810a490 con 0x7f0658074cd0 2024-04-21T09:09:53.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.314+0000 7f06477fe640 1 -- 172.21.15.66:0/2957716348 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f06500bea30 con 0x7f0658074cd0 2024-04-21T09:09:53.315 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:09:53.315 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:09:53.316 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:09:53.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f06400a3d50 msgr2=0x7f06400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f06400a3d50 0x7f06400a6210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0648005fd0 tx=0x7f064800b590 comp rx=0 tx=0).stop 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658074cd0 msgr2=0x7f06581a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658074cd0 0x7f06581a68c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0650005e30 tx=0x7f0650034760 comp rx=0 tx=0).stop 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 shutdown_connections 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f06400a3d50 0x7f06400a6210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0658074cd0 0x7f06581a68c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0658107260 0x7f0658074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 --2- 172.21.15.66:0/2957716348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0658102190 0x7f0658074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 >> 172.21.15.66:0/2957716348 conn(0x7f06580793e0 msgr2=0x7f06580fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 shutdown_connections 2024-04-21T09:09:53.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.317+0000 7f065cf23640 1 -- 172.21.15.66:0/2957716348 wait complete. 2024-04-21T09:09:53.329 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:09:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 -- 172.21.15.66:0/1861027506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f01021a0 msgr2=0x7f20f0102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 --2- 172.21.15.66:0/1861027506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f01021a0 0x7f20f0102580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009900 tx=0x7f20ec02bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 -- 172.21.15.66:0/1861027506 shutdown_connections 2024-04-21T09:09:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 --2- 172.21.15.66:0/1861027506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f0107270 0x7f20f010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 --2- 172.21.15.66:0/1861027506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f0102b50 0x7f20f0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 --2- 172.21.15.66:0/1861027506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f01021a0 0x7f20f0102580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 -- 172.21.15.66:0/1861027506 >> 172.21.15.66:0/1861027506 conn(0x7f20f0079350 msgr2=0x7f20f0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 -- 172.21.15.66:0/1861027506 shutdown_connections 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.438+0000 7f20f849a640 1 -- 172.21.15.66:0/1861027506 wait complete. 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 Processor -- start 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 -- start start 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f01021a0 0x7f20f0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f0107270 0x7f20f0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f0074cd0 0x7f20f01a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01121d0 con 0x7f20f0107270 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0112050 con 0x7f20f0074cd0 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f849a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0112350 con 0x7f20f01021a0 2024-04-21T09:09:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.439+0000 7f20f5a35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f0107270 0x7f20f0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f0107270 0x7f20f0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37494/0 (socket says 172.21.15.66:37494) 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 -- 172.21.15.66:0/2186424339 learned_addr learned my addr 172.21.15.66:0/2186424339 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f6236640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f01021a0 0x7f20f0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f6a37640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f0074cd0 0x7f20f01a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 -- 172.21.15.66:0/2186424339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f01021a0 msgr2=0x7f20f0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f01021a0 0x7f20f0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 -- 172.21.15.66:0/2186424339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f0074cd0 msgr2=0x7f20f01a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f0074cd0 0x7f20f01a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 -- 172.21.15.66:0/2186424339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f01a7080 con 0x7f20f0107270 2024-04-21T09:09:53.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f6236640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f01021a0 0x7f20f0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f5a35640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f0107270 0x7f20f0074790 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d740 tx=0x7f20e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20e77fe640 1 -- 172.21.15.66:0/2186424339 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0017070 con 0x7f20f0107270 2024-04-21T09:09:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20e77fe640 1 -- 172.21.15.66:0/2186424339 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e000de50 con 0x7f20f0107270 2024-04-21T09:09:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f01a7370 con 0x7f20f0107270 2024-04-21T09:09:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20e77fe640 1 -- 172.21.15.66:0/2186424339 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0004ce0 con 0x7f20f0107270 2024-04-21T09:09:53.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.440+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a7870 con 0x7f20f0107270 2024-04-21T09:09:53.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.442+0000 7f20e77fe640 1 -- 172.21.15.66:0/2186424339 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f20e0004e80 con 0x7f20f0107270 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 Processor -- start 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 -- start start 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560de6e7f1f0 con 0x560de6f23a00 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560de6e1c4b0 con 0x560de6f1a4d0 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20fb16d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560de6e7b1b0 con 0x560de6f1ae80 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.443+0000 7f20e77fe640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f20dc0a3d50 0x7f20dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f6a37640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f20dc0a3d50 0x7f20dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f6236640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f6236640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f23430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:48476/0 (socket says 172.21.15.66:48476) 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f25e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37510/0 (socket says 172.21.15.66:37510) 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f6236640 1 -- 172.21.15.66:0/2032424377 learned_addr learned my addr 172.21.15.66:0/2032424377 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f6a37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20e77fe640 1 -- 172.21.15.66:0/2186424339 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f20e00d05a0 con 0x7f20f0107270 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 -- 172.21.15.66:0/2032424377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 msgr2=0x560de6f23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 --2- 172.21.15.66:0/2032424377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f23430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 -- 172.21.15.66:0/2032424377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 msgr2=0x560de6f1a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 --2- 172.21.15.66:0/2032424377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 -- 172.21.15.66:0/2032424377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560de6e0e1b0 con 0x560de6f23a00 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f6a37640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f20dc0a3d50 0x7f20dc0a6210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005c40 tx=0x7f20e8009c90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20f5a35640 1 --2- 172.21.15.66:0/2032424377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f25e50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f20e00cd980 tx=0x7f20e006ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0f2c9cc03955f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.444+0000 7f20e4bfe640 1 -- 172.21.15.66:0/2032424377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00d0030 con 0x560de6f23a00 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20e4bfe640 1 -- 172.21.15.66:0/2032424377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e009c8d0 con 0x560de6f23a00 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20e4bfe640 1 -- 172.21.15.66:0/2032424377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00cdc00 con 0x560de6f23a00 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 -- 172.21.15.66:0/2032424377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 msgr2=0x560de6f25e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 --2- 172.21.15.66:0/2032424377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f25e50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f20e00cd980 tx=0x7f20e006ecc0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 -- 172.21.15.66:0/2032424377 shutdown_connections 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 --2- 172.21.15.66:0/2032424377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f25e50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 --2- 172.21.15.66:0/2032424377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 --2- 172.21.15.66:0/2032424377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 -- 172.21.15.66:0/2032424377 >> 172.21.15.66:0/2032424377 conn(0x560de6f14ba0 msgr2=0x560de6f14f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 -- 172.21.15.66:0/2032424377 shutdown_connections 2024-04-21T09:09:53.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.445+0000 7f20fb16d740 1 -- 172.21.15.66:0/2032424377 wait complete. 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 -- start start 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 Processor -- start 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f23360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f20d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560de6f268b0 con 0x560de6f23a00 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560de6e38500 con 0x560de6f1a4d0 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560de6f26a30 con 0x560de6f1ae80 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20fb16d740 10 client.0 _ll_register_callbacks cb 0x560de6e06b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20f6a37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f20d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20f6a37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f20d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37524/0 (socket says 172.21.15.66:37524) 2024-04-21T09:09:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20f6a37640 1 -- 172.21.15.66:0/1426082596 learned_addr learned my addr 172.21.15.66:0/1426082596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20f6236640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f23360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.446+0000 7f20f5a35640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f6a37640 1 -- 172.21.15.66:0/1426082596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 msgr2=0x560de6f206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f6a37640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f6a37640 1 -- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 msgr2=0x560de6f23360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f6a37640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f23360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f6a37640 1 -- 172.21.15.66:0/1426082596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560de6b913f0 con 0x560de6f23a00 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f5a35640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f6236640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20f6a37640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f20d00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009870 tx=0x7f20e8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9715f99beb17619d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20d9ffb640 1 -- 172.21.15.66:0/1426082596 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8026070 con 0x560de6f23a00 2024-04-21T09:09:53.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20d9ffb640 1 -- 172.21.15.66:0/1426082596 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f20e8020430 con 0x560de6f23a00 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e009c8d0 con 0x560de6f23a00 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20d9ffb640 1 -- 172.21.15.66:0/1426082596 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20e8032050 con 0x560de6f23a00 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20fb16d740 10 client.16278 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560de6c5b7d0 con 0x560de6f23a00 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.447+0000 7f20d9ffb640 1 -- 172.21.15.66:0/1426082596 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e009c8d0 con 0x560de6f23a00 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20d9ffb640 1 -- 172.21.15.66:0/1426082596 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f20e801f070 con 0x560de6f23a00 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20fb16d740 10 client.16278 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20fb16d740 10 client.16278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20fb16d740 20 client.16278 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20fb16d740 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f33ae0 0x560de6f35f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20fb16d740 4 client.16278 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:53.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560de6e1e930 con 0x560de6f33ae0 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20f6236640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f33ae0 0x560de6f35f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20d9ffb640 10 client.16278 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.448+0000 7f20f6236640 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f33ae0 0x560de6f35f20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d3a596f0ea2aad6 server_cookie=d1ac5b774ea3fe9b in_seq=0 out_seq=0 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20d9ffb640 1 -- 172.21.15.66:0/1426082596 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560de6e1e930 con 0x560de6f33ae0 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20d9ffb640 10 client.16278 handle_command_reply: tid=0 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 client.16278 shutdown 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 msgr2=0x560de6f20d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f20d00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009870 tx=0x7f20e8005bd0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 shutdown_connections 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f33ae0 0x560de6f35f20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f20d00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20d9ffb640 0 client.16278 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 --2- 172.21.15.66:0/1426082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 >> 172.21.15.66:0/1426082596 conn(0x560de6f14ba0 msgr2=0x560de6f168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 shutdown_connections 2024-04-21T09:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 1 -- 172.21.15.66:0/1426082596 wait complete. 2024-04-21T09:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.449+0000 7f20fb16d740 20 client.16278 trim_cache size 0 max 16384 2024-04-21T09:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 Processor -- start 2024-04-21T09:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 -- start start 2024-04-21T09:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f2f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f13a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b3e50 con 0x560de6f23a00 2024-04-21T09:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b1720 con 0x560de6f1a4d0 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.463+0000 7f20fb16d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01af550 con 0x560de6f1ae80 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f22e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37540/0 (socket says 172.21.15.66:37540) 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 -- 172.21.15.66:0/3134740664 learned_addr learned my addr 172.21.15.66:0/3134740664 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6236640 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f2f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f5a35640 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f13a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 -- 172.21.15.66:0/3134740664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 msgr2=0x560de6f13a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f13a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 -- 172.21.15.66:0/3134740664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 msgr2=0x560de6f2f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f2f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 -- 172.21.15.66:0/3134740664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560de6f21330 con 0x560de6f23a00 2024-04-21T09:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20f6a37640 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f22e20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005f40 tx=0x7f20e8010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8ebd76ec396797 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20dbfff640 1 -- 172.21.15.66:0/3134740664 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8026040 con 0x560de6f23a00 2024-04-21T09:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20dbfff640 1 -- 172.21.15.66:0/3134740664 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e80095c0 con 0x560de6f23a00 2024-04-21T09:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.464+0000 7f20dbfff640 1 -- 172.21.15.66:0/3134740664 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e801e9d0 con 0x560de6f23a00 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 -- 172.21.15.66:0/3134740664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 msgr2=0x560de6f22e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f22e20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005f40 tx=0x7f20e8010f80 comp rx=0 tx=0).stop 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 -- 172.21.15.66:0/3134740664 shutdown_connections 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f22e20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f13a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 --2- 172.21.15.66:0/3134740664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f2f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 -- 172.21.15.66:0/3134740664 >> 172.21.15.66:0/3134740664 conn(0x560de6f14ba0 msgr2=0x560de6f13370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 -- 172.21.15.66:0/3134740664 shutdown_connections 2024-04-21T09:09:53.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 -- 172.21.15.66:0/3134740664 wait complete. 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.465+0000 7f20fb16d740 1 -- start start 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 1 Processor -- start 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f1dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01afbe0 con 0x560de6f23a00 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b1050 con 0x560de6f1a4d0 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b02b0 con 0x560de6f1ae80 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20fb16d740 10 client.0 _ll_register_callbacks cb 0x7f20e8012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37548/0 (socket says 172.21.15.66:37548) 2024-04-21T09:09:53.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 -- 172.21.15.66:0/183636836 learned_addr learned my addr 172.21.15.66:0/183636836 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f5a35640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f1dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6236640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 -- 172.21.15.66:0/183636836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 msgr2=0x560de6f1dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f1dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 -- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 msgr2=0x560de6f1d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 -- 172.21.15.66:0/183636836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560de6f995f0 con 0x560de6f23a00 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6236640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20f6a37640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f990b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009870 tx=0x7f20e801e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8074ff2c8067d7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.466+0000 7f20d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20d9ffb640 1 -- 172.21.15.66:0/183636836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8004be0 con 0x560de6f23a00 2024-04-21T09:09:53.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20d9ffb640 1 -- 172.21.15.66:0/183636836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f20e8004d80 con 0x560de6f23a00 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e80095c0 con 0x560de6f23a00 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20d9ffb640 1 -- 172.21.15.66:0/183636836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20e8047050 con 0x560de6f23a00 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 10 client.16284 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560de6f9a730 con 0x560de6f23a00 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20d9ffb640 1 -- 172.21.15.66:0/183636836 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e80095c0 con 0x560de6f23a00 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20d9ffb640 1 -- 172.21.15.66:0/183636836 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f20e8005450 con 0x560de6f23a00 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 10 client.16284 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 10 client.16284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 20 client.16284 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f9ad40 0x560de6f9d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 4 client.16284 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:09:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.467+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560de6e8cc10 con 0x560de6f9ad40 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20f6236640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f9ad40 0x560de6f9d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20f6236640 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f9ad40 0x560de6f9d130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e83c858419d4e482 server_cookie=7cfed6b18988ac44 in_seq=0 out_seq=0 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20d9ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20d9ffb640 1 -- 172.21.15.66:0/183636836 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560de6e8cc10 con 0x560de6f9ad40 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20d9ffb640 10 client.16284 handle_command_reply: tid=0 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20fb16d740 1 client.16284 shutdown 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 msgr2=0x560de6f990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20fb16d740 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f990b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009870 tx=0x7f20e801e4c0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 shutdown_connections 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.468+0000 7f20fb16d740 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560de6f9ad40 0x560de6f9d130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20fb16d740 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560de6f23a00 0x560de6f990b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20fb16d740 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560de6f1ae80 0x560de6f1dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20d9ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20fb16d740 1 --2- 172.21.15.66:0/183636836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560de6f1a4d0 0x560de6f1d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 >> 172.21.15.66:0/183636836 conn(0x560de6f14ba0 msgr2=0x560de6f1bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 shutdown_connections 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20fb16d740 1 -- 172.21.15.66:0/183636836 wait complete. 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20fb16d740 20 client.16284 trim_cache size 0 max 16384 2024-04-21T09:09:53.477 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:09:53.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f20dc0a3d50 msgr2=0x7f20dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f20dc0a3d50 0x7f20dc0a6210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005c40 tx=0x7f20e8009c90 comp rx=0 tx=0).stop 2024-04-21T09:09:53.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f0107270 msgr2=0x7f20f0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f0107270 0x7f20f0074790 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d740 tx=0x7f20e000dc00 comp rx=0 tx=0).stop 2024-04-21T09:09:53.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 shutdown_connections 2024-04-21T09:09:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f20dc0a3d50 0x7f20dc0a6210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f0074cd0 0x7f20f01a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f0107270 0x7f20f0074790 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 --2- 172.21.15.66:0/2186424339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20f01021a0 0x7f20f0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 >> 172.21.15.66:0/2186424339 conn(0x7f20f0079350 msgr2=0x7f20f0100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 shutdown_connections 2024-04-21T09:09:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.469+0000 7f20f849a640 1 -- 172.21.15.66:0/2186424339 wait complete. 2024-04-21T09:09:53.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:53 smithi066 ceph-mon[27953]: pgmap v53: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-21T09:09:53.481 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:09:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:53 smithi186 ceph-mon[31505]: pgmap v53: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-21T09:09:53.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 -- 172.21.15.66:0/1280205470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101ac0 msgr2=0x7f8a80105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/1280205470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101ac0 0x7f8a80105b00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009900 tx=0x7f8a6c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:53.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 -- 172.21.15.66:0/1280205470 shutdown_connections 2024-04-21T09:09:53.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/1280205470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a801061e0 0x7f8a8010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/1280205470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101ac0 0x7f8a80105b00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/1280205470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a80101110 0x7f8a801014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 -- 172.21.15.66:0/1280205470 >> 172.21.15.66:0/1280205470 conn(0x7f8a80077f80 msgr2=0x7f8a80078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 -- 172.21.15.66:0/1280205470 shutdown_connections 2024-04-21T09:09:53.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.590+0000 7f8a86ff4640 1 -- 172.21.15.66:0/1280205470 wait complete. 2024-04-21T09:09:53.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 Processor -- start 2024-04-21T09:09:53.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 -- start start 2024-04-21T09:09:53.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101110 0x7f8a8019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a801061e0 0x7f8a8019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a80198620 0x7f8a80198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80111100 con 0x7f8a80101110 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80110f80 con 0x7f8a801061e0 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a86ff4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80111280 con 0x7f8a80198620 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101110 0x7f8a8019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101110 0x7f8a8019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37560/0 (socket says 172.21.15.66:37560) 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 -- 172.21.15.66:0/218102759 learned_addr learned my addr 172.21.15.66:0/218102759 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a801061e0 0x7f8a8019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a85591640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a80198620 0x7f8a80198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 -- 172.21.15.66:0/218102759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a80198620 msgr2=0x7f8a80198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a80198620 0x7f8a80198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 -- 172.21.15.66:0/218102759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a801061e0 msgr2=0x7f8a8019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a801061e0 0x7f8a8019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a84d90640 1 -- 172.21.15.66:0/218102759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a80199140 con 0x7f8a80101110 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.591+0000 7f8a85591640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a80198620 0x7f8a80198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.592+0000 7f8a84d90640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101110 0x7f8a8019e420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ea60 tx=0x7f8a7400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.592+0000 7f8a7dffb640 1 -- 172.21.15.66:0/218102759 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400cde0 con 0x7f8a80101110 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.592+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a80199430 con 0x7f8a80101110 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.592+0000 7f8a7dffb640 1 -- 172.21.15.66:0/218102759 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a740035f0 con 0x7f8a80101110 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.592+0000 7f8a7dffb640 1 -- 172.21.15.66:0/218102759 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74010690 con 0x7f8a80101110 2024-04-21T09:09:53.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.592+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801a5680 con 0x7f8a80101110 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.594+0000 7f8a7dffb640 1 -- 172.21.15.66:0/218102759 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8a740037a0 con 0x7f8a80101110 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.594+0000 7f8a89bde740 1 Processor -- start 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a89bde740 1 -- start start 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a33b0 0x5643bc2a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2ace60 0x5643bc2ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a89bde740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5643bc2081f0 con 0x5643bc2a33b0 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a89bde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5643bc1a54b0 con 0x5643bc2ace60 2024-04-21T09:09:53.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a89bde740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5643bc2041b0 con 0x5643bc2a3d60 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a33b0 0x5643bc2a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a33b0 0x5643bc2a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37568/0 (socket says 172.21.15.66:37568) 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 -- 172.21.15.66:0/3761731908 learned_addr learned my addr 172.21.15.66:0/3761731908 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a84d90640 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2ace60 0x5643bc2ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 -- 172.21.15.66:0/3761731908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 msgr2=0x5643bc2ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 -- 172.21.15.66:0/3761731908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2ace60 msgr2=0x5643bc2ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2ace60 0x5643bc2ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 -- 172.21.15.66:0/3761731908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643bc1971b0 con 0x5643bc2a33b0 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a85591640 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a33b0 0x5643bc2a3790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009fd0 tx=0x7f8a78010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d34b384091a286b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.595+0000 7f8a7dffb640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a700a3d20 0x7f8a700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.596+0000 7f8a84d90640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a700a3d20 0x7f8a700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.596+0000 7f8a6affd640 1 -- 172.21.15.66:0/3761731908 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78011700 con 0x5643bc2a33b0 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.596+0000 7f8a6affd640 1 -- 172.21.15.66:0/3761731908 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78011d50 con 0x5643bc2a33b0 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.596+0000 7f8a6affd640 1 -- 172.21.15.66:0/3761731908 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800e3c0 con 0x5643bc2a33b0 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.596+0000 7f8a7dffb640 1 -- 172.21.15.66:0/218102759 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8a740d0ea0 con 0x7f8a80101110 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 -- 172.21.15.66:0/3761731908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a33b0 msgr2=0x5643bc2a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a84d90640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a700a3d20 0x7f8a700a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5643bc2a41a0 tx=0x7f8a740bd040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a33b0 0x5643bc2a3790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009fd0 tx=0x7f8a78010860 comp rx=0 tx=0).stop 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 -- 172.21.15.66:0/3761731908 shutdown_connections 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2ace60 0x5643bc2ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3761731908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a33b0 0x5643bc2a3790 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 -- 172.21.15.66:0/3761731908 >> 172.21.15.66:0/3761731908 conn(0x5643bc29dba0 msgr2=0x5643bc29df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 -- 172.21.15.66:0/3761731908 shutdown_connections 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 -- 172.21.15.66:0/3761731908 wait complete. 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 -- start start 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.597+0000 7f8a89bde740 1 Processor -- start 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a33b0 0x5643bc2b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a3d60 0x5643bc2b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a89bde740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5643bc2b3da0 con 0x5643bc2ace60 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a89bde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5643bc1c1500 con 0x5643bc2a3d60 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a89bde740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5643bc2b3f20 con 0x5643bc2a33b0 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a89bde740 10 client.0 _ll_register_callbacks cb 0x5643bc18fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a3d60 0x5643bc2b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37582/0 (socket says 172.21.15.66:37582) 2024-04-21T09:09:53.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 -- 172.21.15.66:0/46379769 learned_addr learned my addr 172.21.15.66:0/46379769 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a33b0 0x5643bc2b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 -- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a33b0 msgr2=0x5643bc2b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a33b0 0x5643bc2b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 -- 172.21.15.66:0/46379769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a3d60 msgr2=0x5643bc2b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a3d60 0x5643bc2b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.598+0000 7f8a85591640 1 -- 172.21.15.66:0/46379769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643bbf1a3f0 con 0x5643bc2ace60 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a3d60 0x5643bc2b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a85591640 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b0b90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8a7801bb60 tx=0x7f8a7801ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239b10b0a45dcc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a5ffff640 1 -- 172.21.15.66:0/46379769 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78024040 con 0x5643bc2ace60 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a78011d50 con 0x5643bc2ace60 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a5ffff640 1 -- 172.21.15.66:0/46379769 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8a78011b30 con 0x5643bc2ace60 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a5ffff640 1 -- 172.21.15.66:0/46379769 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a78031050 con 0x5643bc2ace60 2024-04-21T09:09:53.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 10 client.16296 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643bbfe47d0 con 0x5643bc2ace60 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a5ffff640 1 -- 172.21.15.66:0/46379769 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78011d50 con 0x5643bc2ace60 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a5ffff640 1 -- 172.21.15.66:0/46379769 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a78005560 con 0x5643bc2ace60 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 10 client.16296 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 10 client.16296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 20 client.16296 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc2c0fe0 0x5643bc2c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 4 client.16296 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.599+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643bc1a7930 con 0x5643bc2c0fe0 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.600+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc2c0fe0 0x5643bc2c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.600+0000 7f8a5ffff640 10 client.16296 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:53.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.600+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc2c0fe0 0x5643bc2c33d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=179d141334ae2203 server_cookie=55a98171803dddc6 in_seq=0 out_seq=0 2024-04-21T09:09:53.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.601+0000 7f8a5ffff640 1 -- 172.21.15.66:0/46379769 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643bc1a7930 con 0x5643bc2c0fe0 2024-04-21T09:09:53.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.601+0000 7f8a5ffff640 10 client.16296 handle_command_reply: tid=0 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 client.16296 shutdown 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 msgr2=0x5643bc2b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b0b90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8a7801bb60 tx=0x7f8a7801ba60 comp rx=0 tx=0).stop 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 shutdown_connections 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc2c0fe0 0x5643bc2c33d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b0b90 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a3d60 0x5643bc2b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 --2- 172.21.15.66:0/46379769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a33b0 0x5643bc2b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 >> 172.21.15.66:0/46379769 conn(0x5643bc29dba0 msgr2=0x5643bc29f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a5ffff640 0 client.16296 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 shutdown_connections 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 1 -- 172.21.15.66:0/46379769 wait complete. 2024-04-21T09:09:53.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.602+0000 7f8a89bde740 20 client.16296 trim_cache size 0 max 16384 2024-04-21T09:09:53.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:53 smithi083 ceph-mon[31413]: pgmap v53: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-21T09:09:53.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.615+0000 7f8a89bde740 1 Processor -- start 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a89bde740 1 -- start start 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2ace60 0x5643bc2afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a89bde740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801afa20 con 0x5643bc2a3d60 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a89bde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ad2f0 con 0x5643bc2a33b0 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a89bde740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ab120 con 0x5643bc2ace60 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a85591640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2ace60 0x5643bc2afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37590/0 (socket says 172.21.15.66:37590) 2024-04-21T09:09:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 -- 172.21.15.66:0/3169056716 learned_addr learned my addr 172.21.15.66:0/3169056716 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a85591640 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 -- 172.21.15.66:0/3169056716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2ace60 msgr2=0x5643bc2afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2ace60 0x5643bc2afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 -- 172.21.15.66:0/3169056716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 msgr2=0x5643bc2bcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2bcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a7ffff640 1 -- 172.21.15.66:0/3169056716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643bc2c38c0 con 0x5643bc2a3d60 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.616+0000 7f8a85591640 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2ace60 0x5643bc2afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.617+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac660 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009900 tx=0x7f8a6c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17a6a2bf1bafe8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.617+0000 7f8a6a7fc640 1 -- 172.21.15.66:0/3169056716 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c04bbd0 con 0x5643bc2a3d60 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.617+0000 7f8a6a7fc640 1 -- 172.21.15.66:0/3169056716 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a6c02cc90 con 0x5643bc2a3d60 2024-04-21T09:09:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.617+0000 7f8a6a7fc640 1 -- 172.21.15.66:0/3169056716 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c005040 con 0x5643bc2a3d60 2024-04-21T09:09:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 -- 172.21.15.66:0/3169056716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a3d60 msgr2=0x5643bc2ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac660 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009900 tx=0x7f8a6c005fd0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 -- 172.21.15.66:0/3169056716 shutdown_connections 2024-04-21T09:09:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2ace60 0x5643bc2afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2a3d60 0x5643bc2ac660 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 --2- 172.21.15.66:0/3169056716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2bcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 -- 172.21.15.66:0/3169056716 >> 172.21.15.66:0/3169056716 conn(0x5643bc29dba0 msgr2=0x5643bc29ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 -- 172.21.15.66:0/3169056716 shutdown_connections 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 -- 172.21.15.66:0/3169056716 wait complete. 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 -- start start 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.618+0000 7f8a89bde740 1 Processor -- start 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a89bde740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a89bde740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ab7b0 con 0x5643bc2ace60 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a89bde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801acc20 con 0x5643bc2a33b0 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a89bde740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801abe80 con 0x5643bc2a3d60 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a89bde740 10 client.0 _ll_register_callbacks cb 0x7f8a78011500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37604/0 (socket says 172.21.15.66:37604) 2024-04-21T09:09:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 -- 172.21.15.66:0/1987199161 learned_addr learned my addr 172.21.15.66:0/1987199161 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 -- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 msgr2=0x5643bc2b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 -- 172.21.15.66:0/1987199161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 msgr2=0x5643bc2b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 -- 172.21.15.66:0/1987199161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643bc2b25f0 con 0x5643bc2ace60 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a85591640 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b20b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8a7801d010 tx=0x7f8a7801ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8958a65e1390c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.619+0000 7f8a5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a5ffff640 1 -- 172.21.15.66:0/1987199161 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78010d60 con 0x5643bc2ace60 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a5ffff640 1 -- 172.21.15.66:0/1987199161 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8a78011b40 con 0x5643bc2ace60 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a6c02cc90 con 0x5643bc2ace60 2024-04-21T09:09:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a5ffff640 1 -- 172.21.15.66:0/1987199161 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a78009bf0 con 0x5643bc2ace60 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 10 client.16302 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643bc3263c0 con 0x5643bc2ace60 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a5ffff640 1 -- 172.21.15.66:0/1987199161 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a7803f070 con 0x5643bc2ace60 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a5ffff640 1 -- 172.21.15.66:0/1987199161 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a78035430 con 0x5643bc2ace60 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 10 client.16302 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 10 client.16302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 20 client.16302 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc326a30 0x5643bc328e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 4 client.16302 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.620+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643bc215c10 con 0x5643bc326a30 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc326a30 0x5643bc328e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a7ffff640 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc326a30 0x5643bc328e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ff618b2abc825c6 server_cookie=19f2e201c9dcb6d9 in_seq=0 out_seq=0 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a5ffff640 10 client.16302 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a5ffff640 1 -- 172.21.15.66:0/1987199161 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643bc215c10 con 0x5643bc326a30 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a5ffff640 10 client.16302 handle_command_reply: tid=0 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a89bde740 1 client.16302 shutdown 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 msgr2=0x5643bc2b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.621+0000 7f8a89bde740 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b20b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8a7801d010 tx=0x7f8a7801ba60 comp rx=0 tx=0).stop 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 shutdown_connections 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5643bc326a30 0x5643bc328e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643bc2ace60 0x5643bc2b20b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5643bc2a3d60 0x5643bc2b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a5ffff640 0 client.16302 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 --2- 172.21.15.66:0/1987199161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643bc2a33b0 0x5643bc2b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 >> 172.21.15.66:0/1987199161 conn(0x5643bc29dba0 msgr2=0x5643bc29cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 shutdown_connections 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 1 -- 172.21.15.66:0/1987199161 wait complete. 2024-04-21T09:09:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.622+0000 7f8a89bde740 20 client.16302 trim_cache size 0 max 16384 2024-04-21T09:09:53.629 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:09:53.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a700a3d20 msgr2=0x7f8a700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a700a3d20 0x7f8a700a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5643bc2a41a0 tx=0x7f8a740bd040 comp rx=0 tx=0).stop 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101110 msgr2=0x7f8a8019e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101110 0x7f8a8019e420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ea60 tx=0x7f8a7400ef20 comp rx=0 tx=0).stop 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 shutdown_connections 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a700a3d20 0x7f8a700a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a80198620 0x7f8a80198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a801061e0 0x7f8a8019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 --2- 172.21.15.66:0/218102759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a80101110 0x7f8a8019e420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 >> 172.21.15.66:0/218102759 conn(0x7f8a80077f80 msgr2=0x7f8a80104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 shutdown_connections 2024-04-21T09:09:53.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.623+0000 7f8a86ff4640 1 -- 172.21.15.66:0/218102759 wait complete. 2024-04-21T09:09:53.640 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 -- 172.21.15.66:0/4278886806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 msgr2=0x7fe0ac101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/4278886806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac101470 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009900 tx=0x7fe0a002bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 -- 172.21.15.66:0/4278886806 shutdown_connections 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/4278886806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0ac106160 0x7fe0ac10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/4278886806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0ac101a40 0x7fe0ac105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/4278886806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac101470 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 -- 172.21.15.66:0/4278886806 >> 172.21.15.66:0/4278886806 conn(0x7fe0ac077f50 msgr2=0x7fe0ac078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 -- 172.21.15.66:0/4278886806 shutdown_connections 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 -- 172.21.15.66:0/4278886806 wait complete. 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.749+0000 7fe0b20f2640 1 Processor -- start 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0b20f2640 1 -- start start 2024-04-21T09:09:53.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0b20f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0b20f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0ac106160 0x7fe0ac198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0b20f2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0ac19cff0 0x7fe0ac19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0b20f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1110a0 con 0x7fe0ac101090 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0b20f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac110f20 con 0x7fe0ac106160 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0b20f2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac111220 con 0x7fe0ac19cff0 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37624/0 (socket says 172.21.15.66:37624) 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/3161878701 learned_addr learned my addr 172.21.15.66:0/3161878701 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0ac106160 0x7fe0ac198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0abfff640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0ac19cff0 0x7fe0ac19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/3161878701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0ac19cff0 msgr2=0x7fe0ac19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0ac19cff0 0x7fe0ac19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/3161878701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0ac106160 msgr2=0x7fe0ac198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0ac106160 0x7fe0ac198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/3161878701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac19dae0 con 0x7fe0ac101090 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0abfff640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0ac19cff0 0x7fe0ac19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0ac106160 0x7fe0ac198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.750+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac198730 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0005940 tx=0x7fe0a002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.751+0000 7fe0a8ff9640 1 -- 172.21.15.66:0/3161878701 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0043020 con 0x7fe0ac101090 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.751+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac19dd70 con 0x7fe0ac101090 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.751+0000 7fe0a8ff9640 1 -- 172.21.15.66:0/3161878701 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a002cbe0 con 0x7fe0ac101090 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.751+0000 7fe0a8ff9640 1 -- 172.21.15.66:0/3161878701 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0003bf0 con 0x7fe0ac101090 2024-04-21T09:09:53.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.751+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac1a55c0 con 0x7fe0ac101090 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 Processor -- start 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0a8ff9640 1 -- 172.21.15.66:0/3161878701 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe0a0046020 con 0x7fe0ac101090 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 -- start start 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570de9fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570de9fce80 0x5570dea05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570dea05a00 0x5570dea07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5570de9611f0 con 0x5570dea05a00 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570de8fe4b0 con 0x5570de9fc4d0 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.753+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5570de95d1b0 con 0x5570de9fce80 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570dea05a00 0x5570dea07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570dea05a00 0x5570dea07e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37630/0 (socket says 172.21.15.66:37630) 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 -- 172.21.15.66:0/2491445314 learned_addr learned my addr 172.21.15.66:0/2491445314 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0abfff640 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570de9fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570de9fce80 0x5570dea05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0a8ff9640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe0940a3d20 0x7fe0940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 -- 172.21.15.66:0/2491445314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570de9fce80 msgr2=0x5570dea05430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570de9fce80 0x5570dea05430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 -- 172.21.15.66:0/2491445314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 msgr2=0x5570de9fc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570de9fc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 -- 172.21.15.66:0/2491445314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570de8f01b0 con 0x5570dea05a00 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0abfff640 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570de9fc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0abfff640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe0940a3d20 0x7fe0940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0a8ff9640 1 -- 172.21.15.66:0/3161878701 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe0a00f2db0 con 0x7fe0ac101090 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570dea05a00 0x5570dea07e50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00cc40 tx=0x7fe09c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0820c5e91eeb10f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.754+0000 7fe091ffb640 1 -- 172.21.15.66:0/2491445314 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c014070 con 0x5570dea05a00 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe091ffb640 1 -- 172.21.15.66:0/2491445314 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09c0047a0 con 0x5570dea05a00 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe091ffb640 1 -- 172.21.15.66:0/2491445314 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c010670 con 0x5570dea05a00 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/2491445314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570dea05a00 msgr2=0x5570dea07e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0abfff640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe0940a3d20 0x7fe0940a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009370 tx=0x7fe0a4004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570dea05a00 0x5570dea07e50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00cc40 tx=0x7fe09c00e720 comp rx=0 tx=0).stop 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/2491445314 shutdown_connections 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570dea05a00 0x5570dea07e50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570de9fce80 0x5570dea05430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/2491445314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570de9fc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/2491445314 >> 172.21.15.66:0/2491445314 conn(0x5570de9f6ba0 msgr2=0x5570de9f6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/2491445314 shutdown_connections 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.755+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/2491445314 wait complete. 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 -- start start 2024-04-21T09:09:53.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 Processor -- start 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea05290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea02b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5570dea08970 con 0x5570de9fc4d0 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570de91a500 con 0x5570de9fce80 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5570dea08af0 con 0x5570dea05a00 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0b4dbf740 10 client.0 _ll_register_callbacks cb 0x5570de8e8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea05290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea05290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37636/0 (socket says 172.21.15.66:37636) 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.756+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1369667693 learned_addr learned my addr 172.21.15.66:0/1369667693 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0abfff640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea02b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1369667693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 msgr2=0x5570dea02b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea02b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 msgr2=0x5570dea02620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea02620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1369667693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570de6733f0 con 0x5570de9fc4d0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0abfff640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea02b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea05290 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe0a003e040 tx=0x7fe0a00be630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fec497f03ef320 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe08affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe08affd640 1 -- 172.21.15.66:0/1369667693 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00beaf0 con 0x5570de9fc4d0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe09c0047a0 con 0x5570de9fc4d0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe08affd640 1 -- 172.21.15.66:0/1369667693 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe0a0003d90 con 0x5570de9fc4d0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe08affd640 1 -- 172.21.15.66:0/1369667693 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0a00c2ea0 con 0x5570de9fc4d0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0b4dbf740 10 client.16314 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570de73d7d0 con 0x5570de9fc4d0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.757+0000 7fe08affd640 1 -- 172.21.15.66:0/1369667693 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09c0047a0 con 0x5570de9fc4d0 2024-04-21T09:09:53.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe08affd640 1 -- 172.21.15.66:0/1369667693 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe0a00da020 con 0x5570de9fc4d0 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe0b4dbf740 10 client.16314 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe0b4dbf740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe0b4dbf740 20 client.16314 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea15c00 0x5570dea18030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe0b4dbf740 4 client.16314 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570de900930 con 0x5570dea15c00 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.758+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea15c00 0x5570dea18030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.759+0000 7fe08affd640 10 client.16314 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.759+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea15c00 0x5570dea18030 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc8d8e837118537 server_cookie=495a1697cd9b6021 in_seq=0 out_seq=0 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.759+0000 7fe08affd640 1 -- 172.21.15.66:0/1369667693 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570de900930 con 0x5570dea15c00 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.759+0000 7fe08affd640 10 client.16314 handle_command_reply: tid=0 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.759+0000 7fe0b4dbf740 1 client.16314 shutdown 2024-04-21T09:09:53.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 msgr2=0x5570dea05290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea05290 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe0a003e040 tx=0x7fe0a00be630 comp rx=0 tx=0).stop 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 shutdown_connections 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea15c00 0x5570dea18030 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea02b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea02620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1369667693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea05290 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe08affd640 0 client.16314 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 >> 172.21.15.66:0/1369667693 conn(0x5570de9f6ba0 msgr2=0x5570de9f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 shutdown_connections 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1369667693 wait complete. 2024-04-21T09:09:53.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.760+0000 7fe0b4dbf740 20 client.16314 trim_cache size 0 max 16384 2024-04-21T09:09:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.773+0000 7fe0b4dbf740 1 Processor -- start 2024-04-21T09:09:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0b4dbf740 1 -- start start 2024-04-21T09:09:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea11830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea04d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea05290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1b1ae0 con 0x5570de9fc4d0 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1af3b0 con 0x5570de9fce80 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1ad1e0 con 0x5570dea05a00 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea11830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea11830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37640/0 (socket says 172.21.15.66:37640) 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1436548358 learned_addr learned my addr 172.21.15.66:0/1436548358 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0abfff640 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea05290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea04d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1436548358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 msgr2=0x5570dea05290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea05290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1436548358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 msgr2=0x5570dea04d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea04d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 -- 172.21.15.66:0/1436548358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570dea082b0 con 0x5570de9fc4d0 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0abfff640 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea05290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea04d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.774+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea11830 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00c7230 tx=0x7fe0a00be760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5ba94d9825608f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0917fa640 1 -- 172.21.15.66:0/1436548358 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00beaf0 con 0x5570de9fc4d0 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0917fa640 1 -- 172.21.15.66:0/1436548358 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a00c7760 con 0x5570de9fc4d0 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0917fa640 1 -- 172.21.15.66:0/1436548358 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00be450 con 0x5570de9fc4d0 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1436548358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 msgr2=0x5570dea11830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea11830 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00c7230 tx=0x7fe0a00be760 comp rx=0 tx=0).stop 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1436548358 shutdown_connections 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570dea05290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fce80 0x5570dea04d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1436548358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fc4d0 0x5570dea11830 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1436548358 >> 172.21.15.66:0/1436548358 conn(0x5570de9f6ba0 msgr2=0x5570de9f5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1436548358 shutdown_connections 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.775+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1436548358 wait complete. 2024-04-21T09:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 -- start start 2024-04-21T09:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 Processor -- start 2024-04-21T09:09:53.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570dea0dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fce80 0x5570de9fee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570de9ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1ad870 con 0x5570de9fce80 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1aece0 con 0x5570de9fc4d0 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1adf40 con 0x5570dea05a00 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0b4dbf740 10 client.0 _ll_register_callbacks cb 0x7fe0a00bad90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fce80 0x5570de9fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fce80 0x5570de9fee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37646/0 (socket says 172.21.15.66:37646) 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.776+0000 7fe0aaffd640 1 -- 172.21.15.66:0/1349736363 learned_addr learned my addr 172.21.15.66:0/1349736363 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0abfff640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570de9ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570dea0dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0aaffd640 1 -- 172.21.15.66:0/1349736363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 msgr2=0x5570de9ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570de9ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0aaffd640 1 -- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 msgr2=0x5570dea0dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570dea0dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0aaffd640 1 -- 172.21.15.66:0/1349736363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570de9ffa00 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0abfff640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570de9ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0aaffd640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fce80 0x5570de9fee50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00c750 tx=0x7fe09c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47723d2e9c55cad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe08affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe08affd640 1 -- 172.21.15.66:0/1349736363 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c014040 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe08affd640 1 -- 172.21.15.66:0/1349736363 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe09c0273d0 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a00c7760 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe08affd640 1 -- 172.21.15.66:0/1349736363 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe09c035050 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0b4dbf740 10 client.16320 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570dea7c020 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.777+0000 7fe08affd640 1 -- 172.21.15.66:0/1349736363 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a00c7760 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe08affd640 1 -- 172.21.15.66:0/1349736363 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe09c004ad0 con 0x5570de9fce80 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0b4dbf740 10 client.16320 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0b4dbf740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0b4dbf740 20 client.16320 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea7c690 0x5570dea7eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0b4dbf740 4 client.16320 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570de96ec10 con 0x5570dea7c690 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea7c690 0x5570dea7eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe0ab7fe640 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea7c690 0x5570dea7eab0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fefb598053bd33 server_cookie=a65d0e15b51dca8d in_seq=0 out_seq=0 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.778+0000 7fe08affd640 10 client.16320 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe08affd640 1 -- 172.21.15.66:0/1349736363 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570de96ec10 con 0x5570dea7c690 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe08affd640 10 client.16320 handle_command_reply: tid=0 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 client.16320 shutdown 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fce80 msgr2=0x5570de9fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fce80 0x5570de9fee50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00c750 tx=0x7fe09c005270 comp rx=0 tx=0).stop 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 shutdown_connections 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5570dea7c690 0x5570dea7eab0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570dea05a00 0x5570de9ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5570de9fce80 0x5570de9fee50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe08affd640 0 client.16320 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 --2- 172.21.15.66:0/1349736363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570de9fc4d0 0x5570dea0dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 >> 172.21.15.66:0/1349736363 conn(0x5570de9f6ba0 msgr2=0x5570de9fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 shutdown_connections 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 1 -- 172.21.15.66:0/1349736363 wait complete. 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.779+0000 7fe0b4dbf740 20 client.16320 trim_cache size 0 max 16384 2024-04-21T09:09:53.787 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:09:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe0940a3d20 msgr2=0x7fe0940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe0940a3d20 0x7fe0940a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009370 tx=0x7fe0a4004a40 comp rx=0 tx=0).stop 2024-04-21T09:09:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 msgr2=0x7fe0ac198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac198730 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0005940 tx=0x7fe0a002c7e0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 shutdown_connections 2024-04-21T09:09:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe0940a3d20 0x7fe0940a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0ac19cff0 0x7fe0ac19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0ac106160 0x7fe0ac198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 --2- 172.21.15.66:0/3161878701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0ac101090 0x7fe0ac198730 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 >> 172.21.15.66:0/3161878701 conn(0x7fe0ac077f50 msgr2=0x7fe0ac069520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 shutdown_connections 2024-04-21T09:09:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:53.780+0000 7fe0b20f2640 1 -- 172.21.15.66:0/3161878701 wait complete. 2024-04-21T09:09:54.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:54 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2957716348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:09:54.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:54 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2957716348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:09:54.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:54 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2957716348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:09:54.793 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:09:54.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:09:54.793 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:09:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.904+0000 7f8b54043640 1 -- 172.21.15.66:0/718876779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 msgr2=0x7f8b4c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.904+0000 7f8b54043640 1 --2- 172.21.15.66:0/718876779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c101130 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009900 tx=0x7f8b3c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:09:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 -- 172.21.15.66:0/718876779 shutdown_connections 2024-04-21T09:09:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 --2- 172.21.15.66:0/718876779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e900 0x7f8b4c110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 --2- 172.21.15.66:0/718876779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4c101670 0x7f8b4c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 --2- 172.21.15.66:0/718876779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c101130 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 -- 172.21.15.66:0/718876779 >> 172.21.15.66:0/718876779 conn(0x7f8b4c077f50 msgr2=0x7f8b4c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 -- 172.21.15.66:0/718876779 shutdown_connections 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 -- 172.21.15.66:0/718876779 wait complete. 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 Processor -- start 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.905+0000 7f8b54043640 1 -- start start 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b54043640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b54043640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c101670 0x7f8b4c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b54043640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4c10e900 0x7f8b4c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b54043640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c114420 con 0x7f8b4c100d50 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b54043640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1142a0 con 0x7f8b4c101670 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b54043640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1145a0 con 0x7f8b4c10e900 2024-04-21T09:09:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37200/0 (socket says 172.21.15.66:37200) 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2424180868 learned_addr learned my addr 172.21.15.66:0/2424180868 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b515de640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c101670 0x7f8b4c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b525e0640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4c10e900 0x7f8b4c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2424180868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4c10e900 msgr2=0x7f8b4c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4c10e900 0x7f8b4c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2424180868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c101670 msgr2=0x7f8b4c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c101670 0x7f8b4c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2424180868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c19afd0 con 0x7f8b4c100d50 2024-04-21T09:09:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b515de640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c101670 0x7f8b4c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:09:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c1a0640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000c00 tx=0x7f8b3c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.906+0000 7f8b42ffd640 1 -- 172.21.15.66:0/2424180868 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c043020 con 0x7f8b4c100d50 2024-04-21T09:09:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.907+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c19b260 con 0x7f8b4c100d50 2024-04-21T09:09:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.907+0000 7f8b42ffd640 1 -- 172.21.15.66:0/2424180868 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b3c02cbb0 con 0x7f8b4c100d50 2024-04-21T09:09:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.907+0000 7f8b42ffd640 1 -- 172.21.15.66:0/2424180868 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c003ac0 con 0x7f8b4c100d50 2024-04-21T09:09:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.907+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c1a7490 con 0x7f8b4c100d50 2024-04-21T09:09:54.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.909+0000 7f8b42ffd640 1 -- 172.21.15.66:0/2424180868 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8b3c046020 con 0x7f8b4c100d50 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.909+0000 7f8b56c3b740 1 Processor -- start 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b56c3b740 1 -- start start 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b42ffd640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b380a3d20 0x7f8b380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623f4e10 0x55a4623f7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a46234ff00 con 0x55a4623f4e10 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a4622ed590 con 0x55a4623ec290 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a46234bec0 con 0x55a4623eb8e0 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b42ffd640 1 -- 172.21.15.66:0/2424180868 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8b3c0f2c80 con 0x7f8b4c100d50 2024-04-21T09:09:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b525e0640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b380a3d20 0x7f8b380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b525e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623f4e10 0x55a4623f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b51ddf640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b525e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623f4e10 0x55a4623f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37208/0 (socket says 172.21.15.66:37208) 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b515de640 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b525e0640 1 -- 172.21.15.66:0/1861939300 learned_addr learned my addr 172.21.15.66:0/1861939300 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b51ddf640 1 -- 172.21.15.66:0/1861939300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 msgr2=0x55a4623f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b51ddf640 1 -- 172.21.15.66:0/1861939300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623f4e10 msgr2=0x55a4623f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623f4e10 0x55a4623f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.910+0000 7f8b51ddf640 1 -- 172.21.15.66:0/1861939300 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46212a7e0 con 0x55a4623eb8e0 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.911+0000 7f8b525e0640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b380a3d20 0x7f8b380a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b44012b00 tx=0x7f8b44009cd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.911+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623ebcc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0f0170 tx=0x7f8b3c0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5483c11ad9e80018 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.911+0000 7f8b37fff640 1 -- 172.21.15.66:0/1861939300 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0369b0 con 0x55a4623eb8e0 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.911+0000 7f8b37fff640 1 -- 172.21.15.66:0/1861939300 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b3c03c350 con 0x55a4623eb8e0 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.911+0000 7f8b37fff640 1 -- 172.21.15.66:0/1861939300 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0c7630 con 0x55a4623eb8e0 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 -- 172.21.15.66:0/1861939300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 msgr2=0x55a4623ebcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623ebcc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0f0170 tx=0x7f8b3c0035f0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 -- 172.21.15.66:0/1861939300 shutdown_connections 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623f4e10 0x55a4623f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/1861939300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623ebcc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 -- 172.21.15.66:0/1861939300 >> 172.21.15.66:0/1861939300 conn(0x55a4623e5fb0 msgr2=0x55a4623e6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 -- 172.21.15.66:0/1861939300 shutdown_connections 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 -- 172.21.15.66:0/1861939300 wait complete. 2024-04-21T09:09:54.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 -- start start 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.912+0000 7f8b56c3b740 1 Processor -- start 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a4623f7b40 con 0x55a4623eb8e0 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a462309580 con 0x55a4623ec290 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a4623f7cc0 con 0x55a4623f4e10 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 10 client.0 _ll_register_callbacks cb 0x55a46230f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37212/0 (socket says 172.21.15.66:37212) 2024-04-21T09:09:54.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2518749190 learned_addr learned my addr 172.21.15.66:0/2518749190 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b525e0640 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b515de640 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2518749190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 msgr2=0x55a4623f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 msgr2=0x55a4623f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2518749190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46218af70 con 0x55a4623eb8e0 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f1e10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0c2f30 tx=0x7f8b3c0c2e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe90e9803d6417e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b357fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b357fa640 1 -- 172.21.15.66:0/2518749190 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0bf0c0 con 0x55a4623eb8e0 2024-04-21T09:09:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b357fa640 1 -- 172.21.15.66:0/2518749190 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8b3c005420 con 0x55a4623eb8e0 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.913+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3c03c350 con 0x55a4623eb8e0 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b357fa640 1 -- 172.21.15.66:0/2518749190 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b3c0c7630 con 0x55a4623eb8e0 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 10 client.16332 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a462022170 con 0x55a4623eb8e0 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b357fa640 1 -- 172.21.15.66:0/2518749190 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b3c03c350 con 0x55a4623eb8e0 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b357fa640 1 -- 172.21.15.66:0/2518749190 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b3c0f02e0 con 0x55a4623eb8e0 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 10 client.16332 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 20 client.16332 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a462404f30 0x55a462407320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 4 client.16332 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a46235d920 con 0x55a462404f30 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.914+0000 7f8b515de640 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a462404f30 0x55a462407320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.915+0000 7f8b515de640 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a462404f30 0x55a462407320 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf493e1065f1c762 server_cookie=ec4919dbb5609074 in_seq=0 out_seq=0 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.915+0000 7f8b357fa640 10 client.16332 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.915+0000 7f8b357fa640 1 -- 172.21.15.66:0/2518749190 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a46235d920 con 0x55a462404f30 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.915+0000 7f8b357fa640 10 client.16332 handle_command_reply: tid=0 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.915+0000 7f8b56c3b740 1 client.16332 shutdown 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 msgr2=0x55a4623f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f1e10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0c2f30 tx=0x7f8b3c0c2e30 comp rx=0 tx=0).stop 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 shutdown_connections 2024-04-21T09:09:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a462404f30 0x55a462407320 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2518749190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f1e10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b357fa640 0 client.16332 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 >> 172.21.15.66:0/2518749190 conn(0x55a4623e5fb0 msgr2=0x55a4623e7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 shutdown_connections 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2518749190 wait complete. 2024-04-21T09:09:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.916+0000 7f8b56c3b740 20 client.16332 trim_cache size 0 max 16384 2024-04-21T09:09:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 Processor -- start 2024-04-21T09:09:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 -- start start 2024-04-21T09:09:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623ec290 0x55a462400940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623f4e10 0x55a4623e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b3da0 con 0x55a4623ec290 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b1670 con 0x55a4623f4e10 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1af4a0 con 0x55a4623eb8e0 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b515de640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623ec290 0x55a462400940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b525e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623f4e10 0x55a4623e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b525e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623f4e10 0x55a4623e4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:42262/0 (socket says 172.21.15.66:42262) 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b515de640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623ec290 0x55a462400940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37220/0 (socket says 172.21.15.66:37220) 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b525e0640 1 -- 172.21.15.66:0/3612874394 learned_addr learned my addr 172.21.15.66:0/3612874394 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b525e0640 1 -- 172.21.15.66:0/3612874394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 msgr2=0x55a4623f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.934+0000 7f8b525e0640 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b525e0640 1 -- 172.21.15.66:0/3612874394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623ec290 msgr2=0x55a462400940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b525e0640 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623ec290 0x55a462400940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b525e0640 1 -- 172.21.15.66:0/3612874394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4623e1880 con 0x55a4623f4e10 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b515de640 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623ec290 0x55a462400940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b525e0640 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623f4e10 0x55a4623e4e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009f90 tx=0x7f8b4401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216e6f382cd673c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b377fe640 1 -- 172.21.15.66:0/3612874394 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44009670 con 0x55a4623f4e10 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b377fe640 1 -- 172.21.15.66:0/3612874394 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b44020980 con 0x55a4623f4e10 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.935+0000 7f8b377fe640 1 -- 172.21.15.66:0/3612874394 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b440054f0 con 0x55a4623f4e10 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 -- 172.21.15.66:0/3612874394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623f4e10 msgr2=0x55a4623e4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623f4e10 0x55a4623e4e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009f90 tx=0x7f8b4401f000 comp rx=0 tx=0).stop 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 -- 172.21.15.66:0/3612874394 shutdown_connections 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623f4e10 0x55a4623e4e90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623ec290 0x55a462400940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/3612874394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623eb8e0 0x55a4623f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 -- 172.21.15.66:0/3612874394 >> 172.21.15.66:0/3612874394 conn(0x55a4623e5fb0 msgr2=0x55a4623e4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 -- 172.21.15.66:0/3612874394 shutdown_connections 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 -- 172.21.15.66:0/3612874394 wait complete. 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 -- start start 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.936+0000 7f8b56c3b740 1 Processor -- start 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b56c3b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1afb30 con 0x55a4623eb8e0 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b0fa0 con 0x55a4623ec290 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b56c3b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b0200 con 0x55a4623f4e10 2024-04-21T09:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b56c3b740 10 client.0 _ll_register_callbacks cb 0x7f8b44012610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37224/0 (socket says 172.21.15.66:37224) 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b515de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2184731281 learned_addr learned my addr 172.21.15.66:0/2184731281 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b525e0640 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2184731281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 msgr2=0x55a4623f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 msgr2=0x55a4623f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 -- 172.21.15.66:0/2184731281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46246a620 con 0x55a4623eb8e0 2024-04-21T09:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b515de640 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b51ddf640 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f8b20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0c71f0 tx=0x7f8b3c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ebba12765b98fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.937+0000 7f8b357fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b357fa640 1 -- 172.21.15.66:0/2184731281 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0be500 con 0x55a4623eb8e0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b357fa640 1 -- 172.21.15.66:0/2184731281 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8b3c0f0250 con 0x55a4623eb8e0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b44020980 con 0x55a4623eb8e0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b357fa640 1 -- 172.21.15.66:0/2184731281 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b3c0bf0c0 con 0x55a4623eb8e0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 10 client.16338 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a46246a620 con 0x55a4623eb8e0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b357fa640 1 -- 172.21.15.66:0/2184731281 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b44020980 con 0x55a4623eb8e0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b357fa640 1 -- 172.21.15.66:0/2184731281 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8b3c0892a0 con 0x55a4623eb8e0 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 10 client.16338 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 20 client.16338 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a46246bc60 0x55a46246e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 4 client.16338 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.938+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a462357790 con 0x55a46246bc60 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.939+0000 7f8b515de640 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a46246bc60 0x55a46246e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.939+0000 7f8b515de640 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a46246bc60 0x55a46246e050 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c8b969ece88851 server_cookie=79bafeb560664613 in_seq=0 out_seq=0 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.939+0000 7f8b357fa640 10 client.16338 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.940+0000 7f8b357fa640 1 -- 172.21.15.66:0/2184731281 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a462357790 con 0x55a46246bc60 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.940+0000 7f8b357fa640 10 client.16338 handle_command_reply: tid=0 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.940+0000 7f8b56c3b740 1 client.16338 shutdown 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.940+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 msgr2=0x55a4623f8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.940+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f8b20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0c71f0 tx=0x7f8b3c0035f0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 shutdown_connections 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a46246bc60 0x55a46246e050 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4623f4e10 0x55a4623f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4623ec290 0x55a4623f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 --2- 172.21.15.66:0/2184731281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4623eb8e0 0x55a4623f8b20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 >> 172.21.15.66:0/2184731281 conn(0x55a4623e5fb0 msgr2=0x55a4623e5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b357fa640 0 client.16338 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 shutdown_connections 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 1 -- 172.21.15.66:0/2184731281 wait complete. 2024-04-21T09:09:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b56c3b740 20 client.16338 trim_cache size 0 max 16384 2024-04-21T09:09:54.950 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:09:54.950 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:09:54.950 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "c4bde890-71a6-4ea5-8dd6-ab69b52f896b", 2024-04-21T09:09:54.950 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:09:54.950 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b380a3d20 msgr2=0x7f8b380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b54043640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b380a3d20 0x7f8b380a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b44012b00 tx=0x7f8b44009cd0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 msgr2=0x7f8b4c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.941+0000 7f8b54043640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c1a0640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000c00 tx=0x7f8b3c02c6c0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 shutdown_connections 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8b380a3d20 0x7f8b380a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4c10e900 0x7f8b4c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c101670 0x7f8b4c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 --2- 172.21.15.66:0/2424180868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b4c100d50 0x7f8b4c1a0640 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 >> 172.21.15.66:0/2424180868 conn(0x7f8b4c077f50 msgr2=0x7f8b4c10ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 shutdown_connections 2024-04-21T09:09:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:54.942+0000 7f8b54043640 1 -- 172.21.15.66:0/2424180868 wait complete. 2024-04-21T09:09:54.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:09:54.954 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:09:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 -- 172.21.15.66:0/3977451847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530106150 msgr2=0x7f453010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 --2- 172.21.15.66:0/3977451847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530106150 0x7f453010dd20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4528009ca0 tx=0x7f452802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 -- 172.21.15.66:0/3977451847 shutdown_connections 2024-04-21T09:09:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 --2- 172.21.15.66:0/3977451847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530106150 0x7f453010dd20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 --2- 172.21.15.66:0/3977451847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4530101a30 0x7f4530105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 --2- 172.21.15.66:0/3977451847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4530101080 0x7f4530101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 -- 172.21.15.66:0/3977451847 >> 172.21.15.66:0/3977451847 conn(0x7f4530078000 msgr2=0x7f4530078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 -- 172.21.15.66:0/3977451847 shutdown_connections 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.066+0000 7f4537afd640 1 -- 172.21.15.66:0/3977451847 wait complete. 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 Processor -- start 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 -- start start 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530101080 0x7f4530197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4530106150 0x7f45301978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4530197e10 0x7f45301a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45301110c0 con 0x7f4530101080 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4530110f40 con 0x7f4530106150 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4537afd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4530111240 con 0x7f4530197e10 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530101080 0x7f4530197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530101080 0x7f4530197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37246/0 (socket says 172.21.15.66:37246) 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 -- 172.21.15.66:0/168422321 learned_addr learned my addr 172.21.15.66:0/168422321 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f453609a640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4530197e10 0x7f45301a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535098640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4530106150 0x7f45301978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 -- 172.21.15.66:0/168422321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4530197e10 msgr2=0x7f45301a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4530197e10 0x7f45301a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 -- 172.21.15.66:0/168422321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4530106150 msgr2=0x7f45301978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4530106150 0x7f45301978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.067+0000 7f4535899640 1 -- 172.21.15.66:0/168422321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45301a6fa0 con 0x7f4530101080 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.068+0000 7f4535899640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530101080 0x7f4530197390 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f452c00d770 tx=0x7f452c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.068+0000 7f4526ffd640 1 -- 172.21.15.66:0/168422321 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c017070 con 0x7f4530101080 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.068+0000 7f4526ffd640 1 -- 172.21.15.66:0/168422321 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452c0054d0 con 0x7f4530101080 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.068+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45301a7290 con 0x7f4530101080 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.068+0000 7f4526ffd640 1 -- 172.21.15.66:0/168422321 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c004c30 con 0x7f4530101080 2024-04-21T09:09:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.068+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301a7760 con 0x7f4530101080 2024-04-21T09:09:55.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.070+0000 7f4526ffd640 1 -- 172.21.15.66:0/168422321 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f452c004e10 con 0x7f4530101080 2024-04-21T09:09:55.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.071+0000 7f4526ffd640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f451c0a3d20 0x7f451c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.071+0000 7f4526ffd640 1 -- 172.21.15.66:0/168422321 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f452c0d0590 con 0x7f4530101080 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.071+0000 7f4535098640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f451c0a3d20 0x7f451c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.071+0000 7f4535098640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f451c0a3d20 0x7f451c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45301994f0 tx=0x7f452000b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.072+0000 7f453a7c7740 1 Processor -- start 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.072+0000 7f453a7c7740 1 -- start start 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa37960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa40a80 0x55b62aa42e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453a7c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b62a99c1e0 con 0x55b62aa40a80 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453a7c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b62a939390 con 0x55b62aa37ea0 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453a7c7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b62a9981a0 con 0x55b62aa37580 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa40a80 0x55b62aa42e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa40a80 0x55b62aa42e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37258/0 (socket says 172.21.15.66:37258) 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 -- 172.21.15.66:0/1083769941 learned_addr learned my addr 172.21.15.66:0/1083769941 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f4535899640 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa37960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f4535098640 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 -- 172.21.15.66:0/1083769941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 msgr2=0x55b62aa37960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa37960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 -- 172.21.15.66:0/1083769941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 msgr2=0x55b62aa40540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa40540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 -- 172.21.15.66:0/1083769941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62a7787d0 con 0x55b62aa40a80 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.073+0000 7f453609a640 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa40a80 0x55b62aa42e70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4528009ca0 tx=0x7f4528005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2aab9d2227384df server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.074+0000 7f451bfff640 1 -- 172.21.15.66:0/1083769941 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528043070 con 0x55b62aa40a80 2024-04-21T09:09:55.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.074+0000 7f451bfff640 1 -- 172.21.15.66:0/1083769941 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528032970 con 0x55b62aa40a80 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.074+0000 7f451bfff640 1 -- 172.21.15.66:0/1083769941 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528039040 con 0x55b62aa40a80 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 -- 172.21.15.66:0/1083769941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa40a80 msgr2=0x55b62aa42e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa40a80 0x55b62aa42e70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4528009ca0 tx=0x7f4528005f70 comp rx=0 tx=0).stop 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 -- 172.21.15.66:0/1083769941 shutdown_connections 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa40a80 0x55b62aa42e70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 --2- 172.21.15.66:0/1083769941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa37960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 -- 172.21.15.66:0/1083769941 >> 172.21.15.66:0/1083769941 conn(0x55b62aa31cb0 msgr2=0x55b62aa32090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 -- 172.21.15.66:0/1083769941 shutdown_connections 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 -- 172.21.15.66:0/1083769941 wait complete. 2024-04-21T09:09:55.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.075+0000 7f453a7c7740 1 -- start start 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 1 Processor -- start 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa3dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa3e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b62aa43810 con 0x55b62aa37ea0 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b62a9552f0 con 0x55b62aa40a80 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b62aa43990 con 0x55b62aa37580 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453a7c7740 10 client.0 _ll_register_callbacks cb 0x55b62a962580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f4535098640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa3e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f4535098640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa3e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37262/0 (socket says 172.21.15.66:37262) 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f4535098640 1 -- 172.21.15.66:0/3600407826 learned_addr learned my addr 172.21.15.66:0/3600407826 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:55.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.076+0000 7f453609a640 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f4535899640 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa3dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f4535098640 1 -- 172.21.15.66:0/3600407826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 msgr2=0x55b62aa3dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f4535098640 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa3dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f4535098640 1 -- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 msgr2=0x55b62aa3e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f4535098640 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f4535098640 1 -- 172.21.15.66:0/3600407826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b62a6ae3f0 con 0x55b62aa37ea0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f4535098640 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa3e010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4520011700 tx=0x7f4520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9ffd90b901e693 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f45197fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f45197fa640 1 -- 172.21.15.66:0/3600407826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452001fb40 con 0x55b62aa37ea0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f45197fa640 1 -- 172.21.15.66:0/3600407826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f452001fce0 con 0x55b62aa37ea0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4528032970 con 0x55b62aa37ea0 2024-04-21T09:09:55.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f45197fa640 1 -- 172.21.15.66:0/3600407826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4520028440 con 0x55b62aa37ea0 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f453a7c7740 10 client.16350 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b62a66e0f0 con 0x55b62aa37ea0 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.077+0000 7f45197fa640 1 -- 172.21.15.66:0/3600407826 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528032970 con 0x55b62aa37ea0 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f45197fa640 1 -- 172.21.15.66:0/3600407826 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4520026070 con 0x55b62aa37ea0 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f453a7c7740 10 client.16350 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f453a7c7740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f453a7c7740 20 client.16350 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f453a7c7740 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aa50ae0 0x55b62aa52f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f453a7c7740 4 client.16350 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b62a9a9c00 con 0x55b62aa50ae0 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f4535899640 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aa50ae0 0x55b62aa52f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f45197fa640 10 client.16350 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.078+0000 7f4535899640 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aa50ae0 0x55b62aa52f70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668d2997e14c8f server_cookie=3e630855b5e52e27 in_seq=0 out_seq=0 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f45197fa640 1 -- 172.21.15.66:0/3600407826 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b62a9a9c00 con 0x55b62aa50ae0 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f45197fa640 10 client.16350 handle_command_reply: tid=0 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 client.16350 shutdown 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 msgr2=0x55b62aa3e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa3e010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4520011700 tx=0x7f4520004730 comp rx=0 tx=0).stop 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 shutdown_connections 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aa50ae0 0x55b62aa52f70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa3e010 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 --2- 172.21.15.66:0/3600407826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa3dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f45197fa640 0 client.16350 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 >> 172.21.15.66:0/3600407826 conn(0x55b62aa31cb0 msgr2=0x55b62aa33a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 shutdown_connections 2024-04-21T09:09:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.079+0000 7f453a7c7740 1 -- 172.21.15.66:0/3600407826 wait complete. 2024-04-21T09:09:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.080+0000 7f453a7c7740 20 client.16350 trim_cache size 0 max 16384 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.097+0000 7f453a7c7740 1 Processor -- start 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.097+0000 7f453a7c7740 1 -- start start 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa40200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa4c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f453a7c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b3d60 con 0x55b62aa37ea0 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f453a7c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b1630 con 0x55b62aa40a80 2024-04-21T09:09:55.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f453a7c7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45301af460 con 0x55b62aa37580 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa4c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa4c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37266/0 (socket says 172.21.15.66:37266) 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 -- 172.21.15.66:0/758934796 learned_addr learned my addr 172.21.15.66:0/758934796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535899640 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa40200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f453609a640 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 -- 172.21.15.66:0/758934796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 msgr2=0x55b62aa40200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa40200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 -- 172.21.15.66:0/758934796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 msgr2=0x55b62aa3ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 -- 172.21.15.66:0/758934796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62aa42e70 con 0x55b62aa37ea0 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f4535098640 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa4c6d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4520011700 tx=0x7f4520000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51759866b92cd63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.098+0000 7f451b7fe640 1 -- 172.21.15.66:0/758934796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452003b070 con 0x55b62aa37ea0 2024-04-21T09:09:55.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f451b7fe640 1 -- 172.21.15.66:0/758934796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452001f940 con 0x55b62aa37ea0 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f451b7fe640 1 -- 172.21.15.66:0/758934796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520027e20 con 0x55b62aa37ea0 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 -- 172.21.15.66:0/758934796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 msgr2=0x55b62aa4c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa4c6d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4520011700 tx=0x7f4520000bc0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 -- 172.21.15.66:0/758934796 shutdown_connections 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa40a80 0x55b62aa3ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37ea0 0x55b62aa4c6d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 --2- 172.21.15.66:0/758934796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa37580 0x55b62aa40200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 -- 172.21.15.66:0/758934796 >> 172.21.15.66:0/758934796 conn(0x55b62aa31cb0 msgr2=0x55b62aa33990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 -- 172.21.15.66:0/758934796 shutdown_connections 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.099+0000 7f453a7c7740 1 -- 172.21.15.66:0/758934796 wait complete. 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 -- start start 2024-04-21T09:09:55.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 Processor -- start 2024-04-21T09:09:55.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37580 0x55b62aa48be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa40a80 0x55b62aa45c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45301afaf0 con 0x55b62aa37580 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b0f60 con 0x55b62aa37ea0 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b01c0 con 0x55b62aa40a80 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f453a7c7740 10 client.0 _ll_register_callbacks cb 0x7f4520010ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f4535899640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37580 0x55b62aa48be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f4535899640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37580 0x55b62aa48be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37274/0 (socket says 172.21.15.66:37274) 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f4535899640 1 -- 172.21.15.66:0/598235695 learned_addr learned my addr 172.21.15.66:0/598235695 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.100+0000 7f4535098640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f453609a640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa40a80 0x55b62aa45c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f4535899640 1 -- 172.21.15.66:0/598235695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa40a80 msgr2=0x55b62aa45c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f4535899640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa40a80 0x55b62aa45c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f4535899640 1 -- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 msgr2=0x55b62aa49120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f4535899640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa49120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f4535899640 1 -- 172.21.15.66:0/598235695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b62aa46440 con 0x55b62aa37580 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f4535098640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa49120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f4535899640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37580 0x55b62aa48be0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f452c09bf70 tx=0x7f452c098640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a995e6b2ba8b7b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f45197fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f45197fa640 1 -- 172.21.15.66:0/598235695 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c020040 con 0x55b62aa37580 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f45197fa640 1 -- 172.21.15.66:0/598235695 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f452c0cdbf0 con 0x55b62aa37580 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f452001f940 con 0x55b62aa37580 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f45197fa640 1 -- 172.21.15.66:0/598235695 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f452c0bb0d0 con 0x55b62aa37580 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f453a7c7740 10 client.16356 fetch_fsmap learned FSMap version 25 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b62aab7cb0 con 0x55b62aa37580 2024-04-21T09:09:55.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.101+0000 7f45197fa640 1 -- 172.21.15.66:0/598235695 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452001f940 con 0x55b62aa37580 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f45197fa640 1 -- 172.21.15.66:0/598235695 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f452c09c4d0 con 0x55b62aa37580 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f453a7c7740 10 client.16356 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f453a7c7740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f453a7c7740 20 client.16356 populate_metadata read hostname 'smithi066' 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f453a7c7740 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aab8380 0x55b62aaba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f453a7c7740 4 client.16356 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b62a9a3a70 con 0x55b62aab8380 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f4535098640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aab8380 0x55b62aaba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f4535098640 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aab8380 0x55b62aaba770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3074163be8efd658 server_cookie=b0e151cfee01d8c0 in_seq=0 out_seq=0 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.102+0000 7f45197fa640 10 client.16356 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f45197fa640 1 -- 172.21.15.66:0/598235695 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b62a9a3a70 con 0x55b62aab8380 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f45197fa640 10 client.16356 handle_command_reply: tid=0 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 client.16356 shutdown 2024-04-21T09:09:55.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37580 msgr2=0x55b62aa48be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37580 0x55b62aa48be0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f452c09bf70 tx=0x7f452c098640 comp rx=0 tx=0).stop 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 shutdown_connections 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b62aab8380 0x55b62aaba770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b62aa40a80 0x55b62aa45c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f45197fa640 0 client.16356 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b62aa37ea0 0x55b62aa49120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 --2- 172.21.15.66:0/598235695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b62aa37580 0x55b62aa48be0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 >> 172.21.15.66:0/598235695 conn(0x55b62aa31cb0 msgr2=0x55b62aa3da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 shutdown_connections 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 1 -- 172.21.15.66:0/598235695 wait complete. 2024-04-21T09:09:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f453a7c7740 20 client.16356 trim_cache size 0 max 16384 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: "c4bde890-71a6-4ea5-8dd6-ab69b52f896b": { 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "c4bde890-71a6-4ea5-8dd6-ab69b52f896b", 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:09:55.111 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:09:55.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f451c0a3d20 msgr2=0x7f451c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.103+0000 7f4537afd640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f451c0a3d20 0x7f451c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45301994f0 tx=0x7f452000b390 comp rx=0 tx=0).stop 2024-04-21T09:09:55.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530101080 msgr2=0x7f4530197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:09:55.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530101080 0x7f4530197390 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f452c00d770 tx=0x7f452c00dc30 comp rx=0 tx=0).stop 2024-04-21T09:09:55.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 shutdown_connections 2024-04-21T09:09:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f451c0a3d20 0x7f451c0a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4530197e10 0x7f45301a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4530106150 0x7f45301978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 --2- 172.21.15.66:0/168422321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4530101080 0x7f4530197390 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:09:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 >> 172.21.15.66:0/168422321 conn(0x7f4530078000 msgr2=0x7f45300fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:09:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 shutdown_connections 2024-04-21T09:09:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:09:55.104+0000 7f4537afd640 1 -- 172.21.15.66:0/168422321 wait complete. 2024-04-21T09:09:55.117 INFO:tasks.cephfs.filesystem:scrub status for tag:c4bde890-71a6-4ea5-8dd6-ab69b52f896b - {'path': '/', 'tag': 'c4bde890-71a6-4ea5-8dd6-ab69b52f896b', 'options': 'recursive,force'} 2024-04-21T09:09:55.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:55 smithi066 ceph-mon[27953]: pgmap v54: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 21 op/s 2024-04-21T09:09:55.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:55 smithi186 ceph-mon[31505]: pgmap v54: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 21 op/s 2024-04-21T09:09:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:55 smithi083 ceph-mon[31413]: pgmap v54: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 21 op/s 2024-04-21T09:09:56.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:56 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:09:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:56 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:56 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:09:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:56 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:56 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:09:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:56 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:09:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:56 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:56 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:09:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:56 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:56 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:09:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:56 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:09:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:56 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:56 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:09:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:56 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:09:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:56 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:09:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:57 smithi186 ceph-mon[31505]: pgmap v55: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-04-21T09:09:57.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:57 smithi083 ceph-mon[31413]: pgmap v55: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-04-21T09:09:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:57 smithi066 ceph-mon[27953]: pgmap v55: 129 pgs: 129 active+clean; 259 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-04-21T09:09:59.421 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:09:59 smithi066 ceph-mon[27953]: pgmap v56: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:59.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:09:59 smithi186 ceph-mon[31505]: pgmap v56: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:09:59.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:09:59 smithi083 ceph-mon[31413]: pgmap v56: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-21T09:10:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:00 smithi186 ceph-mon[31505]: overall HEALTH_OK 2024-04-21T09:10:00.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:00 smithi083 ceph-mon[31413]: overall HEALTH_OK 2024-04-21T09:10:00.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:00 smithi066 ceph-mon[27953]: overall HEALTH_OK 2024-04-21T09:10:01.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:01 smithi186 ceph-mon[31505]: pgmap v57: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:10:01.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:01 smithi083 ceph-mon[31413]: pgmap v57: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:10:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:01 smithi066 ceph-mon[27953]: pgmap v57: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:10:03.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:03 smithi186 ceph-mon[31505]: pgmap v58: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:10:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:03 smithi083 ceph-mon[31413]: pgmap v58: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:10:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:03 smithi066 ceph-mon[27953]: pgmap v58: 129 pgs: 129 active+clean; 259 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 864 KiB/s wr, 11 op/s 2024-04-21T09:10:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 -- 172.21.15.66:0/2331221548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81061b0 msgr2=0x7fbbc810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 --2- 172.21.15.66:0/2331221548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81061b0 0x7fbbc810dd80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009ca0 tx=0x7fbbc002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 -- 172.21.15.66:0/2331221548 shutdown_connections 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 --2- 172.21.15.66:0/2331221548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81061b0 0x7fbbc810dd80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 --2- 172.21.15.66:0/2331221548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc8101a90 0x7fbbc8105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 --2- 172.21.15.66:0/2331221548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc81010e0 0x7fbbc81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 -- 172.21.15.66:0/2331221548 >> 172.21.15.66:0/2331221548 conn(0x7fbbc80771d0 msgr2=0x7fbbc80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 -- 172.21.15.66:0/2331221548 shutdown_connections 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.395+0000 7fbbcede6640 1 -- 172.21.15.66:0/2331221548 wait complete. 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 Processor -- start 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 -- start start 2024-04-21T09:10:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc81010e0 0x7fbbc8102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc81061b0 0x7fbbc81030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81035f0 0x7fbbc81a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8111120 con 0x7fbbc81035f0 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8110fa0 con 0x7fbbc81010e0 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcede6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81112a0 con 0x7fbbc81061b0 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc81010e0 0x7fbbc8102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc81061b0 0x7fbbc81030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbcd383640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81035f0 0x7fbbc81a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc81010e0 0x7fbbc8102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:42342/0 (socket says 172.21.15.66:42342) 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.396+0000 7fbbccb82640 1 -- 172.21.15.66:0/3377746054 learned_addr learned my addr 172.21.15.66:0/3377746054 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbccb82640 1 -- 172.21.15.66:0/3377746054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc81061b0 msgr2=0x7fbbc81030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc81061b0 0x7fbbc81030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbccb82640 1 -- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81035f0 msgr2=0x7fbbc81a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81035f0 0x7fbbc81a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbccb82640 1 -- 172.21.15.66:0/3377746054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc81a6f40 con 0x7fbbc81010e0 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc81061b0 0x7fbbc81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc81010e0 0x7fbbc8102b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e9f0 tx=0x7fbbbc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbc5ffb640 1 -- 172.21.15.66:0/3377746054 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00cd80 con 0x7fbbc81010e0 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc81a7230 con 0x7fbbc81010e0 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbc5ffb640 1 -- 172.21.15.66:0/3377746054 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc0054d0 con 0x7fbbc81010e0 2024-04-21T09:10:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.397+0000 7fbbc5ffb640 1 -- 172.21.15.66:0/3377746054 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc010690 con 0x7fbbc81010e0 2024-04-21T09:10:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.398+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81a7730 con 0x7fbbc81010e0 2024-04-21T09:10:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.400+0000 7fbbc5ffb640 1 -- 172.21.15.66:0/3377746054 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbbbc004e50 con 0x7fbbc81010e0 2024-04-21T09:10:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.401+0000 7fbbd19d4740 1 Processor -- start 2024-04-21T09:10:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.401+0000 7fbbd19d4740 1 -- start start 2024-04-21T09:10:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.401+0000 7fbbc5ffb640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbb80a3df0 0x7fbbb80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.401+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebe2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.401+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebeece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abaeb3f8d0 con 0x55abaebe3370 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55abaeadce80 con 0x55abaebe40e0 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55abaeb3b890 con 0x55abaebec8f0 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbcd383640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbb80a3df0 0x7fbbb80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebe2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebe2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:42348/0 (socket says 172.21.15.66:42348) 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 -- 172.21.15.66:0/2739128801 learned_addr learned my addr 172.21.15.66:0/2739128801 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbc5ffb640 1 -- 172.21.15.66:0/3377746054 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbbbc0d0f40 con 0x7fbbc81010e0 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbcd383640 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebeece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 -- 172.21.15.66:0/2739128801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 msgr2=0x55abaebeece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebeece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 -- 172.21.15.66:0/2739128801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 msgr2=0x55abaebe37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbccb82640 1 -- 172.21.15.66:0/2739128801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abaeafe320 con 0x55abaebe40e0 2024-04-21T09:10:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.402+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:10:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.403+0000 7fbbccb82640 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebe2e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc099540 tx=0x7fbbbc01bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c8ae19a71bb5ab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.403+0000 7fbbb2ffd640 1 -- 172.21.15.66:0/2739128801 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc09c4d0 con 0x55abaebe40e0 2024-04-21T09:10:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.403+0000 7fbbb2ffd640 1 -- 172.21.15.66:0/2739128801 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc0610c0 con 0x55abaebe40e0 2024-04-21T09:10:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.403+0000 7fbbcd383640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbb80a3df0 0x7fbbb80a62b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0004af0 tx=0x7fbbc0032380 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.403+0000 7fbbb2ffd640 1 -- 172.21.15.66:0/2739128801 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0889f0 con 0x55abaebe40e0 2024-04-21T09:10:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 -- 172.21.15.66:0/2739128801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 msgr2=0x55abaebe2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebe2e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc099540 tx=0x7fbbbc01bb30 comp rx=0 tx=0).stop 2024-04-21T09:10:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 -- 172.21.15.66:0/2739128801 shutdown_connections 2024-04-21T09:10:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebeece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/2739128801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebe2e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 -- 172.21.15.66:0/2739128801 >> 172.21.15.66:0/2739128801 conn(0x55abaeba1c90 msgr2=0x55abaebde060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 -- 172.21.15.66:0/2739128801 shutdown_connections 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 -- 172.21.15.66:0/2739128801 wait complete. 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 -- start start 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.404+0000 7fbbd19d4740 1 Processor -- start 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe3370 0x55abaebec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe40e0 0x55abaebe9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebe9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abaebef620 con 0x55abaebe40e0 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55abaeac7eb0 con 0x55abaebe3370 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55abaebef7a0 con 0x55abaebec8f0 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbd19d4740 10 client.0 _ll_register_callbacks cb 0x55abaeaf4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe3370 0x55abaebec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe3370 0x55abaebec300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:42356/0 (socket says 172.21.15.66:42356) 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbcd383640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebe9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 -- 172.21.15.66:0/3655209175 learned_addr learned my addr 172.21.15.66:0/3655209175 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 -- 172.21.15.66:0/3655209175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 msgr2=0x55abaebe9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebe9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 -- 172.21.15.66:0/3655209175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe40e0 msgr2=0x55abaebe9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe40e0 0x55abaebe9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.405+0000 7fbbccb82640 1 -- 172.21.15.66:0/3655209175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abae852210 con 0x55abaebe3370 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe3370 0x55abaebec300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc09cb00 tx=0x7fbbbc09cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2acc84a70727771 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbba7fff640 1 -- 172.21.15.66:0/3655209175 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0a57e0 con 0x55abaebe3370 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc0610c0 con 0x55abaebe3370 2024-04-21T09:10:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbba7fff640 1 -- 172.21.15.66:0/3655209175 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbbbc005140 con 0x55abaebe3370 2024-04-21T09:10:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbba7fff640 1 -- 172.21.15.66:0/3655209175 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc0b6080 con 0x55abaebe3370 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbbd19d4740 10 client.25970 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55abae91e620 con 0x55abaebe3370 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbba7fff640 1 -- 172.21.15.66:0/3655209175 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc0610c0 con 0x55abaebe3370 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.406+0000 7fbba7fff640 1 -- 172.21.15.66:0/3655209175 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbbbc0b7070 con 0x55abaebe3370 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbd19d4740 10 client.25970 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbd19d4740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbd19d4740 20 client.25970 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaebfc810 0x55abaebfec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbd19d4740 4 client.25970 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abaeb53880 con 0x55abaebfc810 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaebfc810 0x55abaebfec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbba7fff640 10 client.25970 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.407+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaebfc810 0x55abaebfec00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f2ab0da5b3a8f server_cookie=562b9b0eb8f4de53 in_seq=0 out_seq=0 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbba7fff640 1 -- 172.21.15.66:0/3655209175 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abaeb53880 con 0x55abaebfc810 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbba7fff640 10 client.25970 handle_command_reply: tid=0 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 client.25970 shutdown 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe3370 msgr2=0x55abaebec300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe3370 0x55abaebec300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc09cb00 tx=0x7fbbbc09cc60 comp rx=0 tx=0).stop 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 shutdown_connections 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaebfc810 0x55abaebfec00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebe9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbba7fff640 0 client.25970 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe40e0 0x55abaebe9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3655209175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe3370 0x55abaebec300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 >> 172.21.15.66:0/3655209175 conn(0x55abaeba1c90 msgr2=0x55abaebdfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 shutdown_connections 2024-04-21T09:10:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.408+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3655209175 wait complete. 2024-04-21T09:10:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.409+0000 7fbbd19d4740 20 client.25970 trim_cache size 0 max 16384 2024-04-21T09:10:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.428+0000 7fbbd19d4740 1 Processor -- start 2024-04-21T09:10:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.428+0000 7fbbd19d4740 1 -- start start 2024-04-21T09:10:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebe40e0 0x55abaebebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebec8f0 0x55abaebec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81af8f0 con 0x55abaebe3370 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ad1c0 con 0x55abaebec8f0 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ab010 con 0x55abaebe40e0 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37328/0 (socket says 172.21.15.66:37328) 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 -- 172.21.15.66:0/1057058342 learned_addr learned my addr 172.21.15.66:0/1057058342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbcd383640 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebec8f0 0x55abaebec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebe40e0 0x55abaebebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 -- 172.21.15.66:0/1057058342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebe40e0 msgr2=0x55abaebebdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebe40e0 0x55abaebebdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 -- 172.21.15.66:0/1057058342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebec8f0 msgr2=0x55abaebec300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebec8f0 0x55abaebec300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbccb82640 1 -- 172.21.15.66:0/1057058342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abaebe2640 con 0x55abaebe3370 2024-04-21T09:10:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebe40e0 0x55abaebebdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:10:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.429+0000 7fbbcd383640 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebec8f0 0x55abaebec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:10:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.430+0000 7fbbccb82640 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe8690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc09c870 tx=0x7fbbbc0b6220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17016ae6601d3361 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.430+0000 7fbbb27fc640 1 -- 172.21.15.66:0/1057058342 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0a57e0 con 0x55abaebe3370 2024-04-21T09:10:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.430+0000 7fbbb27fc640 1 -- 172.21.15.66:0/1057058342 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc0a0f20 con 0x55abaebe3370 2024-04-21T09:10:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.430+0000 7fbbb27fc640 1 -- 172.21.15.66:0/1057058342 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc09cdd0 con 0x55abaebe3370 2024-04-21T09:10:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 -- 172.21.15.66:0/1057058342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 msgr2=0x55abaebe8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe8690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc09c870 tx=0x7fbbbc0b6220 comp rx=0 tx=0).stop 2024-04-21T09:10:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 -- 172.21.15.66:0/1057058342 shutdown_connections 2024-04-21T09:10:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebec8f0 0x55abaebec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebe40e0 0x55abaebebdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/1057058342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaebe8690 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 -- 172.21.15.66:0/1057058342 >> 172.21.15.66:0/1057058342 conn(0x55abaeba1c90 msgr2=0x55abaebe7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 -- 172.21.15.66:0/1057058342 shutdown_connections 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 -- 172.21.15.66:0/1057058342 wait complete. 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 -- start start 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.431+0000 7fbbd19d4740 1 Processor -- start 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaec020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbd19d4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ab680 con 0x55abaebe3370 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81acaf0 con 0x55abaebe40e0 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbd19d4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81abd50 con 0x55abaebec8f0 2024-04-21T09:10:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbd19d4740 10 client.0 _ll_register_callbacks cb 0x7fbbb4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbcd383640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaec020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbccb82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaec020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37338/0 (socket says 172.21.15.66:37338) 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbcd383640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:59696/0 (socket says 172.21.15.66:59696) 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.432+0000 7fbbccb82640 1 -- 172.21.15.66:0/3237080596 learned_addr learned my addr 172.21.15.66:0/3237080596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbc7fff640 1 -- 172.21.15.66:0/3237080596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 msgr2=0x55abaebff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbc7fff640 1 -- 172.21.15.66:0/3237080596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 msgr2=0x55abaec020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaec020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbc7fff640 1 -- 172.21.15.66:0/3237080596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abaebd0f30 con 0x55abaebe40e0 2024-04-21T09:10:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaec020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbc7fff640 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebff430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55abaebe8ef0 tx=0x7fbbb40139b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7896a65bf31c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbba7fff640 1 -- 172.21.15.66:0/3237080596 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4004220 con 0x55abaebe40e0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbba7fff640 1 -- 172.21.15.66:0/3237080596 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbbb40043c0 con 0x55abaebe40e0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc0a0f20 con 0x55abaebe40e0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbba7fff640 1 -- 172.21.15.66:0/3237080596 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbb402e050 con 0x55abaebe40e0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbd19d4740 10 client.25976 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.433+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55abaec62950 con 0x55abaebe40e0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbba7fff640 1 -- 172.21.15.66:0/3237080596 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc0a0f20 con 0x55abaebe40e0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbba7fff640 1 -- 172.21.15.66:0/3237080596 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbbb4003c30 con 0x55abaebe40e0 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbbd19d4740 10 client.25976 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbbd19d4740 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbbd19d4740 20 client.25976 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaec63000 0x55abaec65420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbbd19d4740 4 client.25976 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T09:10:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abaeadf3d0 con 0x55abaec63000 2024-04-21T09:10:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.434+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaec63000 0x55abaec65420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.435+0000 7fbba7fff640 10 client.25976 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.435+0000 7fbbccb82640 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaec63000 0x55abaec65420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7897c53a85cdb65a server_cookie=4e27883bd2256c1c in_seq=0 out_seq=0 2024-04-21T09:10:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbba7fff640 1 -- 172.21.15.66:0/3237080596 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abaeadf3d0 con 0x55abaec63000 2024-04-21T09:10:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbba7fff640 10 client.25976 handle_command_reply: tid=0 2024-04-21T09:10:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 client.25976 shutdown 2024-04-21T09:10:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 msgr2=0x55abaebff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebff430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55abaebe8ef0 tx=0x7fbbb40139b0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 shutdown_connections 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55abaec63000 0x55abaec65420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abaebec8f0 0x55abaebff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbba7fff640 0 client.25976 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55abaebe40e0 0x55abaebff430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 --2- 172.21.15.66:0/3237080596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abaebe3370 0x55abaec020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 >> 172.21.15.66:0/3237080596 conn(0x55abaeba1c90 msgr2=0x55abaebdf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.445+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 shutdown_connections 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.446+0000 7fbbd19d4740 1 -- 172.21.15.66:0/3237080596 wait complete. 2024-04-21T09:10:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.446+0000 7fbbd19d4740 20 client.25976 trim_cache size 0 max 16384 2024-04-21T09:10:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2024-04-21T09:10:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2024-04-21T09:10:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2024-04-21T09:10:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2024-04-21T09:10:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbb80a3df0 msgr2=0x7fbbb80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbb80a3df0 0x7fbbb80a62b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0004af0 tx=0x7fbbc0032380 comp rx=0 tx=0).stop 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc81010e0 msgr2=0x7fbbc8102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc81010e0 0x7fbbc8102b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e9f0 tx=0x7fbbbc00edf0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 shutdown_connections 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbbb80a3df0 0x7fbbb80a62b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbc81035f0 0x7fbbc81a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc81061b0 0x7fbbc81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 --2- 172.21.15.66:0/3377746054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbc81010e0 0x7fbbc8102b70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 >> 172.21.15.66:0/3377746054 conn(0x7fbbc80771d0 msgr2=0x7fbbc806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 shutdown_connections 2024-04-21T09:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2024-04-21T09:10:04.447+0000 7fbbcede6640 1 -- 172.21.15.66:0/3377746054 wait complete. 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:setting up random write file 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:done setting up random write file 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:starting fsync run 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0083s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0047s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0049s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0048s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0051s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0051s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0049s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0052s 2024-04-21T09:10:05.163 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0048s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0046s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0042s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0043s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0043s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0049s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0056s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0044s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0047s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0054s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0044s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0045s 2024-04-21T09:10:05.164 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0046s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0045s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0046s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0049s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0045s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0042s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0044s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0052s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0043s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0045s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0047s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0044s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0045s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0004s fsync time: 0.0061s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0067s 2024-04-21T09:10:05.165 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0004s fsync time: 0.0066s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0053s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0063s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0052s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0046s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0043s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0043s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0052s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0049s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0047s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0047s 2024-04-21T09:10:05.166 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0043s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0043s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0053s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0053s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0003s fsync time: 0.0055s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0043s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0042s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:write time: 0.0002s fsync time: 0.0043s 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stdout:run done 61 fsyncs total, killing random writer 2024-04-21T09:10:05.167 INFO:tasks.workunit.client.0.smithi066.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-21T09:10:05.168 INFO:tasks.workunit.client.0.smithi066.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-21T09:10:05.168 INFO:tasks.workunit.client.0.smithi066.stderr:+ whereis lsof 2024-04-21T09:10:05.176 INFO:tasks.workunit.client.0.smithi066.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-04-21T09:10:05.176 INFO:tasks.workunit.client.0.smithi066.stderr:+ lsof 2024-04-21T09:10:05.194 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6e6e8460c85900d354f99492666cf45a112a4c5e5d80ed4505d0d695213e4ebf/merged 2024-04-21T09:10:05.194 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.194 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/28c2811ee65339a5f6133fcaf26e0478427064b9e5134460eb14662089134e0a/merged 2024-04-21T09:10:05.194 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/ebacaecddd897b0717be8c5e994f713a59d4b59b4b7cbec947de8056306d4257/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/2a64c64163cddf5406e2227fa5928cb0de234046eb8cf1bdd2bd03ab1dbd4c0d/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/87782a91e1792af1c9f3f137312fc27f25b7f0a2dcd2e14a0064fb2a445c47d2/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/50a23de37d368ae1a88b0b10fe7d274914c26f92e162b6ee414be1b1d59b2e1e/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/e20f8fb5db133b7fd76f44f49c63a760ed8e0091b627ba44d420e609eb3c7e15/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/8c6fbdd2a573cc394556328d9a55b214b5ee0521d1a07bb71891179e6c2c42cc/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/b677b9811d912a62a75447145c4c165f37ad64285d7c9a01b156f4eb128b4fcf/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/bacf92df245ca9e132a8fde2aca28cef4652195e21fac7b75aff553fce0adadd/merged 2024-04-21T09:10:05.195 INFO:tasks.workunit.client.0.smithi066.stderr: Output information may be incomplete. 2024-04-21T09:10:05.259 INFO:tasks.workunit.client.0.smithi066.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-04-21T09:10:05.259 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-04-21T09:10:05.259 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-04-21T09:10:05.259 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-04-21T09:10:05.259 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-04-21T09:10:05.259 INFO:tasks.workunit.client.0.smithi066.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-04-21T09:10:05.260 INFO:tasks.workunit.client.0.smithi066.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-04-21T09:10:05.261 INFO:tasks.workunit.client.0.smithi066.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-04-21T09:10:05.262 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-04-21T09:10:05.263 INFO:tasks.workunit.client.0.smithi066.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-04-21T09:10:05.264 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-04-21T09:10:05.265 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-04-21T09:10:05.266 INFO:tasks.workunit.client.0.smithi066.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-04-21T09:10:05.267 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-04-21T09:10:05.268 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-04-21T09:10:05.269 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-04-21T09:10:05.270 INFO:tasks.workunit.client.0.smithi066.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-04-21T09:10:05.271 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-04-21T09:10:05.272 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-04-21T09:10:05.273 INFO:tasks.workunit.client.0.smithi066.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-04-21T09:10:05.274 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-04-21T09:10:05.275 INFO:tasks.workunit.client.0.smithi066.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-04-21T09:10:05.276 INFO:tasks.workunit.client.0.smithi066.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:khugepage 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:khugepage 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:khugepage 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:khugepage 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:cryptd 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:cryptd 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:cryptd 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:cryptd 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:kintegrit 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:kintegrit 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:kintegrit 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:kintegrit 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:kblockd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-04-21T09:10:05.277 INFO:tasks.workunit.client.0.smithi066.stdout:kblockd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:kblockd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:kblockd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:blkcg_pun 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:blkcg_pun 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:blkcg_pun 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:blkcg_pun 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:tpm_dev_w 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:tpm_dev_w 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:tpm_dev_w 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:md 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-04-21T09:10:05.278 INFO:tasks.workunit.client.0.smithi066.stdout:md 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:md 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:md 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:md_bitmap 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:md_bitmap 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:md_bitmap 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:md_bitmap 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:edac-poll 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:edac-poll 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:edac-poll 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:edac-poll 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:watchdogd 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:watchdogd 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:watchdogd 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:watchdogd 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-04-21T09:10:05.279 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kswapd0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kswapd0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kswapd0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kswapd0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kthrotld 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kthrotld 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kthrotld 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kthrotld 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 102 root cwd unknown /proc/102/cwd (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 102 root rtd unknown /proc/102/root (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 102 root txt unknown /proc/102/exe (readlink: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 102 root NOFD /proc/102/fd (opendir: Permission denied) 2024-04-21T09:10:05.280 INFO:tasks.workunit.client.0.smithi066.stdout:acpi_ther 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:acpi_ther 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:acpi_ther 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:acpi_ther 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kmpath_rd 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kmpath_rd 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kmpath_rd 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kmpath_rd 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kaluad 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kaluad 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kaluad 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kaluad 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:mld 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:mld 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:mld 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:mld 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-04-21T09:10:05.281 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:ipv6_addr 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:ipv6_addr 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:ipv6_addr 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:ipv6_addr 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 120 root cwd unknown /proc/120/cwd (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 120 root rtd unknown /proc/120/root (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 120 root txt unknown /proc/120/exe (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 120 root NOFD /proc/120/fd (opendir: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kstrp 123 root cwd unknown /proc/123/cwd (readlink: Permission denied) 2024-04-21T09:10:05.282 INFO:tasks.workunit.client.0.smithi066.stdout:kstrp 123 root rtd unknown /proc/123/root (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kstrp 123 root txt unknown /proc/123/exe (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kstrp 123 root NOFD /proc/123/fd (opendir: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 126 root cwd unknown /proc/126/cwd (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 126 root rtd unknown /proc/126/root (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 126 root txt unknown /proc/126/exe (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 126 root NOFD /proc/126/fd (opendir: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:zswap-shr 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:zswap-shr 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:zswap-shr 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:zswap-shr 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 132 root cwd unknown /proc/132/cwd (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 132 root rtd unknown /proc/132/root (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 132 root txt unknown /proc/132/exe (readlink: Permission denied) 2024-04-21T09:10:05.283 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 132 root NOFD /proc/132/fd (opendir: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 241 root cwd unknown /proc/241/cwd (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 241 root rtd unknown /proc/241/root (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 241 root txt unknown /proc/241/exe (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 241 root NOFD /proc/241/fd (opendir: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 279 root cwd unknown /proc/279/cwd (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 279 root rtd unknown /proc/279/root (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 279 root txt unknown /proc/279/exe (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 279 root NOFD /proc/279/fd (opendir: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 281 root cwd unknown /proc/281/cwd (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 281 root rtd unknown /proc/281/root (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 281 root txt unknown /proc/281/exe (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 281 root NOFD /proc/281/fd (opendir: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 283 root cwd unknown /proc/283/cwd (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 283 root rtd unknown /proc/283/root (readlink: Permission denied) 2024-04-21T09:10:05.284 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 283 root txt unknown /proc/283/exe (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 283 root NOFD /proc/283/fd (opendir: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 293 root cwd unknown /proc/293/cwd (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 293 root rtd unknown /proc/293/root (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 293 root txt unknown /proc/293/exe (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 293 root NOFD /proc/293/fd (opendir: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 378 root cwd unknown /proc/378/cwd (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 378 root rtd unknown /proc/378/root (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 378 root txt unknown /proc/378/exe (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 378 root NOFD /proc/378/fd (opendir: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-wq 427 root cwd unknown /proc/427/cwd (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-wq 427 root rtd unknown /proc/427/root (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-wq 427 root txt unknown /proc/427/exe (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-wq 427 root NOFD /proc/427/fd (opendir: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-rese 428 root cwd unknown /proc/428/cwd (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-rese 428 root rtd unknown /proc/428/root (readlink: Permission denied) 2024-04-21T09:10:05.285 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-rese 428 root txt unknown /proc/428/exe (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-rese 428 root NOFD /proc/428/fd (opendir: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:ata_sff 429 root cwd unknown /proc/429/cwd (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:ata_sff 429 root rtd unknown /proc/429/root (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:ata_sff 429 root txt unknown /proc/429/exe (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:ata_sff 429 root NOFD /proc/429/fd (opendir: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-dele 437 root cwd unknown /proc/437/cwd (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-dele 437 root rtd unknown /proc/437/root (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-dele 437 root txt unknown /proc/437/exe (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-dele 437 root NOFD /proc/437/fd (opendir: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-auth 438 root cwd unknown /proc/438/cwd (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-auth 438 root rtd unknown /proc/438/root (readlink: Permission denied) 2024-04-21T09:10:05.286 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-auth 438 root txt unknown /proc/438/exe (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:nvme-auth 438 root NOFD /proc/438/fd (opendir: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:ixgbe 440 root cwd unknown /proc/440/cwd (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:ixgbe 440 root rtd unknown /proc/440/root (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:ixgbe 440 root txt unknown /proc/440/exe (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:ixgbe 440 root NOFD /proc/440/fd (opendir: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_0 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_0 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_0 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_0 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-04-21T09:10:05.287 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_1 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_1 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_1 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_1 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_2 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_2 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_2 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_2 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-04-21T09:10:05.288 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_3 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_3 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_3 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_3 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_4 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_4 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_4 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_4 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-04-21T09:10:05.289 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_5 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_5 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_5 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_5 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_6 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_6 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_6 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-04-21T09:10:05.290 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_6 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_7 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_7 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_7 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_7 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_8 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_8 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-04-21T09:10:05.291 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_8 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_8 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_9 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_9 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_9 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_eh_9 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:scsi_tmf_ 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-04-21T09:10:05.292 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 494 root cwd unknown /proc/494/cwd (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 494 root rtd unknown /proc/494/root (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 494 root txt unknown /proc/494/exe (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 494 root NOFD /proc/494/fd (opendir: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:jbd2/sda1 497 root cwd unknown /proc/497/cwd (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:jbd2/sda1 497 root rtd unknown /proc/497/root (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:jbd2/sda1 497 root txt unknown /proc/497/exe (readlink: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:jbd2/sda1 497 root NOFD /proc/497/fd (opendir: Permission denied) 2024-04-21T09:10:05.293 INFO:tasks.workunit.client.0.smithi066.stdout:ext4-rsv- 498 root cwd unknown /proc/498/cwd (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:ext4-rsv- 498 root rtd unknown /proc/498/root (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:ext4-rsv- 498 root txt unknown /proc/498/exe (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:ext4-rsv- 498 root NOFD /proc/498/fd (opendir: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-j 573 root cwd unknown /proc/573/cwd (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-j 573 root rtd unknown /proc/573/root (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-j 573 root txt unknown /proc/573/exe (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-j 573 root NOFD /proc/573/fd (opendir: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:ext4lazyi 579 root cwd unknown /proc/579/cwd (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:ext4lazyi 579 root rtd unknown /proc/579/root (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:ext4lazyi 579 root txt unknown /proc/579/exe (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:ext4lazyi 579 root NOFD /proc/579/fd (opendir: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-u 594 root cwd unknown /proc/594/cwd (readlink: Permission denied) 2024-04-21T09:10:05.294 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-u 594 root rtd unknown /proc/594/root (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-u 594 root txt unknown /proc/594/exe (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-u 594 root NOFD /proc/594/fd (opendir: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:ipmi-msgh 657 root cwd unknown /proc/657/cwd (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:ipmi-msgh 657 root rtd unknown /proc/657/root (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:ipmi-msgh 657 root txt unknown /proc/657/exe (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:ipmi-msgh 657 root NOFD /proc/657/fd (opendir: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:rpcbind 667 rpc cwd unknown /proc/667/cwd (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:rpcbind 667 rpc rtd unknown /proc/667/root (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:rpcbind 667 rpc txt unknown /proc/667/exe (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:rpcbind 667 rpc NOFD /proc/667/fd (opendir: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:kipmi0 669 root cwd unknown /proc/669/cwd (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:kipmi0 669 root rtd unknown /proc/669/root (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:kipmi0 669 root txt unknown /proc/669/exe (readlink: Permission denied) 2024-04-21T09:10:05.295 INFO:tasks.workunit.client.0.smithi066.stdout:kipmi0 669 root NOFD /proc/669/fd (opendir: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 root cwd unknown /proc/670/cwd (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 root rtd unknown /proc/670/root (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 root txt unknown /proc/670/exe (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 root NOFD /proc/670/fd (opendir: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 671 auditd root cwd unknown /proc/670/task/671/cwd (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 671 auditd root rtd unknown /proc/670/task/671/root (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 671 auditd root txt unknown /proc/670/task/671/exe (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 671 auditd root NOFD /proc/670/task/671/fd (opendir: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 673 auditd root cwd unknown /proc/670/task/673/cwd (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 673 auditd root rtd unknown /proc/670/task/673/root (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 673 auditd root txt unknown /proc/670/task/673/exe (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:auditd 670 673 auditd root NOFD /proc/670/task/673/fd (opendir: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:sedispatc 672 root cwd unknown /proc/672/cwd (readlink: Permission denied) 2024-04-21T09:10:05.296 INFO:tasks.workunit.client.0.smithi066.stdout:sedispatc 672 root rtd unknown /proc/672/root (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:sedispatc 672 root txt unknown /proc/672/exe (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:sedispatc 672 root NOFD /proc/672/fd (opendir: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:rpciod 674 root cwd unknown /proc/674/cwd (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:rpciod 674 root rtd unknown /proc/674/root (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:rpciod 674 root txt unknown /proc/674/exe (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:rpciod 674 root NOFD /proc/674/fd (opendir: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:xprtiod 675 root cwd unknown /proc/675/cwd (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:xprtiod 675 root rtd unknown /proc/675/root (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:xprtiod 675 root txt unknown /proc/675/exe (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:xprtiod 675 root NOFD /proc/675/fd (opendir: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 716 dbus cwd unknown /proc/716/cwd (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 716 dbus rtd unknown /proc/716/root (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 716 dbus txt unknown /proc/716/exe (readlink: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 716 dbus NOFD /proc/716/fd (opendir: Permission denied) 2024-04-21T09:10:05.297 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 717 dbus cwd unknown /proc/717/cwd (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 717 dbus rtd unknown /proc/717/root (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 717 dbus txt unknown /proc/717/exe (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:dbus-brok 717 dbus NOFD /proc/717/fd (opendir: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 root cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 root rtd unknown /proc/718/root (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 root txt unknown /proc/718/exe (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 root NOFD /proc/718/fd (opendir: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 759 gmain root cwd unknown /proc/718/task/759/cwd (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 759 gmain root rtd unknown /proc/718/task/759/root (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 759 gmain root txt unknown /proc/718/task/759/exe (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 759 gmain root NOFD /proc/718/task/759/fd (opendir: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 760 gdbus root cwd unknown /proc/718/task/760/cwd (readlink: Permission denied) 2024-04-21T09:10:05.298 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 760 gdbus root rtd unknown /proc/718/task/760/root (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 760 gdbus root txt unknown /proc/718/task/760/exe (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:NetworkMa 718 760 gdbus root NOFD /proc/718/task/760/fd (opendir: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 root cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 root rtd unknown /proc/721/root (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 root txt unknown /proc/721/exe (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 root NOFD /proc/721/fd (opendir: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 725 gmain root cwd unknown /proc/721/task/725/cwd (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 725 gmain root rtd unknown /proc/721/task/725/root (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 725 gmain root txt unknown /proc/721/task/725/exe (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:irqbalanc 721 725 gmain root NOFD /proc/721/task/725/fd (opendir: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:lsmd 722 libstoragemgmt cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:lsmd 722 libstoragemgmt rtd unknown /proc/722/root (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:lsmd 722 libstoragemgmt txt unknown /proc/722/exe (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:lsmd 722 libstoragemgmt NOFD /proc/722/fd (opendir: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:mcelog 723 root cwd unknown /proc/723/cwd (readlink: Permission denied) 2024-04-21T09:10:05.299 INFO:tasks.workunit.client.0.smithi066.stdout:mcelog 723 root rtd unknown /proc/723/root (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:mcelog 723 root txt unknown /proc/723/exe (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:mcelog 723 root NOFD /proc/723/fd (opendir: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:smartd 724 root cwd unknown /proc/724/cwd (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:smartd 724 root rtd unknown /proc/724/root (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:smartd 724 root txt unknown /proc/724/exe (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:smartd 724 root NOFD /proc/724/fd (opendir: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-l 730 root cwd unknown /proc/730/cwd (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-l 730 root rtd unknown /proc/730/root (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-l 730 root txt unknown /proc/730/exe (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:systemd-l 730 root NOFD /proc/730/fd (opendir: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 791 root cwd unknown /proc/791/cwd (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 791 root rtd unknown /proc/791/root (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 791 root txt unknown /proc/791/exe (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 791 root NOFD /proc/791/fd (opendir: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 root cwd unknown /proc/799/cwd (readlink: Permission denied) 2024-04-21T09:10:05.300 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 root rtd unknown /proc/799/root (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 root txt unknown /proc/799/exe (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 root NOFD /proc/799/fd (opendir: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 800 gssproxy root cwd unknown /proc/799/task/800/cwd (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 800 gssproxy root rtd unknown /proc/799/task/800/root (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 800 gssproxy root txt unknown /proc/799/task/800/exe (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 800 gssproxy root NOFD /proc/799/task/800/fd (opendir: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 801 gssproxy root cwd unknown /proc/799/task/801/cwd (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 801 gssproxy root rtd unknown /proc/799/task/801/root (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 801 gssproxy root txt unknown /proc/799/task/801/exe (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 801 gssproxy root NOFD /proc/799/task/801/fd (opendir: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 802 gssproxy root cwd unknown /proc/799/task/802/cwd (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 802 gssproxy root rtd unknown /proc/799/task/802/root (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 802 gssproxy root txt unknown /proc/799/task/802/exe (readlink: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 802 gssproxy root NOFD /proc/799/task/802/fd (opendir: Permission denied) 2024-04-21T09:10:05.301 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 803 gssproxy root cwd unknown /proc/799/task/803/cwd (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 803 gssproxy root rtd unknown /proc/799/task/803/root (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 803 gssproxy root txt unknown /proc/799/task/803/exe (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 803 gssproxy root NOFD /proc/799/task/803/fd (opendir: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 804 gssproxy root cwd unknown /proc/799/task/804/cwd (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 804 gssproxy root rtd unknown /proc/799/task/804/root (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 804 gssproxy root txt unknown /proc/799/task/804/exe (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:gssproxy 799 804 gssproxy root NOFD /proc/799/task/804/fd (opendir: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:atd 806 root cwd unknown /proc/806/cwd (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:atd 806 root rtd unknown /proc/806/root (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:atd 806 root txt unknown /proc/806/exe (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:atd 806 root NOFD /proc/806/fd (opendir: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:crond 808 root cwd unknown /proc/808/cwd (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:crond 808 root rtd unknown /proc/808/root (readlink: Permission denied) 2024-04-21T09:10:05.302 INFO:tasks.workunit.client.0.smithi066.stdout:crond 808 root txt unknown /proc/808/exe (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:crond 808 root NOFD /proc/808/fd (opendir: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:nrpe 830 nrpe cwd unknown /proc/830/cwd (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:nrpe 830 nrpe rtd unknown /proc/830/root (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:nrpe 830 nrpe txt unknown /proc/830/exe (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:nrpe 830 nrpe NOFD /proc/830/fd (opendir: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5185 root cwd unknown /proc/5185/cwd (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5185 root rtd unknown /proc/5185/root (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5185 root txt unknown /proc/5185/exe (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5185 root NOFD /proc/5185/fd (opendir: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 5186 root cwd unknown /proc/5186/cwd (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 5186 root rtd unknown /proc/5186/root (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 5186 root txt unknown /proc/5186/exe (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 5186 root NOFD /proc/5186/fd (opendir: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5187 root cwd unknown /proc/5187/cwd (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5187 root rtd unknown /proc/5187/root (readlink: Permission denied) 2024-04-21T09:10:05.303 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5187 root txt unknown /proc/5187/exe (readlink: Permission denied) 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:agetty 5187 root NOFD /proc/5187/fd (opendir: Permission denied) 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu cwd DIR 8,1 4096 2 / 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu rtd DIR 8,1 4096 2 / 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu DEL REG 8,1 596 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu DEL REG 8,1 62167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-21T09:10:05.304 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-21T09:10:05.305 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 1u unix 0x0000000000000000 0t0 23331 type=STREAM (CONNECTED) 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 2u unix 0x0000000000000000 0t0 23331 type=STREAM (CONNECTED) 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 3u unix 0x0000000000000000 0t0 23359 type=DGRAM (CONNECTED) 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 4u a_inode 0,14 0 13316 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 5u a_inode 0,14 0 13316 [signalfd] 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 6r a_inode 0,14 0 13316 inotify 2024-04-21T09:10:05.306 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 7r DIR 0,25 0 3609 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 8u a_inode 0,14 0 13316 [timerfd] 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 9u a_inode 0,14 0 13316 [eventpoll:10,13] 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 10r REG 0,20 0 23348 /proc/5195/mountinfo 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 11r a_inode 0,14 0 13316 inotify 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 12u unix 0x0000000000000000 0t0 23381 /run/user/1000/bus type=STREAM (LISTEN) 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 13r a_inode 0,14 0 13316 inotify 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 15u netlink 0t0 23363 KOBJECT_UEVENT 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 16u unix 0x0000000000000000 0t0 23368 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 17u unix 0x0000000000000000 0t0 23369 type=DGRAM (CONNECTED) 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 18u unix 0x0000000000000000 0t0 23370 type=DGRAM (CONNECTED) 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 19u unix 0x0000000000000000 0t0 23371 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 20u unix 0x0000000000000000 0t0 23373 type=STREAM (CONNECTED) 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 5195 ubuntu 21u a_inode 0,14 0 13316 [timerfd] 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 5197 ubuntu cwd unknown /proc/5197/cwd (readlink: Permission denied) 2024-04-21T09:10:05.307 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 5197 ubuntu rtd unknown /proc/5197/root (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 5197 ubuntu txt unknown /proc/5197/exe (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 5197 ubuntu NOFD /proc/5197/fd (opendir: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:tls-strp 5548 root cwd unknown /proc/5548/cwd (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:tls-strp 5548 root rtd unknown /proc/5548/root (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:tls-strp 5548 root txt unknown /proc/5548/exe (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:tls-strp 5548 root NOFD /proc/5548/fd (opendir: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 polkitd cwd unknown /proc/5633/cwd (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 polkitd rtd unknown /proc/5633/root (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 polkitd txt unknown /proc/5633/exe (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 polkitd NOFD /proc/5633/fd (opendir: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5635 gmain polkitd cwd unknown /proc/5633/task/5635/cwd (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5635 gmain polkitd rtd unknown /proc/5633/task/5635/root (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5635 gmain polkitd txt unknown /proc/5633/task/5635/exe (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5635 gmain polkitd NOFD /proc/5633/task/5635/fd (opendir: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5636 gdbus polkitd cwd unknown /proc/5633/task/5636/cwd (readlink: Permission denied) 2024-04-21T09:10:05.308 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5636 gdbus polkitd rtd unknown /proc/5633/task/5636/root (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5636 gdbus polkitd txt unknown /proc/5633/task/5636/exe (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5636 gdbus polkitd NOFD /proc/5633/task/5636/fd (opendir: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5637 JS\x20Hel polkitd cwd unknown /proc/5633/task/5637/cwd (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5637 JS\x20Hel polkitd rtd unknown /proc/5633/task/5637/root (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5637 JS\x20Hel polkitd txt unknown /proc/5633/task/5637/exe (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5637 JS\x20Hel polkitd NOFD /proc/5633/task/5637/fd (opendir: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5638 JS\x20Hel polkitd cwd unknown /proc/5633/task/5638/cwd (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5638 JS\x20Hel polkitd rtd unknown /proc/5633/task/5638/root (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5638 JS\x20Hel polkitd txt unknown /proc/5633/task/5638/exe (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5638 JS\x20Hel polkitd NOFD /proc/5633/task/5638/fd (opendir: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5639 JS\x20Hel polkitd cwd unknown /proc/5633/task/5639/cwd (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5639 JS\x20Hel polkitd rtd unknown /proc/5633/task/5639/root (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5639 JS\x20Hel polkitd txt unknown /proc/5633/task/5639/exe (readlink: Permission denied) 2024-04-21T09:10:05.309 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5639 JS\x20Hel polkitd NOFD /proc/5633/task/5639/fd (opendir: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5640 JS\x20Hel polkitd cwd unknown /proc/5633/task/5640/cwd (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5640 JS\x20Hel polkitd rtd unknown /proc/5633/task/5640/root (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5640 JS\x20Hel polkitd txt unknown /proc/5633/task/5640/exe (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5640 JS\x20Hel polkitd NOFD /proc/5633/task/5640/fd (opendir: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5641 JS\x20Hel polkitd cwd unknown /proc/5633/task/5641/cwd (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5641 JS\x20Hel polkitd rtd unknown /proc/5633/task/5641/root (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5641 JS\x20Hel polkitd txt unknown /proc/5633/task/5641/exe (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5641 JS\x20Hel polkitd NOFD /proc/5633/task/5641/fd (opendir: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5642 JS\x20Hel polkitd cwd unknown /proc/5633/task/5642/cwd (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5642 JS\x20Hel polkitd rtd unknown /proc/5633/task/5642/root (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5642 JS\x20Hel polkitd txt unknown /proc/5633/task/5642/exe (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5642 JS\x20Hel polkitd NOFD /proc/5633/task/5642/fd (opendir: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5643 JS\x20Hel polkitd cwd unknown /proc/5633/task/5643/cwd (readlink: Permission denied) 2024-04-21T09:10:05.310 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5643 JS\x20Hel polkitd rtd unknown /proc/5633/task/5643/root (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5643 JS\x20Hel polkitd txt unknown /proc/5633/task/5643/exe (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5643 JS\x20Hel polkitd NOFD /proc/5633/task/5643/fd (opendir: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5644 JS\x20Hel polkitd cwd unknown /proc/5633/task/5644/cwd (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5644 JS\x20Hel polkitd rtd unknown /proc/5633/task/5644/root (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5644 JS\x20Hel polkitd txt unknown /proc/5633/task/5644/exe (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5644 JS\x20Hel polkitd NOFD /proc/5633/task/5644/fd (opendir: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5645 polkitd polkitd cwd unknown /proc/5633/task/5645/cwd (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5645 polkitd polkitd rtd unknown /proc/5633/task/5645/root (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5645 polkitd polkitd txt unknown /proc/5633/task/5645/exe (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:polkitd 5633 5645 polkitd polkitd NOFD /proc/5633/task/5645/fd (opendir: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 root cwd unknown /proc/5863/cwd (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 root rtd unknown /proc/5863/root (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 root txt unknown /proc/5863/exe (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 root NOFD /proc/5863/fd (opendir: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5865 in:imjour root cwd unknown /proc/5863/task/5865/cwd (readlink: Permission denied) 2024-04-21T09:10:05.311 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5865 in:imjour root rtd unknown /proc/5863/task/5865/root (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5865 in:imjour root txt unknown /proc/5863/task/5865/exe (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5865 in:imjour root NOFD /proc/5863/task/5865/fd (opendir: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5866 rs:main root cwd unknown /proc/5863/task/5866/cwd (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5866 rs:main root rtd unknown /proc/5863/task/5866/root (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5866 rs:main root txt unknown /proc/5863/task/5866/exe (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:rsyslogd 5863 5866 rs:main root NOFD /proc/5863/task/5866/fd (opendir: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19258 root cwd unknown /proc/19258/cwd (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19258 root rtd unknown /proc/19258/root (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19258 root txt unknown /proc/19258/exe (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19258 root NOFD /proc/19258/fd (opendir: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19287 root cwd unknown /proc/19287/cwd (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19287 root rtd unknown /proc/19287/root (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19287 root txt unknown /proc/19287/exe (readlink: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19287 root NOFD /proc/19287/fd (opendir: Permission denied) 2024-04-21T09:10:05.312 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19316 root cwd unknown /proc/19316/cwd (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19316 root rtd unknown /proc/19316/root (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19316 root txt unknown /proc/19316/exe (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19316 root NOFD /proc/19316/fd (opendir: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19345 root cwd unknown /proc/19345/cwd (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19345 root rtd unknown /proc/19345/root (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19345 root txt unknown /proc/19345/exe (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19345 root NOFD /proc/19345/fd (opendir: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19374 root cwd unknown /proc/19374/cwd (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19374 root rtd unknown /proc/19374/root (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19374 root txt unknown /proc/19374/exe (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:kdmflush/ 19374 root NOFD /proc/19374/fd (opendir: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:xfsalloc 19538 root cwd unknown /proc/19538/cwd (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:xfsalloc 19538 root rtd unknown /proc/19538/root (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:xfsalloc 19538 root txt unknown /proc/19538/exe (readlink: Permission denied) 2024-04-21T09:10:05.313 INFO:tasks.workunit.client.0.smithi066.stdout:xfsalloc 19538 root NOFD /proc/19538/fd (opendir: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs_mru_c 19539 root cwd unknown /proc/19539/cwd (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs_mru_c 19539 root rtd unknown /proc/19539/root (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs_mru_c 19539 root txt unknown /proc/19539/exe (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs_mru_c 19539 root NOFD /proc/19539/fd (opendir: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-buf/d 19540 root cwd unknown /proc/19540/cwd (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-buf/d 19540 root rtd unknown /proc/19540/root (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-buf/d 19540 root txt unknown /proc/19540/exe (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-buf/d 19540 root NOFD /proc/19540/fd (opendir: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-conv/ 19541 root cwd unknown /proc/19541/cwd (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-conv/ 19541 root rtd unknown /proc/19541/root (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-conv/ 19541 root txt unknown /proc/19541/exe (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-conv/ 19541 root NOFD /proc/19541/fd (opendir: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-recla 19542 root cwd unknown /proc/19542/cwd (readlink: Permission denied) 2024-04-21T09:10:05.314 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-recla 19542 root rtd unknown /proc/19542/root (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-recla 19542 root txt unknown /proc/19542/exe (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-recla 19542 root NOFD /proc/19542/fd (opendir: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-block 19543 root cwd unknown /proc/19543/cwd (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-block 19543 root rtd unknown /proc/19543/root (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-block 19543 root txt unknown /proc/19543/exe (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-block 19543 root NOFD /proc/19543/fd (opendir: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-inode 19544 root cwd unknown /proc/19544/cwd (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-inode 19544 root rtd unknown /proc/19544/root (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-inode 19544 root txt unknown /proc/19544/exe (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-inode 19544 root NOFD /proc/19544/fd (opendir: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-log/d 19545 root cwd unknown /proc/19545/cwd (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-log/d 19545 root rtd unknown /proc/19545/root (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-log/d 19545 root txt unknown /proc/19545/exe (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-log/d 19545 root NOFD /proc/19545/fd (opendir: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-cil/d 19546 root cwd unknown /proc/19546/cwd (readlink: Permission denied) 2024-04-21T09:10:05.315 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-cil/d 19546 root rtd unknown /proc/19546/root (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-cil/d 19546 root txt unknown /proc/19546/exe (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:xfs-cil/d 19546 root NOFD /proc/19546/fd (opendir: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:xfsaild/d 19547 root cwd unknown /proc/19547/cwd (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:xfsaild/d 19547 root rtd unknown /proc/19547/root (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:xfsaild/d 19547 root txt unknown /proc/19547/exe (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:xfsaild/d 19547 root NOFD /proc/19547/fd (opendir: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20475 root cwd unknown /proc/20475/cwd (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20475 root rtd unknown /proc/20475/root (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20475 root txt unknown /proc/20475/exe (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20475 root NOFD /proc/20475/fd (opendir: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20478 ubuntu cwd unknown /proc/20478/cwd (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20478 ubuntu rtd unknown /proc/20478/root (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20478 ubuntu txt unknown /proc/20478/exe (readlink: Permission denied) 2024-04-21T09:10:05.316 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 20478 ubuntu NOFD /proc/20478/fd (opendir: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:chronyd 20539 chrony cwd unknown /proc/20539/cwd (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:chronyd 20539 chrony rtd unknown /proc/20539/root (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:chronyd 20539 chrony txt unknown /proc/20539/exe (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:chronyd 20539 chrony NOFD /proc/20539/fd (opendir: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-cras 20828 ceph cwd unknown /proc/20828/cwd (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-cras 20828 ceph rtd unknown /proc/20828/root (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-cras 20828 ceph txt unknown /proc/20828/exe (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-cras 20828 ceph NOFD /proc/20828/fd (opendir: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 21461 root cwd unknown /proc/21461/cwd (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 21461 root rtd unknown /proc/21461/root (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 21461 root txt unknown /proc/21461/exe (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 21461 root NOFD /proc/21461/fd (opendir: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26935 root cwd unknown /proc/26935/cwd (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26935 root rtd unknown /proc/26935/root (readlink: Permission denied) 2024-04-21T09:10:05.317 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26935 root txt unknown /proc/26935/exe (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26935 root NOFD /proc/26935/fd (opendir: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26952 root cwd unknown /proc/26952/cwd (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26952 root rtd unknown /proc/26952/root (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26952 root txt unknown /proc/26952/exe (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26952 root NOFD /proc/26952/fd (opendir: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26953 root cwd unknown /proc/26953/cwd (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26953 root rtd unknown /proc/26953/root (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26953 root txt unknown /proc/26953/exe (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 26953 root NOFD /proc/26953/fd (opendir: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26970 root cwd unknown /proc/26970/cwd (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26970 root rtd unknown /proc/26970/root (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26970 root txt unknown /proc/26970/exe (readlink: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 26970 root NOFD /proc/26970/fd (opendir: Permission denied) 2024-04-21T09:10:05.318 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 27754 root cwd unknown /proc/27754/cwd (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 27754 root rtd unknown /proc/27754/root (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 27754 root txt unknown /proc/27754/exe (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 27754 root NOFD /proc/27754/fd (opendir: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 27949 root cwd unknown /proc/27949/cwd (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 27949 root rtd unknown /proc/27949/root (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 27949 root txt unknown /proc/27949/exe (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 27949 root NOFD /proc/27949/fd (opendir: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 27951 root cwd unknown /proc/27951/cwd (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 27951 root rtd unknown /proc/27951/root (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 27951 root txt unknown /proc/27951/exe (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 27951 root NOFD /proc/27951/fd (opendir: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 ceph cwd unknown /proc/27953/cwd (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 ceph rtd unknown /proc/27953/root (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 ceph txt unknown /proc/27953/exe (readlink: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 ceph NOFD /proc/27953/fd (opendir: Permission denied) 2024-04-21T09:10:05.319 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27954 log ceph cwd unknown /proc/27953/task/27954/cwd (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27954 log ceph rtd unknown /proc/27953/task/27954/root (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27954 log ceph txt unknown /proc/27953/task/27954/exe (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27954 log ceph NOFD /proc/27953/task/27954/fd (opendir: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27955 service ceph cwd unknown /proc/27953/task/27955/cwd (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27955 service ceph rtd unknown /proc/27953/task/27955/root (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27955 service ceph txt unknown /proc/27953/task/27955/exe (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27955 service ceph NOFD /proc/27953/task/27955/fd (opendir: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27956 admin_soc ceph cwd unknown /proc/27953/task/27956/cwd (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27956 admin_soc ceph rtd unknown /proc/27953/task/27956/root (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27956 admin_soc ceph txt unknown /proc/27953/task/27956/exe (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27956 admin_soc ceph NOFD /proc/27953/task/27956/fd (opendir: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27957 signal_ha ceph cwd unknown /proc/27953/task/27957/cwd (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27957 signal_ha ceph rtd unknown /proc/27953/task/27957/root (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27957 signal_ha ceph txt unknown /proc/27953/task/27957/exe (readlink: Permission denied) 2024-04-21T09:10:05.320 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27957 signal_ha ceph NOFD /proc/27953/task/27957/fd (opendir: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27958 rocksdb:l ceph cwd unknown /proc/27953/task/27958/cwd (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27958 rocksdb:l ceph rtd unknown /proc/27953/task/27958/root (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27958 rocksdb:l ceph txt unknown /proc/27953/task/27958/exe (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27958 rocksdb:l ceph NOFD /proc/27953/task/27958/fd (opendir: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27959 rocksdb:h ceph cwd unknown /proc/27953/task/27959/cwd (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27959 rocksdb:h ceph rtd unknown /proc/27953/task/27959/root (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27959 rocksdb:h ceph txt unknown /proc/27953/task/27959/exe (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27959 rocksdb:h ceph NOFD /proc/27953/task/27959/fd (opendir: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27975 ceph-mon ceph cwd unknown /proc/27953/task/27975/cwd (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27975 ceph-mon ceph rtd unknown /proc/27953/task/27975/root (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27975 ceph-mon ceph txt unknown /proc/27953/task/27975/exe (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27975 ceph-mon ceph NOFD /proc/27953/task/27975/fd (opendir: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27976 fn_monsto ceph cwd unknown /proc/27953/task/27976/cwd (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27976 fn_monsto ceph rtd unknown /proc/27953/task/27976/root (readlink: Permission denied) 2024-04-21T09:10:05.321 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27976 fn_monsto ceph txt unknown /proc/27953/task/27976/exe (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27976 fn_monsto ceph NOFD /proc/27953/task/27976/fd (opendir: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27977 msgr-work ceph cwd unknown /proc/27953/task/27977/cwd (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27977 msgr-work ceph rtd unknown /proc/27953/task/27977/root (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27977 msgr-work ceph txt unknown /proc/27953/task/27977/exe (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27977 msgr-work ceph NOFD /proc/27953/task/27977/fd (opendir: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27978 msgr-work ceph cwd unknown /proc/27953/task/27978/cwd (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27978 msgr-work ceph rtd unknown /proc/27953/task/27978/root (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27978 msgr-work ceph txt unknown /proc/27953/task/27978/exe (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27978 msgr-work ceph NOFD /proc/27953/task/27978/fd (opendir: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27979 msgr-work ceph cwd unknown /proc/27953/task/27979/cwd (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27979 msgr-work ceph rtd unknown /proc/27953/task/27979/root (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27979 msgr-work ceph txt unknown /proc/27953/task/27979/exe (readlink: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27979 msgr-work ceph NOFD /proc/27953/task/27979/fd (opendir: Permission denied) 2024-04-21T09:10:05.322 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27980 OpHistory ceph cwd unknown /proc/27953/task/27980/cwd (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27980 OpHistory ceph rtd unknown /proc/27953/task/27980/root (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27980 OpHistory ceph txt unknown /proc/27953/task/27980/exe (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27980 OpHistory ceph NOFD /proc/27953/task/27980/fd (opendir: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27981 fin ceph cwd unknown /proc/27953/task/27981/cwd (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27981 fin ceph rtd unknown /proc/27953/task/27981/root (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27981 fin ceph txt unknown /proc/27953/task/27981/exe (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27981 fin ceph NOFD /proc/27953/task/27981/fd (opendir: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27982 safe_time ceph cwd unknown /proc/27953/task/27982/cwd (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27982 safe_time ceph rtd unknown /proc/27953/task/27982/root (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27982 safe_time ceph txt unknown /proc/27953/task/27982/exe (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27982 safe_time ceph NOFD /proc/27953/task/27982/fd (opendir: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27983 cpu_tp ceph cwd unknown /proc/27953/task/27983/cwd (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27983 cpu_tp ceph rtd unknown /proc/27953/task/27983/root (readlink: Permission denied) 2024-04-21T09:10:05.323 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27983 cpu_tp ceph txt unknown /proc/27953/task/27983/exe (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27983 cpu_tp ceph NOFD /proc/27953/task/27983/fd (opendir: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27984 cpu_tp ceph cwd unknown /proc/27953/task/27984/cwd (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27984 cpu_tp ceph rtd unknown /proc/27953/task/27984/root (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27984 cpu_tp ceph txt unknown /proc/27953/task/27984/exe (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27984 cpu_tp ceph NOFD /proc/27953/task/27984/fd (opendir: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27985 cpu_tp ceph cwd unknown /proc/27953/task/27985/cwd (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27985 cpu_tp ceph rtd unknown /proc/27953/task/27985/root (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27985 cpu_tp ceph txt unknown /proc/27953/task/27985/exe (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27985 cpu_tp ceph NOFD /proc/27953/task/27985/fd (opendir: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27986 cpu_tp ceph cwd unknown /proc/27953/task/27986/cwd (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27986 cpu_tp ceph rtd unknown /proc/27953/task/27986/root (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27986 cpu_tp ceph txt unknown /proc/27953/task/27986/exe (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27986 cpu_tp ceph NOFD /proc/27953/task/27986/fd (opendir: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27987 ms_dispat ceph cwd unknown /proc/27953/task/27987/cwd (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27987 ms_dispat ceph rtd unknown /proc/27953/task/27987/root (readlink: Permission denied) 2024-04-21T09:10:05.324 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27987 ms_dispat ceph txt unknown /proc/27953/task/27987/exe (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27987 ms_dispat ceph NOFD /proc/27953/task/27987/fd (opendir: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27988 ms_local ceph cwd unknown /proc/27953/task/27988/cwd (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27988 ms_local ceph rtd unknown /proc/27953/task/27988/root (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27988 ms_local ceph txt unknown /proc/27953/task/27988/exe (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27988 ms_local ceph NOFD /proc/27953/task/27988/fd (opendir: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27989 safe_time ceph cwd unknown /proc/27953/task/27989/cwd (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27989 safe_time ceph rtd unknown /proc/27953/task/27989/root (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27989 safe_time ceph txt unknown /proc/27953/task/27989/exe (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27989 safe_time ceph NOFD /proc/27953/task/27989/fd (opendir: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27990 ms_dispat ceph cwd unknown /proc/27953/task/27990/cwd (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27990 ms_dispat ceph rtd unknown /proc/27953/task/27990/root (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27990 ms_dispat ceph txt unknown /proc/27953/task/27990/exe (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27990 ms_dispat ceph NOFD /proc/27953/task/27990/fd (opendir: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27991 ms_local ceph cwd unknown /proc/27953/task/27991/cwd (readlink: Permission denied) 2024-04-21T09:10:05.325 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27991 ms_local ceph rtd unknown /proc/27953/task/27991/root (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27991 ms_local ceph txt unknown /proc/27953/task/27991/exe (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 27991 ms_local ceph NOFD /proc/27953/task/27991/fd (opendir: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 64585 rstore_co ceph cwd unknown /proc/27953/task/64585/cwd (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 64585 rstore_co ceph rtd unknown /proc/27953/task/64585/root (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 64585 rstore_co ceph txt unknown /proc/27953/task/64585/exe (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mon 27953 64585 rstore_co ceph NOFD /proc/27953/task/64585/fd (opendir: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28135 root cwd unknown /proc/28135/cwd (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28135 root rtd unknown /proc/28135/root (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28135 root txt unknown /proc/28135/exe (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28135 root NOFD /proc/28135/fd (opendir: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 28187 root cwd unknown /proc/28187/cwd (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 28187 root rtd unknown /proc/28187/root (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 28187 root txt unknown /proc/28187/exe (readlink: Permission denied) 2024-04-21T09:10:05.326 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 28187 root NOFD /proc/28187/fd (opendir: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 28189 root cwd unknown /proc/28189/cwd (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 28189 root rtd unknown /proc/28189/root (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 28189 root txt unknown /proc/28189/exe (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 28189 root NOFD /proc/28189/fd (opendir: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 ceph cwd unknown /proc/28191/cwd (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 ceph rtd unknown /proc/28191/root (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 ceph txt unknown /proc/28191/exe (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 ceph NOFD /proc/28191/fd (opendir: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78823 log ceph cwd unknown /proc/28191/task/78823/cwd (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78823 log ceph rtd unknown /proc/28191/task/78823/root (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78823 log ceph txt unknown /proc/28191/task/78823/exe (readlink: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78823 log ceph NOFD /proc/28191/task/78823/fd (opendir: Permission denied) 2024-04-21T09:10:05.327 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78825 msgr-work ceph cwd unknown /proc/28191/task/78825/cwd (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78825 msgr-work ceph rtd unknown /proc/28191/task/78825/root (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78825 msgr-work ceph txt unknown /proc/28191/task/78825/exe (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78825 msgr-work ceph NOFD /proc/28191/task/78825/fd (opendir: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78826 msgr-work ceph cwd unknown /proc/28191/task/78826/cwd (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78826 msgr-work ceph rtd unknown /proc/28191/task/78826/root (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78826 msgr-work ceph txt unknown /proc/28191/task/78826/exe (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78826 msgr-work ceph NOFD /proc/28191/task/78826/fd (opendir: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78827 msgr-work ceph cwd unknown /proc/28191/task/78827/cwd (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78827 msgr-work ceph rtd unknown /proc/28191/task/78827/root (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78827 msgr-work ceph txt unknown /proc/28191/task/78827/exe (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78827 msgr-work ceph NOFD /proc/28191/task/78827/fd (opendir: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78831 service ceph cwd unknown /proc/28191/task/78831/cwd (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78831 service ceph rtd unknown /proc/28191/task/78831/root (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78831 service ceph txt unknown /proc/28191/task/78831/exe (readlink: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78831 service ceph NOFD /proc/28191/task/78831/fd (opendir: Permission denied) 2024-04-21T09:10:05.328 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78832 admin_soc ceph cwd unknown /proc/28191/task/78832/cwd (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78832 admin_soc ceph rtd unknown /proc/28191/task/78832/root (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78832 admin_soc ceph txt unknown /proc/28191/task/78832/exe (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78832 admin_soc ceph NOFD /proc/28191/task/78832/fd (opendir: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78833 ceph_time ceph cwd unknown /proc/28191/task/78833/cwd (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78833 ceph_time ceph rtd unknown /proc/28191/task/78833/root (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78833 ceph_time ceph txt unknown /proc/28191/task/78833/exe (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78833 ceph_time ceph NOFD /proc/28191/task/78833/fd (opendir: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78834 fn_anonym ceph cwd unknown /proc/28191/task/78834/cwd (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78834 fn_anonym ceph rtd unknown /proc/28191/task/78834/root (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78834 fn_anonym ceph txt unknown /proc/28191/task/78834/exe (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78834 fn_anonym ceph NOFD /proc/28191/task/78834/fd (opendir: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78835 signal_ha ceph cwd unknown /proc/28191/task/78835/cwd (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78835 signal_ha ceph rtd unknown /proc/28191/task/78835/root (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78835 signal_ha ceph txt unknown /proc/28191/task/78835/exe (readlink: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78835 signal_ha ceph NOFD /proc/28191/task/78835/fd (opendir: Permission denied) 2024-04-21T09:10:05.329 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78836 mgrsb-fin ceph cwd unknown /proc/28191/task/78836/cwd (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78836 mgrsb-fin ceph rtd unknown /proc/28191/task/78836/root (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78836 mgrsb-fin ceph txt unknown /proc/28191/task/78836/exe (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78836 mgrsb-fin ceph NOFD /proc/28191/task/78836/fd (opendir: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78837 ms_dispat ceph cwd unknown /proc/28191/task/78837/cwd (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78837 ms_dispat ceph rtd unknown /proc/28191/task/78837/root (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78837 ms_dispat ceph txt unknown /proc/28191/task/78837/exe (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78837 ms_dispat ceph NOFD /proc/28191/task/78837/fd (opendir: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78838 ms_local ceph cwd unknown /proc/28191/task/78838/cwd (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78838 ms_local ceph rtd unknown /proc/28191/task/78838/root (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78838 ms_local ceph txt unknown /proc/28191/task/78838/exe (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78838 ms_local ceph NOFD /proc/28191/task/78838/fd (opendir: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78839 io_contex ceph cwd unknown /proc/28191/task/78839/cwd (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78839 io_contex ceph rtd unknown /proc/28191/task/78839/root (readlink: Permission denied) 2024-04-21T09:10:05.330 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78839 io_contex ceph txt unknown /proc/28191/task/78839/exe (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78839 io_contex ceph NOFD /proc/28191/task/78839/fd (opendir: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78840 io_contex ceph cwd unknown /proc/28191/task/78840/cwd (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78840 io_contex ceph rtd unknown /proc/28191/task/78840/root (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78840 io_contex ceph txt unknown /proc/28191/task/78840/exe (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78840 io_contex ceph NOFD /proc/28191/task/78840/fd (opendir: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78841 safe_time ceph cwd unknown /proc/28191/task/78841/cwd (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78841 safe_time ceph rtd unknown /proc/28191/task/78841/root (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78841 safe_time ceph txt unknown /proc/28191/task/78841/exe (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78841 safe_time ceph NOFD /proc/28191/task/78841/fd (opendir: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78842 safe_time ceph cwd unknown /proc/28191/task/78842/cwd (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78842 safe_time ceph rtd unknown /proc/28191/task/78842/root (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78842 safe_time ceph txt unknown /proc/28191/task/78842/exe (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78842 safe_time ceph NOFD /proc/28191/task/78842/fd (opendir: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78843 safe_time ceph cwd unknown /proc/28191/task/78843/cwd (readlink: Permission denied) 2024-04-21T09:10:05.331 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78843 safe_time ceph rtd unknown /proc/28191/task/78843/root (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78843 safe_time ceph txt unknown /proc/28191/task/78843/exe (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78843 safe_time ceph NOFD /proc/28191/task/78843/fd (opendir: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78844 fn_anonym ceph cwd unknown /proc/28191/task/78844/cwd (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78844 fn_anonym ceph rtd unknown /proc/28191/task/78844/root (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78844 fn_anonym ceph txt unknown /proc/28191/task/78844/exe (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78844 fn_anonym ceph NOFD /proc/28191/task/78844/fd (opendir: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78845 flusher ceph cwd unknown /proc/28191/task/78845/cwd (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78845 flusher ceph rtd unknown /proc/28191/task/78845/root (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78845 flusher ceph txt unknown /proc/28191/task/78845/exe (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78845 flusher ceph NOFD /proc/28191/task/78845/fd (opendir: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78846 safe_time ceph cwd unknown /proc/28191/task/78846/cwd (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78846 safe_time ceph rtd unknown /proc/28191/task/78846/root (readlink: Permission denied) 2024-04-21T09:10:05.332 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78846 safe_time ceph txt unknown /proc/28191/task/78846/exe (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 78846 safe_time ceph NOFD /proc/28191/task/78846/fd (opendir: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79890 mgr-fin ceph cwd unknown /proc/28191/task/79890/cwd (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79890 mgr-fin ceph rtd unknown /proc/28191/task/79890/root (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79890 mgr-fin ceph txt unknown /proc/28191/task/79890/exe (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79890 mgr-fin ceph NOFD /proc/28191/task/79890/fd (opendir: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79891 ms_dispat ceph cwd unknown /proc/28191/task/79891/cwd (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79891 ms_dispat ceph rtd unknown /proc/28191/task/79891/root (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79891 ms_dispat ceph txt unknown /proc/28191/task/79891/exe (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79891 ms_dispat ceph NOFD /proc/28191/task/79891/fd (opendir: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79892 ms_local ceph cwd unknown /proc/28191/task/79892/cwd (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79892 ms_local ceph rtd unknown /proc/28191/task/79892/root (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79892 ms_local ceph txt unknown /proc/28191/task/79892/exe (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79892 ms_local ceph NOFD /proc/28191/task/79892/fd (opendir: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79893 safe_time ceph cwd unknown /proc/28191/task/79893/cwd (readlink: Permission denied) 2024-04-21T09:10:05.333 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79893 safe_time ceph rtd unknown /proc/28191/task/79893/root (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79893 safe_time ceph txt unknown /proc/28191/task/79893/exe (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79893 safe_time ceph NOFD /proc/28191/task/79893/fd (opendir: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79894 cmdfin ceph cwd unknown /proc/28191/task/79894/cwd (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79894 cmdfin ceph rtd unknown /proc/28191/task/79894/root (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79894 cmdfin ceph txt unknown /proc/28191/task/79894/exe (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79894 cmdfin ceph NOFD /proc/28191/task/79894/fd (opendir: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79898 io_contex ceph cwd unknown /proc/28191/task/79898/cwd (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79898 io_contex ceph rtd unknown /proc/28191/task/79898/root (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79898 io_contex ceph txt unknown /proc/28191/task/79898/exe (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79898 io_contex ceph NOFD /proc/28191/task/79898/fd (opendir: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79899 io_contex ceph cwd unknown /proc/28191/task/79899/cwd (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79899 io_contex ceph rtd unknown /proc/28191/task/79899/root (readlink: Permission denied) 2024-04-21T09:10:05.334 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79899 io_contex ceph txt unknown /proc/28191/task/79899/exe (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79899 io_contex ceph NOFD /proc/28191/task/79899/fd (opendir: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79900 ceph_time ceph cwd unknown /proc/28191/task/79900/cwd (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79900 ceph_time ceph rtd unknown /proc/28191/task/79900/root (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79900 ceph_time ceph txt unknown /proc/28191/task/79900/exe (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79900 ceph_time ceph NOFD /proc/28191/task/79900/fd (opendir: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79901 ms_dispat ceph cwd unknown /proc/28191/task/79901/cwd (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79901 ms_dispat ceph rtd unknown /proc/28191/task/79901/root (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79901 ms_dispat ceph txt unknown /proc/28191/task/79901/exe (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79901 ms_dispat ceph NOFD /proc/28191/task/79901/fd (opendir: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79902 ms_local ceph cwd unknown /proc/28191/task/79902/cwd (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79902 ms_local ceph rtd unknown /proc/28191/task/79902/root (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79902 ms_local ceph txt unknown /proc/28191/task/79902/exe (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79902 ms_local ceph NOFD /proc/28191/task/79902/fd (opendir: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79903 safe_time ceph cwd unknown /proc/28191/task/79903/cwd (readlink: Permission denied) 2024-04-21T09:10:05.335 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79903 safe_time ceph rtd unknown /proc/28191/task/79903/root (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79903 safe_time ceph txt unknown /proc/28191/task/79903/exe (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79903 safe_time ceph NOFD /proc/28191/task/79903/fd (opendir: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79904 safe_time ceph cwd unknown /proc/28191/task/79904/cwd (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79904 safe_time ceph rtd unknown /proc/28191/task/79904/root (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79904 safe_time ceph txt unknown /proc/28191/task/79904/exe (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79904 safe_time ceph NOFD /proc/28191/task/79904/fd (opendir: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79905 balancer ceph cwd unknown /proc/28191/task/79905/cwd (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79905 balancer ceph rtd unknown /proc/28191/task/79905/root (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79905 balancer ceph txt unknown /proc/28191/task/79905/exe (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79905 balancer ceph NOFD /proc/28191/task/79905/fd (opendir: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79906 m-fin-bal ceph cwd unknown /proc/28191/task/79906/cwd (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79906 m-fin-bal ceph rtd unknown /proc/28191/task/79906/root (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79906 m-fin-bal ceph txt unknown /proc/28191/task/79906/exe (readlink: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79906 m-fin-bal ceph NOFD /proc/28191/task/79906/fd (opendir: Permission denied) 2024-04-21T09:10:05.336 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79907 mgr-fin ceph cwd unknown /proc/28191/task/79907/cwd (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79907 mgr-fin ceph rtd unknown /proc/28191/task/79907/root (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79907 mgr-fin ceph txt unknown /proc/28191/task/79907/exe (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79907 mgr-fin ceph NOFD /proc/28191/task/79907/fd (opendir: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79908 mgr-fin ceph cwd unknown /proc/28191/task/79908/cwd (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79908 mgr-fin ceph rtd unknown /proc/28191/task/79908/root (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79908 mgr-fin ceph txt unknown /proc/28191/task/79908/exe (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79908 mgr-fin ceph NOFD /proc/28191/task/79908/fd (opendir: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79909 mgr-fin ceph cwd unknown /proc/28191/task/79909/cwd (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79909 mgr-fin ceph rtd unknown /proc/28191/task/79909/root (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79909 mgr-fin ceph txt unknown /proc/28191/task/79909/exe (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79909 mgr-fin ceph NOFD /proc/28191/task/79909/fd (opendir: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79910 mgr-fin ceph cwd unknown /proc/28191/task/79910/cwd (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79910 mgr-fin ceph rtd unknown /proc/28191/task/79910/root (readlink: Permission denied) 2024-04-21T09:10:05.337 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79910 mgr-fin ceph txt unknown /proc/28191/task/79910/exe (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79910 mgr-fin ceph NOFD /proc/28191/task/79910/fd (opendir: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79911 mgr-fin ceph cwd unknown /proc/28191/task/79911/cwd (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79911 mgr-fin ceph rtd unknown /proc/28191/task/79911/root (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79911 mgr-fin ceph txt unknown /proc/28191/task/79911/exe (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79911 mgr-fin ceph NOFD /proc/28191/task/79911/fd (opendir: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79912 mgr-fin ceph cwd unknown /proc/28191/task/79912/cwd (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79912 mgr-fin ceph rtd unknown /proc/28191/task/79912/root (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79912 mgr-fin ceph txt unknown /proc/28191/task/79912/exe (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79912 mgr-fin ceph NOFD /proc/28191/task/79912/fd (opendir: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79913 mgr-fin ceph cwd unknown /proc/28191/task/79913/cwd (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79913 mgr-fin ceph rtd unknown /proc/28191/task/79913/root (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79913 mgr-fin ceph txt unknown /proc/28191/task/79913/exe (readlink: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79913 mgr-fin ceph NOFD /proc/28191/task/79913/fd (opendir: Permission denied) 2024-04-21T09:10:05.338 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79914 mgr-fin ceph cwd unknown /proc/28191/task/79914/cwd (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79914 mgr-fin ceph rtd unknown /proc/28191/task/79914/root (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79914 mgr-fin ceph txt unknown /proc/28191/task/79914/exe (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79914 mgr-fin ceph NOFD /proc/28191/task/79914/fd (opendir: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79915 mgr-fin ceph cwd unknown /proc/28191/task/79915/cwd (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79915 mgr-fin ceph rtd unknown /proc/28191/task/79915/root (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79915 mgr-fin ceph txt unknown /proc/28191/task/79915/exe (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79915 mgr-fin ceph NOFD /proc/28191/task/79915/fd (opendir: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79916 mgr-fin ceph cwd unknown /proc/28191/task/79916/cwd (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79916 mgr-fin ceph rtd unknown /proc/28191/task/79916/root (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79916 mgr-fin ceph txt unknown /proc/28191/task/79916/exe (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79916 mgr-fin ceph NOFD /proc/28191/task/79916/fd (opendir: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79917 mgr-fin ceph cwd unknown /proc/28191/task/79917/cwd (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79917 mgr-fin ceph rtd unknown /proc/28191/task/79917/root (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79917 mgr-fin ceph txt unknown /proc/28191/task/79917/exe (readlink: Permission denied) 2024-04-21T09:10:05.339 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79917 mgr-fin ceph NOFD /proc/28191/task/79917/fd (opendir: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79918 mgr-fin ceph cwd unknown /proc/28191/task/79918/cwd (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79918 mgr-fin ceph rtd unknown /proc/28191/task/79918/root (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79918 mgr-fin ceph txt unknown /proc/28191/task/79918/exe (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79918 mgr-fin ceph NOFD /proc/28191/task/79918/fd (opendir: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79919 mgr-fin ceph cwd unknown /proc/28191/task/79919/cwd (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79919 mgr-fin ceph rtd unknown /proc/28191/task/79919/root (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79919 mgr-fin ceph txt unknown /proc/28191/task/79919/exe (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79919 mgr-fin ceph NOFD /proc/28191/task/79919/fd (opendir: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79920 mgr-fin ceph cwd unknown /proc/28191/task/79920/cwd (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79920 mgr-fin ceph rtd unknown /proc/28191/task/79920/root (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79920 mgr-fin ceph txt unknown /proc/28191/task/79920/exe (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79920 mgr-fin ceph NOFD /proc/28191/task/79920/fd (opendir: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79921 mgr-fin ceph cwd unknown /proc/28191/task/79921/cwd (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79921 mgr-fin ceph rtd unknown /proc/28191/task/79921/root (readlink: Permission denied) 2024-04-21T09:10:05.340 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79921 mgr-fin ceph txt unknown /proc/28191/task/79921/exe (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79921 mgr-fin ceph NOFD /proc/28191/task/79921/fd (opendir: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79922 mgr-fin ceph cwd unknown /proc/28191/task/79922/cwd (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79922 mgr-fin ceph rtd unknown /proc/28191/task/79922/root (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79922 mgr-fin ceph txt unknown /proc/28191/task/79922/exe (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79922 mgr-fin ceph NOFD /proc/28191/task/79922/fd (opendir: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79923 cephadm ceph cwd unknown /proc/28191/task/79923/cwd (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79923 cephadm ceph rtd unknown /proc/28191/task/79923/root (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79923 cephadm ceph txt unknown /proc/28191/task/79923/exe (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79923 cephadm ceph NOFD /proc/28191/task/79923/fd (opendir: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79924 m-fin-cep ceph cwd unknown /proc/28191/task/79924/cwd (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79924 m-fin-cep ceph rtd unknown /proc/28191/task/79924/root (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79924 m-fin-cep ceph txt unknown /proc/28191/task/79924/exe (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79924 m-fin-cep ceph NOFD /proc/28191/task/79924/fd (opendir: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79925 mgr-fin ceph cwd unknown /proc/28191/task/79925/cwd (readlink: Permission denied) 2024-04-21T09:10:05.341 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79925 mgr-fin ceph rtd unknown /proc/28191/task/79925/root (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79925 mgr-fin ceph txt unknown /proc/28191/task/79925/exe (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79925 mgr-fin ceph NOFD /proc/28191/task/79925/fd (opendir: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79926 crash ceph cwd unknown /proc/28191/task/79926/cwd (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79926 crash ceph rtd unknown /proc/28191/task/79926/root (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79926 crash ceph txt unknown /proc/28191/task/79926/exe (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79926 crash ceph NOFD /proc/28191/task/79926/fd (opendir: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79927 m-fin-cra ceph cwd unknown /proc/28191/task/79927/cwd (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79927 m-fin-cra ceph rtd unknown /proc/28191/task/79927/root (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79927 m-fin-cra ceph txt unknown /proc/28191/task/79927/exe (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79927 m-fin-cra ceph NOFD /proc/28191/task/79927/fd (opendir: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79928 mgr-fin ceph cwd unknown /proc/28191/task/79928/cwd (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79928 mgr-fin ceph rtd unknown /proc/28191/task/79928/root (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79928 mgr-fin ceph txt unknown /proc/28191/task/79928/exe (readlink: Permission denied) 2024-04-21T09:10:05.342 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79928 mgr-fin ceph NOFD /proc/28191/task/79928/fd (opendir: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79929 cephadm ceph cwd unknown /proc/28191/task/79929/cwd (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79929 cephadm ceph rtd unknown /proc/28191/task/79929/root (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79929 cephadm ceph txt unknown /proc/28191/task/79929/exe (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79929 cephadm ceph NOFD /proc/28191/task/79929/fd (opendir: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79930 mgr-fin ceph cwd unknown /proc/28191/task/79930/cwd (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79930 mgr-fin ceph rtd unknown /proc/28191/task/79930/root (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79930 mgr-fin ceph txt unknown /proc/28191/task/79930/exe (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79930 mgr-fin ceph NOFD /proc/28191/task/79930/fd (opendir: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79931 mgr-fin ceph cwd unknown /proc/28191/task/79931/cwd (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79931 mgr-fin ceph rtd unknown /proc/28191/task/79931/root (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79931 mgr-fin ceph txt unknown /proc/28191/task/79931/exe (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79931 mgr-fin ceph NOFD /proc/28191/task/79931/fd (opendir: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79932 dashboard ceph cwd unknown /proc/28191/task/79932/cwd (readlink: Permission denied) 2024-04-21T09:10:05.343 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79932 dashboard ceph rtd unknown /proc/28191/task/79932/root (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79932 dashboard ceph txt unknown /proc/28191/task/79932/exe (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79932 dashboard ceph NOFD /proc/28191/task/79932/fd (opendir: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79933 m-fin-das ceph cwd unknown /proc/28191/task/79933/cwd (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79933 m-fin-das ceph rtd unknown /proc/28191/task/79933/root (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79933 m-fin-das ceph txt unknown /proc/28191/task/79933/exe (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79933 m-fin-das ceph NOFD /proc/28191/task/79933/fd (opendir: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79934 mgr-fin ceph cwd unknown /proc/28191/task/79934/cwd (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79934 mgr-fin ceph rtd unknown /proc/28191/task/79934/root (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79934 mgr-fin ceph txt unknown /proc/28191/task/79934/exe (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79934 mgr-fin ceph NOFD /proc/28191/task/79934/fd (opendir: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79935 mgr-fin ceph cwd unknown /proc/28191/task/79935/cwd (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79935 mgr-fin ceph rtd unknown /proc/28191/task/79935/root (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79935 mgr-fin ceph txt unknown /proc/28191/task/79935/exe (readlink: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79935 mgr-fin ceph NOFD /proc/28191/task/79935/fd (opendir: Permission denied) 2024-04-21T09:10:05.344 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79936 mgr-fin ceph cwd unknown /proc/28191/task/79936/cwd (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79936 mgr-fin ceph rtd unknown /proc/28191/task/79936/root (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79936 mgr-fin ceph txt unknown /proc/28191/task/79936/exe (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79936 mgr-fin ceph NOFD /proc/28191/task/79936/fd (opendir: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79937 devicehea ceph cwd unknown /proc/28191/task/79937/cwd (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79937 devicehea ceph rtd unknown /proc/28191/task/79937/root (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79937 devicehea ceph txt unknown /proc/28191/task/79937/exe (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79937 devicehea ceph NOFD /proc/28191/task/79937/fd (opendir: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79938 m-fin-dev ceph cwd unknown /proc/28191/task/79938/cwd (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79938 m-fin-dev ceph rtd unknown /proc/28191/task/79938/root (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79938 m-fin-dev ceph txt unknown /proc/28191/task/79938/exe (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79938 m-fin-dev ceph NOFD /proc/28191/task/79938/fd (opendir: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79939 mgr-fin ceph cwd unknown /proc/28191/task/79939/cwd (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79939 mgr-fin ceph rtd unknown /proc/28191/task/79939/root (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79939 mgr-fin ceph txt unknown /proc/28191/task/79939/exe (readlink: Permission denied) 2024-04-21T09:10:05.345 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79939 mgr-fin ceph NOFD /proc/28191/task/79939/fd (opendir: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79941 m-fin-ios ceph cwd unknown /proc/28191/task/79941/cwd (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79941 m-fin-ios ceph rtd unknown /proc/28191/task/79941/root (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79941 m-fin-ios ceph txt unknown /proc/28191/task/79941/exe (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79941 m-fin-ios ceph NOFD /proc/28191/task/79941/fd (opendir: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79942 mgr-fin ceph cwd unknown /proc/28191/task/79942/cwd (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79942 mgr-fin ceph rtd unknown /proc/28191/task/79942/root (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79942 mgr-fin ceph txt unknown /proc/28191/task/79942/exe (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79942 mgr-fin ceph NOFD /proc/28191/task/79942/fd (opendir: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79943 mgr-fin ceph cwd unknown /proc/28191/task/79943/cwd (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79943 mgr-fin ceph rtd unknown /proc/28191/task/79943/root (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79943 mgr-fin ceph txt unknown /proc/28191/task/79943/exe (readlink: Permission denied) 2024-04-21T09:10:05.346 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79943 mgr-fin ceph NOFD /proc/28191/task/79943/fd (opendir: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79944 mgr-fin ceph cwd unknown /proc/28191/task/79944/cwd (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79944 mgr-fin ceph rtd unknown /proc/28191/task/79944/root (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79944 mgr-fin ceph txt unknown /proc/28191/task/79944/exe (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79944 mgr-fin ceph NOFD /proc/28191/task/79944/fd (opendir: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79946 m-fin-nfs ceph cwd unknown /proc/28191/task/79946/cwd (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79946 m-fin-nfs ceph rtd unknown /proc/28191/task/79946/root (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79946 m-fin-nfs ceph txt unknown /proc/28191/task/79946/exe (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79946 m-fin-nfs ceph NOFD /proc/28191/task/79946/fd (opendir: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79947 mgr-fin ceph cwd unknown /proc/28191/task/79947/cwd (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79947 mgr-fin ceph rtd unknown /proc/28191/task/79947/root (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79947 mgr-fin ceph txt unknown /proc/28191/task/79947/exe (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79947 mgr-fin ceph NOFD /proc/28191/task/79947/fd (opendir: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79949 m-fin-orc ceph cwd unknown /proc/28191/task/79949/cwd (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79949 m-fin-orc ceph rtd unknown /proc/28191/task/79949/root (readlink: Permission denied) 2024-04-21T09:10:05.347 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79949 m-fin-orc ceph txt unknown /proc/28191/task/79949/exe (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79949 m-fin-orc ceph NOFD /proc/28191/task/79949/fd (opendir: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79950 pg_autosc ceph cwd unknown /proc/28191/task/79950/cwd (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79950 pg_autosc ceph rtd unknown /proc/28191/task/79950/root (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79950 pg_autosc ceph txt unknown /proc/28191/task/79950/exe (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79950 pg_autosc ceph NOFD /proc/28191/task/79950/fd (opendir: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79951 m-fin-pg_ ceph cwd unknown /proc/28191/task/79951/cwd (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79951 m-fin-pg_ ceph rtd unknown /proc/28191/task/79951/root (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79951 m-fin-pg_ ceph txt unknown /proc/28191/task/79951/exe (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79951 m-fin-pg_ ceph NOFD /proc/28191/task/79951/fd (opendir: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79952 progress ceph cwd unknown /proc/28191/task/79952/cwd (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79952 progress ceph rtd unknown /proc/28191/task/79952/root (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79952 progress ceph txt unknown /proc/28191/task/79952/exe (readlink: Permission denied) 2024-04-21T09:10:05.348 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79952 progress ceph NOFD /proc/28191/task/79952/fd (opendir: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79953 m-fin-pro ceph cwd unknown /proc/28191/task/79953/cwd (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79953 m-fin-pro ceph rtd unknown /proc/28191/task/79953/root (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79953 m-fin-pro ceph txt unknown /proc/28191/task/79953/exe (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79953 m-fin-pro ceph NOFD /proc/28191/task/79953/fd (opendir: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79957 io_contex ceph cwd unknown /proc/28191/task/79957/cwd (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79957 io_contex ceph rtd unknown /proc/28191/task/79957/root (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79957 io_contex ceph txt unknown /proc/28191/task/79957/exe (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79957 io_contex ceph NOFD /proc/28191/task/79957/fd (opendir: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79958 io_contex ceph cwd unknown /proc/28191/task/79958/cwd (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79958 io_contex ceph rtd unknown /proc/28191/task/79958/root (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79958 io_contex ceph txt unknown /proc/28191/task/79958/exe (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79958 io_contex ceph NOFD /proc/28191/task/79958/fd (opendir: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79959 ceph_time ceph cwd unknown /proc/28191/task/79959/cwd (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79959 ceph_time ceph rtd unknown /proc/28191/task/79959/root (readlink: Permission denied) 2024-04-21T09:10:05.349 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79959 ceph_time ceph txt unknown /proc/28191/task/79959/exe (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79959 ceph_time ceph NOFD /proc/28191/task/79959/fd (opendir: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79960 ms_dispat ceph cwd unknown /proc/28191/task/79960/cwd (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79960 ms_dispat ceph rtd unknown /proc/28191/task/79960/root (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79960 ms_dispat ceph txt unknown /proc/28191/task/79960/exe (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79960 ms_dispat ceph NOFD /proc/28191/task/79960/fd (opendir: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79961 ms_local ceph cwd unknown /proc/28191/task/79961/cwd (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79961 ms_local ceph rtd unknown /proc/28191/task/79961/root (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79961 ms_local ceph txt unknown /proc/28191/task/79961/exe (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79961 ms_local ceph NOFD /proc/28191/task/79961/fd (opendir: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79962 safe_time ceph cwd unknown /proc/28191/task/79962/cwd (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79962 safe_time ceph rtd unknown /proc/28191/task/79962/root (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79962 safe_time ceph txt unknown /proc/28191/task/79962/exe (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79962 safe_time ceph NOFD /proc/28191/task/79962/fd (opendir: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79963 safe_time ceph cwd unknown /proc/28191/task/79963/cwd (readlink: Permission denied) 2024-04-21T09:10:05.350 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79963 safe_time ceph rtd unknown /proc/28191/task/79963/root (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79963 safe_time ceph txt unknown /proc/28191/task/79963/exe (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79963 safe_time ceph NOFD /proc/28191/task/79963/fd (opendir: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79964 mgr-fin ceph cwd unknown /proc/28191/task/79964/cwd (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79964 mgr-fin ceph rtd unknown /proc/28191/task/79964/root (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79964 mgr-fin ceph txt unknown /proc/28191/task/79964/exe (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79964 mgr-fin ceph NOFD /proc/28191/task/79964/fd (opendir: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79966 m-fin-rbd ceph cwd unknown /proc/28191/task/79966/cwd (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79966 m-fin-rbd ceph rtd unknown /proc/28191/task/79966/root (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79966 m-fin-rbd ceph txt unknown /proc/28191/task/79966/exe (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79966 m-fin-rbd ceph NOFD /proc/28191/task/79966/fd (opendir: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79967 restful ceph cwd unknown /proc/28191/task/79967/cwd (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79967 restful ceph rtd unknown /proc/28191/task/79967/root (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79967 restful ceph txt unknown /proc/28191/task/79967/exe (readlink: Permission denied) 2024-04-21T09:10:05.351 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79967 restful ceph NOFD /proc/28191/task/79967/fd (opendir: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79968 m-fin-res ceph cwd unknown /proc/28191/task/79968/cwd (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79968 m-fin-res ceph rtd unknown /proc/28191/task/79968/root (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79968 m-fin-res ceph txt unknown /proc/28191/task/79968/exe (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79968 m-fin-res ceph NOFD /proc/28191/task/79968/fd (opendir: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79969 mgr-fin ceph cwd unknown /proc/28191/task/79969/cwd (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79969 mgr-fin ceph rtd unknown /proc/28191/task/79969/root (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79969 mgr-fin ceph txt unknown /proc/28191/task/79969/exe (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79969 mgr-fin ceph NOFD /proc/28191/task/79969/fd (opendir: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79970 mgr-fin ceph cwd unknown /proc/28191/task/79970/cwd (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79970 mgr-fin ceph rtd unknown /proc/28191/task/79970/root (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79970 mgr-fin ceph txt unknown /proc/28191/task/79970/exe (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79970 mgr-fin ceph NOFD /proc/28191/task/79970/fd (opendir: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79971 mgr-fin ceph cwd unknown /proc/28191/task/79971/cwd (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79971 mgr-fin ceph rtd unknown /proc/28191/task/79971/root (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79971 mgr-fin ceph txt unknown /proc/28191/task/79971/exe (readlink: Permission denied) 2024-04-21T09:10:05.352 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79971 mgr-fin ceph NOFD /proc/28191/task/79971/fd (opendir: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79972 mgr-fin ceph cwd unknown /proc/28191/task/79972/cwd (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79972 mgr-fin ceph rtd unknown /proc/28191/task/79972/root (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79972 mgr-fin ceph txt unknown /proc/28191/task/79972/exe (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79972 mgr-fin ceph NOFD /proc/28191/task/79972/fd (opendir: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79973 mgr-fin ceph cwd unknown /proc/28191/task/79973/cwd (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79973 mgr-fin ceph rtd unknown /proc/28191/task/79973/root (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79973 mgr-fin ceph txt unknown /proc/28191/task/79973/exe (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79973 mgr-fin ceph NOFD /proc/28191/task/79973/fd (opendir: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79974 mgr-fin ceph cwd unknown /proc/28191/task/79974/cwd (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79974 mgr-fin ceph rtd unknown /proc/28191/task/79974/root (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79974 mgr-fin ceph txt unknown /proc/28191/task/79974/exe (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79974 mgr-fin ceph NOFD /proc/28191/task/79974/fd (opendir: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79978 io_contex ceph cwd unknown /proc/28191/task/79978/cwd (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79978 io_contex ceph rtd unknown /proc/28191/task/79978/root (readlink: Permission denied) 2024-04-21T09:10:05.353 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79978 io_contex ceph txt unknown /proc/28191/task/79978/exe (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79978 io_contex ceph NOFD /proc/28191/task/79978/fd (opendir: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79979 io_contex ceph cwd unknown /proc/28191/task/79979/cwd (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79979 io_contex ceph rtd unknown /proc/28191/task/79979/root (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79979 io_contex ceph txt unknown /proc/28191/task/79979/exe (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79979 io_contex ceph NOFD /proc/28191/task/79979/fd (opendir: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79980 ceph_time ceph cwd unknown /proc/28191/task/79980/cwd (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79980 ceph_time ceph rtd unknown /proc/28191/task/79980/root (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79980 ceph_time ceph txt unknown /proc/28191/task/79980/exe (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79980 ceph_time ceph NOFD /proc/28191/task/79980/fd (opendir: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79981 ms_dispat ceph cwd unknown /proc/28191/task/79981/cwd (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79981 ms_dispat ceph rtd unknown /proc/28191/task/79981/root (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79981 ms_dispat ceph txt unknown /proc/28191/task/79981/exe (readlink: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79981 ms_dispat ceph NOFD /proc/28191/task/79981/fd (opendir: Permission denied) 2024-04-21T09:10:05.354 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79982 ms_local ceph cwd unknown /proc/28191/task/79982/cwd (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79982 ms_local ceph rtd unknown /proc/28191/task/79982/root (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79982 ms_local ceph txt unknown /proc/28191/task/79982/exe (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79982 ms_local ceph NOFD /proc/28191/task/79982/fd (opendir: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79983 safe_time ceph cwd unknown /proc/28191/task/79983/cwd (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79983 safe_time ceph rtd unknown /proc/28191/task/79983/root (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79983 safe_time ceph txt unknown /proc/28191/task/79983/exe (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79983 safe_time ceph NOFD /proc/28191/task/79983/fd (opendir: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79984 safe_time ceph cwd unknown /proc/28191/task/79984/cwd (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79984 safe_time ceph rtd unknown /proc/28191/task/79984/root (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79984 safe_time ceph txt unknown /proc/28191/task/79984/exe (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79984 safe_time ceph NOFD /proc/28191/task/79984/fd (opendir: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79988 m-fin-sna ceph cwd unknown /proc/28191/task/79988/cwd (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79988 m-fin-sna ceph rtd unknown /proc/28191/task/79988/root (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79988 m-fin-sna ceph txt unknown /proc/28191/task/79988/exe (readlink: Permission denied) 2024-04-21T09:10:05.355 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79988 m-fin-sna ceph NOFD /proc/28191/task/79988/fd (opendir: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79990 m-fin-sta ceph cwd unknown /proc/28191/task/79990/cwd (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79990 m-fin-sta ceph rtd unknown /proc/28191/task/79990/root (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79990 m-fin-sta ceph txt unknown /proc/28191/task/79990/exe (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79990 m-fin-sta ceph NOFD /proc/28191/task/79990/fd (opendir: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79992 telemetry ceph cwd unknown /proc/28191/task/79992/cwd (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79992 telemetry ceph rtd unknown /proc/28191/task/79992/root (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79992 telemetry ceph txt unknown /proc/28191/task/79992/exe (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79992 telemetry ceph NOFD /proc/28191/task/79992/fd (opendir: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79993 m-fin-tel ceph cwd unknown /proc/28191/task/79993/cwd (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79993 m-fin-tel ceph rtd unknown /proc/28191/task/79993/root (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79993 m-fin-tel ceph txt unknown /proc/28191/task/79993/exe (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79993 m-fin-tel ceph NOFD /proc/28191/task/79993/fd (opendir: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79995 mgr-fin ceph cwd unknown /proc/28191/task/79995/cwd (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79995 mgr-fin ceph rtd unknown /proc/28191/task/79995/root (readlink: Permission denied) 2024-04-21T09:10:05.356 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79995 mgr-fin ceph txt unknown /proc/28191/task/79995/exe (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79995 mgr-fin ceph NOFD /proc/28191/task/79995/fd (opendir: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79999 io_contex ceph cwd unknown /proc/28191/task/79999/cwd (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79999 io_contex ceph rtd unknown /proc/28191/task/79999/root (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79999 io_contex ceph txt unknown /proc/28191/task/79999/exe (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 79999 io_contex ceph NOFD /proc/28191/task/79999/fd (opendir: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80000 io_contex ceph cwd unknown /proc/28191/task/80000/cwd (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80000 io_contex ceph rtd unknown /proc/28191/task/80000/root (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80000 io_contex ceph txt unknown /proc/28191/task/80000/exe (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80000 io_contex ceph NOFD /proc/28191/task/80000/fd (opendir: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80001 ceph_time ceph cwd unknown /proc/28191/task/80001/cwd (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80001 ceph_time ceph rtd unknown /proc/28191/task/80001/root (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80001 ceph_time ceph txt unknown /proc/28191/task/80001/exe (readlink: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80001 ceph_time ceph NOFD /proc/28191/task/80001/fd (opendir: Permission denied) 2024-04-21T09:10:05.357 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80002 ms_dispat ceph cwd unknown /proc/28191/task/80002/cwd (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80002 ms_dispat ceph rtd unknown /proc/28191/task/80002/root (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80002 ms_dispat ceph txt unknown /proc/28191/task/80002/exe (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80002 ms_dispat ceph NOFD /proc/28191/task/80002/fd (opendir: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80003 ms_local ceph cwd unknown /proc/28191/task/80003/cwd (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80003 ms_local ceph rtd unknown /proc/28191/task/80003/root (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80003 ms_local ceph txt unknown /proc/28191/task/80003/exe (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80003 ms_local ceph NOFD /proc/28191/task/80003/fd (opendir: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80004 safe_time ceph cwd unknown /proc/28191/task/80004/cwd (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80004 safe_time ceph rtd unknown /proc/28191/task/80004/root (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80004 safe_time ceph txt unknown /proc/28191/task/80004/exe (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80004 safe_time ceph NOFD /proc/28191/task/80004/fd (opendir: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80005 safe_time ceph cwd unknown /proc/28191/task/80005/cwd (readlink: Permission denied) 2024-04-21T09:10:05.358 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80005 safe_time ceph rtd unknown /proc/28191/task/80005/root (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80005 safe_time ceph txt unknown /proc/28191/task/80005/exe (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80005 safe_time ceph NOFD /proc/28191/task/80005/fd (opendir: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80006 mgr-fin ceph cwd unknown /proc/28191/task/80006/cwd (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80006 mgr-fin ceph rtd unknown /proc/28191/task/80006/root (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80006 mgr-fin ceph txt unknown /proc/28191/task/80006/exe (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80006 mgr-fin ceph NOFD /proc/28191/task/80006/fd (opendir: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80007 mgr-fin ceph cwd unknown /proc/28191/task/80007/cwd (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80007 mgr-fin ceph rtd unknown /proc/28191/task/80007/root (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80007 mgr-fin ceph txt unknown /proc/28191/task/80007/exe (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80007 mgr-fin ceph NOFD /proc/28191/task/80007/fd (opendir: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80008 mgr-fin ceph cwd unknown /proc/28191/task/80008/cwd (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80008 mgr-fin ceph rtd unknown /proc/28191/task/80008/root (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80008 mgr-fin ceph txt unknown /proc/28191/task/80008/exe (readlink: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80008 mgr-fin ceph NOFD /proc/28191/task/80008/fd (opendir: Permission denied) 2024-04-21T09:10:05.359 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80009 mgr-fin ceph cwd unknown /proc/28191/task/80009/cwd (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80009 mgr-fin ceph rtd unknown /proc/28191/task/80009/root (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80009 mgr-fin ceph txt unknown /proc/28191/task/80009/exe (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80009 mgr-fin ceph NOFD /proc/28191/task/80009/fd (opendir: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80010 mgr-fin ceph cwd unknown /proc/28191/task/80010/cwd (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80010 mgr-fin ceph rtd unknown /proc/28191/task/80010/root (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80010 mgr-fin ceph txt unknown /proc/28191/task/80010/exe (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80010 mgr-fin ceph NOFD /proc/28191/task/80010/fd (opendir: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80011 mgr-fin ceph cwd unknown /proc/28191/task/80011/cwd (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80011 mgr-fin ceph rtd unknown /proc/28191/task/80011/root (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80011 mgr-fin ceph txt unknown /proc/28191/task/80011/exe (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80011 mgr-fin ceph NOFD /proc/28191/task/80011/fd (opendir: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80012 mgr-fin ceph cwd unknown /proc/28191/task/80012/cwd (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80012 mgr-fin ceph rtd unknown /proc/28191/task/80012/root (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80012 mgr-fin ceph txt unknown /proc/28191/task/80012/exe (readlink: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80012 mgr-fin ceph NOFD /proc/28191/task/80012/fd (opendir: Permission denied) 2024-04-21T09:10:05.360 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80013 mgr-fin ceph cwd unknown /proc/28191/task/80013/cwd (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80013 mgr-fin ceph rtd unknown /proc/28191/task/80013/root (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80013 mgr-fin ceph txt unknown /proc/28191/task/80013/exe (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80013 mgr-fin ceph NOFD /proc/28191/task/80013/fd (opendir: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80014 mgr-fin ceph cwd unknown /proc/28191/task/80014/cwd (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80014 mgr-fin ceph rtd unknown /proc/28191/task/80014/root (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80014 mgr-fin ceph txt unknown /proc/28191/task/80014/exe (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80014 mgr-fin ceph NOFD /proc/28191/task/80014/fd (opendir: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80015 mgr-fin ceph cwd unknown /proc/28191/task/80015/cwd (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80015 mgr-fin ceph rtd unknown /proc/28191/task/80015/root (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80015 mgr-fin ceph txt unknown /proc/28191/task/80015/exe (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80015 mgr-fin ceph NOFD /proc/28191/task/80015/fd (opendir: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80017 mgr-fin ceph cwd unknown /proc/28191/task/80017/cwd (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80017 mgr-fin ceph rtd unknown /proc/28191/task/80017/root (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80017 mgr-fin ceph txt unknown /proc/28191/task/80017/exe (readlink: Permission denied) 2024-04-21T09:10:05.361 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80017 mgr-fin ceph NOFD /proc/28191/task/80017/fd (opendir: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80018 mgr-fin ceph cwd unknown /proc/28191/task/80018/cwd (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80018 mgr-fin ceph rtd unknown /proc/28191/task/80018/root (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80018 mgr-fin ceph txt unknown /proc/28191/task/80018/exe (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80018 mgr-fin ceph NOFD /proc/28191/task/80018/fd (opendir: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80022 m-fin-vol ceph cwd unknown /proc/28191/task/80022/cwd (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80022 m-fin-vol ceph rtd unknown /proc/28191/task/80022/root (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80022 m-fin-vol ceph txt unknown /proc/28191/task/80022/exe (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80022 m-fin-vol ceph NOFD /proc/28191/task/80022/fd (opendir: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80025 io_contex ceph cwd unknown /proc/28191/task/80025/cwd (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80025 io_contex ceph rtd unknown /proc/28191/task/80025/root (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80025 io_contex ceph txt unknown /proc/28191/task/80025/exe (readlink: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80025 io_contex ceph NOFD /proc/28191/task/80025/fd (opendir: Permission denied) 2024-04-21T09:10:05.362 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80026 io_contex ceph cwd unknown /proc/28191/task/80026/cwd (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80026 io_contex ceph rtd unknown /proc/28191/task/80026/root (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80026 io_contex ceph txt unknown /proc/28191/task/80026/exe (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80026 io_contex ceph NOFD /proc/28191/task/80026/fd (opendir: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80088 dashboard ceph cwd unknown /proc/28191/task/80088/cwd (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80088 dashboard ceph rtd unknown /proc/28191/task/80088/root (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80088 dashboard ceph txt unknown /proc/28191/task/80088/exe (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80088 dashboard ceph NOFD /proc/28191/task/80088/fd (opendir: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80089 dashboard ceph cwd unknown /proc/28191/task/80089/cwd (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80089 dashboard ceph rtd unknown /proc/28191/task/80089/root (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80089 dashboard ceph txt unknown /proc/28191/task/80089/exe (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80089 dashboard ceph NOFD /proc/28191/task/80089/fd (opendir: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80090 dashboard ceph cwd unknown /proc/28191/task/80090/cwd (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80090 dashboard ceph rtd unknown /proc/28191/task/80090/root (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80090 dashboard ceph txt unknown /proc/28191/task/80090/exe (readlink: Permission denied) 2024-04-21T09:10:05.363 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80090 dashboard ceph NOFD /proc/28191/task/80090/fd (opendir: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80091 dashboard ceph cwd unknown /proc/28191/task/80091/cwd (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80091 dashboard ceph rtd unknown /proc/28191/task/80091/root (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80091 dashboard ceph txt unknown /proc/28191/task/80091/exe (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80091 dashboard ceph NOFD /proc/28191/task/80091/fd (opendir: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80092 dashboard ceph cwd unknown /proc/28191/task/80092/cwd (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80092 dashboard ceph rtd unknown /proc/28191/task/80092/root (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80092 dashboard ceph txt unknown /proc/28191/task/80092/exe (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80092 dashboard ceph NOFD /proc/28191/task/80092/fd (opendir: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80093 dashboard ceph cwd unknown /proc/28191/task/80093/cwd (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80093 dashboard ceph rtd unknown /proc/28191/task/80093/root (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80093 dashboard ceph txt unknown /proc/28191/task/80093/exe (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80093 dashboard ceph NOFD /proc/28191/task/80093/fd (opendir: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80094 dashboard ceph cwd unknown /proc/28191/task/80094/cwd (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80094 dashboard ceph rtd unknown /proc/28191/task/80094/root (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80094 dashboard ceph txt unknown /proc/28191/task/80094/exe (readlink: Permission denied) 2024-04-21T09:10:05.364 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80094 dashboard ceph NOFD /proc/28191/task/80094/fd (opendir: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80095 dashboard ceph cwd unknown /proc/28191/task/80095/cwd (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80095 dashboard ceph rtd unknown /proc/28191/task/80095/root (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80095 dashboard ceph txt unknown /proc/28191/task/80095/exe (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80095 dashboard ceph NOFD /proc/28191/task/80095/fd (opendir: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80096 dashboard ceph cwd unknown /proc/28191/task/80096/cwd (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80096 dashboard ceph rtd unknown /proc/28191/task/80096/root (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80096 dashboard ceph txt unknown /proc/28191/task/80096/exe (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80096 dashboard ceph NOFD /proc/28191/task/80096/fd (opendir: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80097 dashboard ceph cwd unknown /proc/28191/task/80097/cwd (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80097 dashboard ceph rtd unknown /proc/28191/task/80097/root (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80097 dashboard ceph txt unknown /proc/28191/task/80097/exe (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80097 dashboard ceph NOFD /proc/28191/task/80097/fd (opendir: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80098 dashboard ceph cwd unknown /proc/28191/task/80098/cwd (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80098 dashboard ceph rtd unknown /proc/28191/task/80098/root (readlink: Permission denied) 2024-04-21T09:10:05.365 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80098 dashboard ceph txt unknown /proc/28191/task/80098/exe (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80098 dashboard ceph NOFD /proc/28191/task/80098/fd (opendir: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80165 dashboard ceph cwd unknown /proc/28191/task/80165/cwd (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80165 dashboard ceph rtd unknown /proc/28191/task/80165/root (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80165 dashboard ceph txt unknown /proc/28191/task/80165/exe (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80165 dashboard ceph NOFD /proc/28191/task/80165/fd (opendir: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80166 mgr-fin ceph cwd unknown /proc/28191/task/80166/cwd (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80166 mgr-fin ceph rtd unknown /proc/28191/task/80166/root (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80166 mgr-fin ceph txt unknown /proc/28191/task/80166/exe (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80166 mgr-fin ceph NOFD /proc/28191/task/80166/fd (opendir: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80167 mgr-fin ceph cwd unknown /proc/28191/task/80167/cwd (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80167 mgr-fin ceph rtd unknown /proc/28191/task/80167/root (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80167 mgr-fin ceph txt unknown /proc/28191/task/80167/exe (readlink: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80167 mgr-fin ceph NOFD /proc/28191/task/80167/fd (opendir: Permission denied) 2024-04-21T09:10:05.366 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80168 mgr-fin ceph cwd unknown /proc/28191/task/80168/cwd (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80168 mgr-fin ceph rtd unknown /proc/28191/task/80168/root (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80168 mgr-fin ceph txt unknown /proc/28191/task/80168/exe (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80168 mgr-fin ceph NOFD /proc/28191/task/80168/fd (opendir: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80169 mgr-fin ceph cwd unknown /proc/28191/task/80169/cwd (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80169 mgr-fin ceph rtd unknown /proc/28191/task/80169/root (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80169 mgr-fin ceph txt unknown /proc/28191/task/80169/exe (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80169 mgr-fin ceph NOFD /proc/28191/task/80169/fd (opendir: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80170 mgr-fin ceph cwd unknown /proc/28191/task/80170/cwd (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80170 mgr-fin ceph rtd unknown /proc/28191/task/80170/root (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80170 mgr-fin ceph txt unknown /proc/28191/task/80170/exe (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80170 mgr-fin ceph NOFD /proc/28191/task/80170/fd (opendir: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80171 mgr-fin ceph cwd unknown /proc/28191/task/80171/cwd (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80171 mgr-fin ceph rtd unknown /proc/28191/task/80171/root (readlink: Permission denied) 2024-04-21T09:10:05.367 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80171 mgr-fin ceph txt unknown /proc/28191/task/80171/exe (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80171 mgr-fin ceph NOFD /proc/28191/task/80171/fd (opendir: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80172 mgr-fin ceph cwd unknown /proc/28191/task/80172/cwd (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80172 mgr-fin ceph rtd unknown /proc/28191/task/80172/root (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80172 mgr-fin ceph txt unknown /proc/28191/task/80172/exe (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80172 mgr-fin ceph NOFD /proc/28191/task/80172/fd (opendir: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80173 mgr-fin ceph cwd unknown /proc/28191/task/80173/cwd (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80173 mgr-fin ceph rtd unknown /proc/28191/task/80173/root (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80173 mgr-fin ceph txt unknown /proc/28191/task/80173/exe (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80173 mgr-fin ceph NOFD /proc/28191/task/80173/fd (opendir: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80174 mgr-fin ceph cwd unknown /proc/28191/task/80174/cwd (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80174 mgr-fin ceph rtd unknown /proc/28191/task/80174/root (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80174 mgr-fin ceph txt unknown /proc/28191/task/80174/exe (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80174 mgr-fin ceph NOFD /proc/28191/task/80174/fd (opendir: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80175 mgr-fin ceph cwd unknown /proc/28191/task/80175/cwd (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80175 mgr-fin ceph rtd unknown /proc/28191/task/80175/root (readlink: Permission denied) 2024-04-21T09:10:05.368 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80175 mgr-fin ceph txt unknown /proc/28191/task/80175/exe (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80175 mgr-fin ceph NOFD /proc/28191/task/80175/fd (opendir: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80176 mgr-fin ceph cwd unknown /proc/28191/task/80176/cwd (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80176 mgr-fin ceph rtd unknown /proc/28191/task/80176/root (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80176 mgr-fin ceph txt unknown /proc/28191/task/80176/exe (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80176 mgr-fin ceph NOFD /proc/28191/task/80176/fd (opendir: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80200 mgr-fin ceph cwd unknown /proc/28191/task/80200/cwd (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80200 mgr-fin ceph rtd unknown /proc/28191/task/80200/root (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80200 mgr-fin ceph txt unknown /proc/28191/task/80200/exe (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80200 mgr-fin ceph NOFD /proc/28191/task/80200/fd (opendir: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80201 mgr-fin ceph cwd unknown /proc/28191/task/80201/cwd (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80201 mgr-fin ceph rtd unknown /proc/28191/task/80201/root (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80201 mgr-fin ceph txt unknown /proc/28191/task/80201/exe (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80201 mgr-fin ceph NOFD /proc/28191/task/80201/fd (opendir: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80202 mgr-fin ceph cwd unknown /proc/28191/task/80202/cwd (readlink: Permission denied) 2024-04-21T09:10:05.369 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80202 mgr-fin ceph rtd unknown /proc/28191/task/80202/root (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80202 mgr-fin ceph txt unknown /proc/28191/task/80202/exe (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80202 mgr-fin ceph NOFD /proc/28191/task/80202/fd (opendir: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80203 mgr-fin ceph cwd unknown /proc/28191/task/80203/cwd (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80203 mgr-fin ceph rtd unknown /proc/28191/task/80203/root (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80203 mgr-fin ceph txt unknown /proc/28191/task/80203/exe (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80203 mgr-fin ceph NOFD /proc/28191/task/80203/fd (opendir: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80204 mgr-fin ceph cwd unknown /proc/28191/task/80204/cwd (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80204 mgr-fin ceph rtd unknown /proc/28191/task/80204/root (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80204 mgr-fin ceph txt unknown /proc/28191/task/80204/exe (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80204 mgr-fin ceph NOFD /proc/28191/task/80204/fd (opendir: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80205 mgr-fin ceph cwd unknown /proc/28191/task/80205/cwd (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80205 mgr-fin ceph rtd unknown /proc/28191/task/80205/root (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80205 mgr-fin ceph txt unknown /proc/28191/task/80205/exe (readlink: Permission denied) 2024-04-21T09:10:05.370 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80205 mgr-fin ceph NOFD /proc/28191/task/80205/fd (opendir: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80206 mgr-fin ceph cwd unknown /proc/28191/task/80206/cwd (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80206 mgr-fin ceph rtd unknown /proc/28191/task/80206/root (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80206 mgr-fin ceph txt unknown /proc/28191/task/80206/exe (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80206 mgr-fin ceph NOFD /proc/28191/task/80206/fd (opendir: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80207 mgr-fin ceph cwd unknown /proc/28191/task/80207/cwd (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80207 mgr-fin ceph rtd unknown /proc/28191/task/80207/root (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80207 mgr-fin ceph txt unknown /proc/28191/task/80207/exe (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80207 mgr-fin ceph NOFD /proc/28191/task/80207/fd (opendir: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80208 mgr-fin ceph cwd unknown /proc/28191/task/80208/cwd (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80208 mgr-fin ceph rtd unknown /proc/28191/task/80208/root (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80208 mgr-fin ceph txt unknown /proc/28191/task/80208/exe (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80208 mgr-fin ceph NOFD /proc/28191/task/80208/fd (opendir: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80209 mgr-fin ceph cwd unknown /proc/28191/task/80209/cwd (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80209 mgr-fin ceph rtd unknown /proc/28191/task/80209/root (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80209 mgr-fin ceph txt unknown /proc/28191/task/80209/exe (readlink: Permission denied) 2024-04-21T09:10:05.371 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80209 mgr-fin ceph NOFD /proc/28191/task/80209/fd (opendir: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80210 mgr-fin ceph cwd unknown /proc/28191/task/80210/cwd (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80210 mgr-fin ceph rtd unknown /proc/28191/task/80210/root (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80210 mgr-fin ceph txt unknown /proc/28191/task/80210/exe (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80210 mgr-fin ceph NOFD /proc/28191/task/80210/fd (opendir: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80548 devicehea ceph cwd unknown /proc/28191/task/80548/cwd (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80548 devicehea ceph rtd unknown /proc/28191/task/80548/root (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80548 devicehea ceph txt unknown /proc/28191/task/80548/exe (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80548 devicehea ceph NOFD /proc/28191/task/80548/fd (opendir: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80587 io_contex ceph cwd unknown /proc/28191/task/80587/cwd (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80587 io_contex ceph rtd unknown /proc/28191/task/80587/root (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80587 io_contex ceph txt unknown /proc/28191/task/80587/exe (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80587 io_contex ceph NOFD /proc/28191/task/80587/fd (opendir: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80588 io_contex ceph cwd unknown /proc/28191/task/80588/cwd (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80588 io_contex ceph rtd unknown /proc/28191/task/80588/root (readlink: Permission denied) 2024-04-21T09:10:05.372 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80588 io_contex ceph txt unknown /proc/28191/task/80588/exe (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80588 io_contex ceph NOFD /proc/28191/task/80588/fd (opendir: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80589 ceph_time ceph cwd unknown /proc/28191/task/80589/cwd (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80589 ceph_time ceph rtd unknown /proc/28191/task/80589/root (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80589 ceph_time ceph txt unknown /proc/28191/task/80589/exe (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80589 ceph_time ceph NOFD /proc/28191/task/80589/fd (opendir: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80590 ms_dispat ceph cwd unknown /proc/28191/task/80590/cwd (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80590 ms_dispat ceph rtd unknown /proc/28191/task/80590/root (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80590 ms_dispat ceph txt unknown /proc/28191/task/80590/exe (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80590 ms_dispat ceph NOFD /proc/28191/task/80590/fd (opendir: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80591 ms_local ceph cwd unknown /proc/28191/task/80591/cwd (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80591 ms_local ceph rtd unknown /proc/28191/task/80591/root (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80591 ms_local ceph txt unknown /proc/28191/task/80591/exe (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80591 ms_local ceph NOFD /proc/28191/task/80591/fd (opendir: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80593 safe_time ceph cwd unknown /proc/28191/task/80593/cwd (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80593 safe_time ceph rtd unknown /proc/28191/task/80593/root (readlink: Permission denied) 2024-04-21T09:10:05.373 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80593 safe_time ceph txt unknown /proc/28191/task/80593/exe (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80593 safe_time ceph NOFD /proc/28191/task/80593/fd (opendir: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80597 safe_time ceph cwd unknown /proc/28191/task/80597/cwd (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80597 safe_time ceph rtd unknown /proc/28191/task/80597/root (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80597 safe_time ceph txt unknown /proc/28191/task/80597/exe (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 80597 safe_time ceph NOFD /proc/28191/task/80597/fd (opendir: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84193 ceph_time ceph cwd unknown /proc/28191/task/84193/cwd (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84193 ceph_time ceph rtd unknown /proc/28191/task/84193/root (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84193 ceph_time ceph txt unknown /proc/28191/task/84193/exe (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84193 ceph_time ceph NOFD /proc/28191/task/84193/fd (opendir: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84194 fn_anonym ceph cwd unknown /proc/28191/task/84194/cwd (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84194 fn_anonym ceph rtd unknown /proc/28191/task/84194/root (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84194 fn_anonym ceph txt unknown /proc/28191/task/84194/exe (readlink: Permission denied) 2024-04-21T09:10:05.374 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84194 fn_anonym ceph NOFD /proc/28191/task/84194/fd (opendir: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84195 safe_time ceph cwd unknown /proc/28191/task/84195/cwd (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84195 safe_time ceph rtd unknown /proc/28191/task/84195/root (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84195 safe_time ceph txt unknown /proc/28191/task/84195/exe (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84195 safe_time ceph NOFD /proc/28191/task/84195/fd (opendir: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84196 fn_anonym ceph cwd unknown /proc/28191/task/84196/cwd (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84196 fn_anonym ceph rtd unknown /proc/28191/task/84196/root (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84196 fn_anonym ceph txt unknown /proc/28191/task/84196/exe (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84196 fn_anonym ceph NOFD /proc/28191/task/84196/fd (opendir: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84197 flusher ceph cwd unknown /proc/28191/task/84197/cwd (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84197 flusher ceph rtd unknown /proc/28191/task/84197/root (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84197 flusher ceph txt unknown /proc/28191/task/84197/exe (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84197 flusher ceph NOFD /proc/28191/task/84197/fd (opendir: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84198 ms_dispat ceph cwd unknown /proc/28191/task/84198/cwd (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84198 ms_dispat ceph rtd unknown /proc/28191/task/84198/root (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84198 ms_dispat ceph txt unknown /proc/28191/task/84198/exe (readlink: Permission denied) 2024-04-21T09:10:05.375 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84198 ms_dispat ceph NOFD /proc/28191/task/84198/fd (opendir: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84199 ms_local ceph cwd unknown /proc/28191/task/84199/cwd (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84199 ms_local ceph rtd unknown /proc/28191/task/84199/root (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84199 ms_local ceph txt unknown /proc/28191/task/84199/exe (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84199 ms_local ceph NOFD /proc/28191/task/84199/fd (opendir: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84200 safe_time ceph cwd unknown /proc/28191/task/84200/cwd (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84200 safe_time ceph rtd unknown /proc/28191/task/84200/root (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84200 safe_time ceph txt unknown /proc/28191/task/84200/exe (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84200 safe_time ceph NOFD /proc/28191/task/84200/fd (opendir: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84201 m-fin-sna ceph cwd unknown /proc/28191/task/84201/cwd (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84201 m-fin-sna ceph rtd unknown /proc/28191/task/84201/root (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84201 m-fin-sna ceph txt unknown /proc/28191/task/84201/exe (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 84201 m-fin-sna ceph NOFD /proc/28191/task/84201/fd (opendir: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 85597 m-fin-sna ceph cwd unknown /proc/28191/task/85597/cwd (readlink: Permission denied) 2024-04-21T09:10:05.376 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 85597 m-fin-sna ceph rtd unknown /proc/28191/task/85597/root (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 85597 m-fin-sna ceph txt unknown /proc/28191/task/85597/exe (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mgr 28191 85597 m-fin-sna ceph NOFD /proc/28191/task/85597/fd (opendir: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28778 root cwd unknown /proc/28778/cwd (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28778 root rtd unknown /proc/28778/root (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28778 root txt unknown /proc/28778/exe (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 28778 root NOFD /proc/28778/fd (opendir: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 29340 root cwd unknown /proc/29340/cwd (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 29340 root rtd unknown /proc/29340/root (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 29340 root txt unknown /proc/29340/exe (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:systemd 29340 root NOFD /proc/29340/fd (opendir: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 29343 root cwd unknown /proc/29343/cwd (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 29343 root rtd unknown /proc/29343/root (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 29343 root txt unknown /proc/29343/exe (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:(sd-pam) 29343 root NOFD /proc/29343/fd (opendir: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 30773 root cwd unknown /proc/30773/cwd (readlink: Permission denied) 2024-04-21T09:10:05.377 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 30773 root rtd unknown /proc/30773/root (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 30773 root txt unknown /proc/30773/exe (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 30773 root NOFD /proc/30773/fd (opendir: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 37074 root cwd unknown /proc/37074/cwd (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 37074 root rtd unknown /proc/37074/root (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 37074 root txt unknown /proc/37074/exe (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 37074 root NOFD /proc/37074/fd (opendir: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 40464 root cwd unknown /proc/40464/cwd (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 40464 root rtd unknown /proc/40464/root (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 40464 root txt unknown /proc/40464/exe (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 40464 root NOFD /proc/40464/fd (opendir: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 40466 root cwd unknown /proc/40466/cwd (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 40466 root rtd unknown /proc/40466/root (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 40466 root txt unknown /proc/40466/exe (readlink: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 40466 root NOFD /proc/40466/fd (opendir: Permission denied) 2024-04-21T09:10:05.378 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 ceph cwd unknown /proc/40468/cwd (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 ceph rtd unknown /proc/40468/root (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 ceph txt unknown /proc/40468/exe (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 ceph NOFD /proc/40468/fd (opendir: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40469 log ceph cwd unknown /proc/40468/task/40469/cwd (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40469 log ceph rtd unknown /proc/40468/task/40469/root (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40469 log ceph txt unknown /proc/40468/task/40469/exe (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40469 log ceph NOFD /proc/40468/task/40469/fd (opendir: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40471 msgr-work ceph cwd unknown /proc/40468/task/40471/cwd (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40471 msgr-work ceph rtd unknown /proc/40468/task/40471/root (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40471 msgr-work ceph txt unknown /proc/40468/task/40471/exe (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40471 msgr-work ceph NOFD /proc/40468/task/40471/fd (opendir: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40472 msgr-work ceph cwd unknown /proc/40468/task/40472/cwd (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40472 msgr-work ceph rtd unknown /proc/40468/task/40472/root (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40472 msgr-work ceph txt unknown /proc/40468/task/40472/exe (readlink: Permission denied) 2024-04-21T09:10:05.379 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40472 msgr-work ceph NOFD /proc/40468/task/40472/fd (opendir: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40473 msgr-work ceph cwd unknown /proc/40468/task/40473/cwd (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40473 msgr-work ceph rtd unknown /proc/40468/task/40473/root (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40473 msgr-work ceph txt unknown /proc/40468/task/40473/exe (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40473 msgr-work ceph NOFD /proc/40468/task/40473/fd (opendir: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40477 service ceph cwd unknown /proc/40468/task/40477/cwd (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40477 service ceph rtd unknown /proc/40468/task/40477/root (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40477 service ceph txt unknown /proc/40468/task/40477/exe (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40477 service ceph NOFD /proc/40468/task/40477/fd (opendir: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40478 admin_soc ceph cwd unknown /proc/40468/task/40478/cwd (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40478 admin_soc ceph rtd unknown /proc/40468/task/40478/root (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40478 admin_soc ceph txt unknown /proc/40468/task/40478/exe (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40478 admin_soc ceph NOFD /proc/40468/task/40478/fd (opendir: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40620 signal_ha ceph cwd unknown /proc/40468/task/40620/cwd (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40620 signal_ha ceph rtd unknown /proc/40468/task/40620/root (readlink: Permission denied) 2024-04-21T09:10:05.380 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40620 signal_ha ceph txt unknown /proc/40468/task/40620/exe (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40620 signal_ha ceph NOFD /proc/40468/task/40620/fd (opendir: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40621 io_contex ceph cwd unknown /proc/40468/task/40621/cwd (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40621 io_contex ceph rtd unknown /proc/40468/task/40621/root (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40621 io_contex ceph txt unknown /proc/40468/task/40621/exe (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40621 io_contex ceph NOFD /proc/40468/task/40621/fd (opendir: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40622 io_contex ceph cwd unknown /proc/40468/task/40622/cwd (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40622 io_contex ceph rtd unknown /proc/40468/task/40622/root (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40622 io_contex ceph txt unknown /proc/40468/task/40622/exe (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40622 io_contex ceph NOFD /proc/40468/task/40622/fd (opendir: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40734 OpHistory ceph cwd unknown /proc/40468/task/40734/cwd (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40734 OpHistory ceph rtd unknown /proc/40468/task/40734/root (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40734 OpHistory ceph txt unknown /proc/40468/task/40734/exe (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40734 OpHistory ceph NOFD /proc/40468/task/40734/fd (opendir: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40735 ceph_time ceph cwd unknown /proc/40468/task/40735/cwd (readlink: Permission denied) 2024-04-21T09:10:05.381 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40735 ceph_time ceph rtd unknown /proc/40468/task/40735/root (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40735 ceph_time ceph txt unknown /proc/40468/task/40735/exe (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40735 ceph_time ceph NOFD /proc/40468/task/40735/fd (opendir: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40757 ceph-osd ceph cwd unknown /proc/40468/task/40757/cwd (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40757 ceph-osd ceph rtd unknown /proc/40468/task/40757/root (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40757 ceph-osd ceph txt unknown /proc/40468/task/40757/exe (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40757 ceph-osd ceph NOFD /proc/40468/task/40757/fd (opendir: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40773 ceph-osd ceph cwd unknown /proc/40468/task/40773/cwd (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40773 ceph-osd ceph rtd unknown /proc/40468/task/40773/root (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40773 ceph-osd ceph txt unknown /proc/40468/task/40773/exe (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40773 ceph-osd ceph NOFD /proc/40468/task/40773/fd (opendir: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40793 ceph-osd ceph cwd unknown /proc/40468/task/40793/cwd (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40793 ceph-osd ceph rtd unknown /proc/40468/task/40793/root (readlink: Permission denied) 2024-04-21T09:10:05.382 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40793 ceph-osd ceph txt unknown /proc/40468/task/40793/exe (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40793 ceph-osd ceph NOFD /proc/40468/task/40793/fd (opendir: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40817 ceph-osd ceph cwd unknown /proc/40468/task/40817/cwd (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40817 ceph-osd ceph rtd unknown /proc/40468/task/40817/root (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40817 ceph-osd ceph txt unknown /proc/40468/task/40817/exe (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40817 ceph-osd ceph NOFD /proc/40468/task/40817/fd (opendir: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40832 ceph-osd ceph cwd unknown /proc/40468/task/40832/cwd (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40832 ceph-osd ceph rtd unknown /proc/40468/task/40832/root (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40832 ceph-osd ceph txt unknown /proc/40468/task/40832/exe (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40832 ceph-osd ceph NOFD /proc/40468/task/40832/fd (opendir: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40849 ceph-osd ceph cwd unknown /proc/40468/task/40849/cwd (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40849 ceph-osd ceph rtd unknown /proc/40468/task/40849/root (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40849 ceph-osd ceph txt unknown /proc/40468/task/40849/exe (readlink: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40849 ceph-osd ceph NOFD /proc/40468/task/40849/fd (opendir: Permission denied) 2024-04-21T09:10:05.383 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40868 ceph-osd ceph cwd unknown /proc/40468/task/40868/cwd (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40868 ceph-osd ceph rtd unknown /proc/40468/task/40868/root (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40868 ceph-osd ceph txt unknown /proc/40468/task/40868/exe (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40868 ceph-osd ceph NOFD /proc/40468/task/40868/fd (opendir: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40897 ceph-osd ceph cwd unknown /proc/40468/task/40897/cwd (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40897 ceph-osd ceph rtd unknown /proc/40468/task/40897/root (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40897 ceph-osd ceph txt unknown /proc/40468/task/40897/exe (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40897 ceph-osd ceph NOFD /proc/40468/task/40897/fd (opendir: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40899 ceph-osd ceph cwd unknown /proc/40468/task/40899/cwd (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40899 ceph-osd ceph rtd unknown /proc/40468/task/40899/root (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40899 ceph-osd ceph txt unknown /proc/40468/task/40899/exe (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40899 ceph-osd ceph NOFD /proc/40468/task/40899/fd (opendir: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40900 safe_time ceph cwd unknown /proc/40468/task/40900/cwd (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40900 safe_time ceph rtd unknown /proc/40468/task/40900/root (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40900 safe_time ceph txt unknown /proc/40468/task/40900/exe (readlink: Permission denied) 2024-04-21T09:10:05.384 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40900 safe_time ceph NOFD /proc/40468/task/40900/fd (opendir: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40901 safe_time ceph cwd unknown /proc/40468/task/40901/cwd (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40901 safe_time ceph rtd unknown /proc/40468/task/40901/root (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40901 safe_time ceph txt unknown /proc/40468/task/40901/exe (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40901 safe_time ceph NOFD /proc/40468/task/40901/fd (opendir: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40902 safe_time ceph cwd unknown /proc/40468/task/40902/cwd (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40902 safe_time ceph rtd unknown /proc/40468/task/40902/root (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40902 safe_time ceph txt unknown /proc/40468/task/40902/exe (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40902 safe_time ceph NOFD /proc/40468/task/40902/fd (opendir: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40903 safe_time ceph cwd unknown /proc/40468/task/40903/cwd (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40903 safe_time ceph rtd unknown /proc/40468/task/40903/root (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40903 safe_time ceph txt unknown /proc/40468/task/40903/exe (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40903 safe_time ceph NOFD /proc/40468/task/40903/fd (opendir: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40904 fn_anonym ceph cwd unknown /proc/40468/task/40904/cwd (readlink: Permission denied) 2024-04-21T09:10:05.385 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40904 fn_anonym ceph rtd unknown /proc/40468/task/40904/root (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40904 fn_anonym ceph txt unknown /proc/40468/task/40904/exe (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40904 fn_anonym ceph NOFD /proc/40468/task/40904/fd (opendir: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40908 rocksdb:l ceph cwd unknown /proc/40468/task/40908/cwd (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40908 rocksdb:l ceph rtd unknown /proc/40468/task/40908/root (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40908 rocksdb:l ceph txt unknown /proc/40468/task/40908/exe (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40908 rocksdb:l ceph NOFD /proc/40468/task/40908/fd (opendir: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40909 rocksdb:l ceph cwd unknown /proc/40468/task/40909/cwd (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40909 rocksdb:l ceph rtd unknown /proc/40468/task/40909/root (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40909 rocksdb:l ceph txt unknown /proc/40468/task/40909/exe (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40909 rocksdb:l ceph NOFD /proc/40468/task/40909/fd (opendir: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40910 rocksdb:l ceph cwd unknown /proc/40468/task/40910/cwd (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40910 rocksdb:l ceph rtd unknown /proc/40468/task/40910/root (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40910 rocksdb:l ceph txt unknown /proc/40468/task/40910/exe (readlink: Permission denied) 2024-04-21T09:10:05.386 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40910 rocksdb:l ceph NOFD /proc/40468/task/40910/fd (opendir: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40911 rocksdb:h ceph cwd unknown /proc/40468/task/40911/cwd (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40911 rocksdb:h ceph rtd unknown /proc/40468/task/40911/root (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40911 rocksdb:h ceph txt unknown /proc/40468/task/40911/exe (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 40911 rocksdb:h ceph NOFD /proc/40468/task/40911/fd (opendir: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41398 bstore_ai ceph cwd unknown /proc/40468/task/41398/cwd (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41398 bstore_ai ceph rtd unknown /proc/40468/task/41398/root (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41398 bstore_ai ceph txt unknown /proc/40468/task/41398/exe (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41398 bstore_ai ceph NOFD /proc/40468/task/41398/fd (opendir: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41582 bstore_ai ceph cwd unknown /proc/40468/task/41582/cwd (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41582 bstore_ai ceph rtd unknown /proc/40468/task/41582/root (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41582 bstore_ai ceph txt unknown /proc/40468/task/41582/exe (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41582 bstore_ai ceph NOFD /proc/40468/task/41582/fd (opendir: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41764 ceph-osd ceph cwd unknown /proc/40468/task/41764/cwd (readlink: Permission denied) 2024-04-21T09:10:05.387 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41764 ceph-osd ceph rtd unknown /proc/40468/task/41764/root (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41764 ceph-osd ceph txt unknown /proc/40468/task/41764/exe (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41764 ceph-osd ceph NOFD /proc/40468/task/41764/fd (opendir: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41765 cfin ceph cwd unknown /proc/40468/task/41765/cwd (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41765 cfin ceph rtd unknown /proc/40468/task/41765/root (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41765 cfin ceph txt unknown /proc/40468/task/41765/exe (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41765 cfin ceph NOFD /proc/40468/task/41765/fd (opendir: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41766 bstore_kv ceph cwd unknown /proc/40468/task/41766/cwd (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41766 bstore_kv ceph rtd unknown /proc/40468/task/41766/root (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41766 bstore_kv ceph txt unknown /proc/40468/task/41766/exe (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41766 bstore_kv ceph NOFD /proc/40468/task/41766/fd (opendir: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41767 bstore_kv ceph cwd unknown /proc/40468/task/41767/cwd (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41767 bstore_kv ceph rtd unknown /proc/40468/task/41767/root (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41767 bstore_kv ceph txt unknown /proc/40468/task/41767/exe (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41767 bstore_kv ceph NOFD /proc/40468/task/41767/fd (opendir: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41768 bstore_me ceph cwd unknown /proc/40468/task/41768/cwd (readlink: Permission denied) 2024-04-21T09:10:05.388 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41768 bstore_me ceph rtd unknown /proc/40468/task/41768/root (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41768 bstore_me ceph txt unknown /proc/40468/task/41768/exe (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41768 bstore_me ceph NOFD /proc/40468/task/41768/fd (opendir: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41769 ms_dispat ceph cwd unknown /proc/40468/task/41769/cwd (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41769 ms_dispat ceph rtd unknown /proc/40468/task/41769/root (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41769 ms_dispat ceph txt unknown /proc/40468/task/41769/exe (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41769 ms_dispat ceph NOFD /proc/40468/task/41769/fd (opendir: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41770 ms_local ceph cwd unknown /proc/40468/task/41770/cwd (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41770 ms_local ceph rtd unknown /proc/40468/task/41770/root (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41770 ms_local ceph txt unknown /proc/40468/task/41770/exe (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41770 ms_local ceph NOFD /proc/40468/task/41770/fd (opendir: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41771 safe_time ceph cwd unknown /proc/40468/task/41771/cwd (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41771 safe_time ceph rtd unknown /proc/40468/task/41771/root (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41771 safe_time ceph txt unknown /proc/40468/task/41771/exe (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41771 safe_time ceph NOFD /proc/40468/task/41771/fd (opendir: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41772 safe_time ceph cwd unknown /proc/40468/task/41772/cwd (readlink: Permission denied) 2024-04-21T09:10:05.389 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41772 safe_time ceph rtd unknown /proc/40468/task/41772/root (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41772 safe_time ceph txt unknown /proc/40468/task/41772/exe (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41772 safe_time ceph NOFD /proc/40468/task/41772/fd (opendir: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41773 ms_dispat ceph cwd unknown /proc/40468/task/41773/cwd (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41773 ms_dispat ceph rtd unknown /proc/40468/task/41773/root (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41773 ms_dispat ceph txt unknown /proc/40468/task/41773/exe (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41773 ms_dispat ceph NOFD /proc/40468/task/41773/fd (opendir: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41774 ms_local ceph cwd unknown /proc/40468/task/41774/cwd (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41774 ms_local ceph rtd unknown /proc/40468/task/41774/root (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41774 ms_local ceph txt unknown /proc/40468/task/41774/exe (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41774 ms_local ceph NOFD /proc/40468/task/41774/fd (opendir: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41775 ms_dispat ceph cwd unknown /proc/40468/task/41775/cwd (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41775 ms_dispat ceph rtd unknown /proc/40468/task/41775/root (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41775 ms_dispat ceph txt unknown /proc/40468/task/41775/exe (readlink: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41775 ms_dispat ceph NOFD /proc/40468/task/41775/fd (opendir: Permission denied) 2024-04-21T09:10:05.390 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41776 ms_local ceph cwd unknown /proc/40468/task/41776/cwd (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41776 ms_local ceph rtd unknown /proc/40468/task/41776/root (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41776 ms_local ceph txt unknown /proc/40468/task/41776/exe (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41776 ms_local ceph NOFD /proc/40468/task/41776/fd (opendir: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41777 ms_dispat ceph cwd unknown /proc/40468/task/41777/cwd (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41777 ms_dispat ceph rtd unknown /proc/40468/task/41777/root (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41777 ms_dispat ceph txt unknown /proc/40468/task/41777/exe (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41777 ms_dispat ceph NOFD /proc/40468/task/41777/fd (opendir: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41778 ms_local ceph cwd unknown /proc/40468/task/41778/cwd (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41778 ms_local ceph rtd unknown /proc/40468/task/41778/root (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41778 ms_local ceph txt unknown /proc/40468/task/41778/exe (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41778 ms_local ceph NOFD /proc/40468/task/41778/fd (opendir: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41779 ms_dispat ceph cwd unknown /proc/40468/task/41779/cwd (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41779 ms_dispat ceph rtd unknown /proc/40468/task/41779/root (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41779 ms_dispat ceph txt unknown /proc/40468/task/41779/exe (readlink: Permission denied) 2024-04-21T09:10:05.391 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41779 ms_dispat ceph NOFD /proc/40468/task/41779/fd (opendir: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41780 ms_local ceph cwd unknown /proc/40468/task/41780/cwd (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41780 ms_local ceph rtd unknown /proc/40468/task/41780/root (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41780 ms_local ceph txt unknown /proc/40468/task/41780/exe (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41780 ms_local ceph NOFD /proc/40468/task/41780/fd (opendir: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41781 ms_dispat ceph cwd unknown /proc/40468/task/41781/cwd (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41781 ms_dispat ceph rtd unknown /proc/40468/task/41781/root (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41781 ms_dispat ceph txt unknown /proc/40468/task/41781/exe (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41781 ms_dispat ceph NOFD /proc/40468/task/41781/fd (opendir: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41782 ms_local ceph cwd unknown /proc/40468/task/41782/cwd (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41782 ms_local ceph rtd unknown /proc/40468/task/41782/root (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41782 ms_local ceph txt unknown /proc/40468/task/41782/exe (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41782 ms_local ceph NOFD /proc/40468/task/41782/fd (opendir: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41783 ms_dispat ceph cwd unknown /proc/40468/task/41783/cwd (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41783 ms_dispat ceph rtd unknown /proc/40468/task/41783/root (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41783 ms_dispat ceph txt unknown /proc/40468/task/41783/exe (readlink: Permission denied) 2024-04-21T09:10:05.392 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41783 ms_dispat ceph NOFD /proc/40468/task/41783/fd (opendir: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41784 ms_local ceph cwd unknown /proc/40468/task/41784/cwd (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41784 ms_local ceph rtd unknown /proc/40468/task/41784/root (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41784 ms_local ceph txt unknown /proc/40468/task/41784/exe (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41784 ms_local ceph NOFD /proc/40468/task/41784/fd (opendir: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41785 fn_anonym ceph cwd unknown /proc/40468/task/41785/cwd (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41785 fn_anonym ceph rtd unknown /proc/40468/task/41785/root (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41785 fn_anonym ceph txt unknown /proc/40468/task/41785/exe (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41785 fn_anonym ceph NOFD /proc/40468/task/41785/fd (opendir: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41786 finisher ceph cwd unknown /proc/40468/task/41786/cwd (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41786 finisher ceph rtd unknown /proc/40468/task/41786/root (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41786 finisher ceph txt unknown /proc/40468/task/41786/exe (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41786 finisher ceph NOFD /proc/40468/task/41786/fd (opendir: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41787 safe_time ceph cwd unknown /proc/40468/task/41787/cwd (readlink: Permission denied) 2024-04-21T09:10:05.393 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41787 safe_time ceph rtd unknown /proc/40468/task/41787/root (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41787 safe_time ceph txt unknown /proc/40468/task/41787/exe (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41787 safe_time ceph NOFD /proc/40468/task/41787/fd (opendir: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41788 safe_time ceph cwd unknown /proc/40468/task/41788/cwd (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41788 safe_time ceph rtd unknown /proc/40468/task/41788/root (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41788 safe_time ceph txt unknown /proc/40468/task/41788/exe (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41788 safe_time ceph NOFD /proc/40468/task/41788/fd (opendir: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41789 ceph-osd ceph cwd unknown /proc/40468/task/41789/cwd (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41789 ceph-osd ceph rtd unknown /proc/40468/task/41789/root (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41789 ceph-osd ceph txt unknown /proc/40468/task/41789/exe (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41789 ceph-osd ceph NOFD /proc/40468/task/41789/fd (opendir: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41790 osd_srv_a ceph cwd unknown /proc/40468/task/41790/cwd (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41790 osd_srv_a ceph rtd unknown /proc/40468/task/41790/root (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41790 osd_srv_a ceph txt unknown /proc/40468/task/41790/exe (readlink: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41790 osd_srv_a ceph NOFD /proc/40468/task/41790/fd (opendir: Permission denied) 2024-04-21T09:10:05.394 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41791 tp_osd_tp ceph cwd unknown /proc/40468/task/41791/cwd (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41791 tp_osd_tp ceph rtd unknown /proc/40468/task/41791/root (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41791 tp_osd_tp ceph txt unknown /proc/40468/task/41791/exe (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41791 tp_osd_tp ceph NOFD /proc/40468/task/41791/fd (opendir: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41792 tp_osd_tp ceph cwd unknown /proc/40468/task/41792/cwd (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41792 tp_osd_tp ceph rtd unknown /proc/40468/task/41792/root (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41792 tp_osd_tp ceph txt unknown /proc/40468/task/41792/exe (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41792 tp_osd_tp ceph NOFD /proc/40468/task/41792/fd (opendir: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41793 tp_osd_tp ceph cwd unknown /proc/40468/task/41793/cwd (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41793 tp_osd_tp ceph rtd unknown /proc/40468/task/41793/root (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41793 tp_osd_tp ceph txt unknown /proc/40468/task/41793/exe (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41793 tp_osd_tp ceph NOFD /proc/40468/task/41793/fd (opendir: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41794 tp_osd_tp ceph cwd unknown /proc/40468/task/41794/cwd (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41794 tp_osd_tp ceph rtd unknown /proc/40468/task/41794/root (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41794 tp_osd_tp ceph txt unknown /proc/40468/task/41794/exe (readlink: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41794 tp_osd_tp ceph NOFD /proc/40468/task/41794/fd (opendir: Permission denied) 2024-04-21T09:10:05.395 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41795 tp_osd_tp ceph cwd unknown /proc/40468/task/41795/cwd (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41795 tp_osd_tp ceph rtd unknown /proc/40468/task/41795/root (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41795 tp_osd_tp ceph txt unknown /proc/40468/task/41795/exe (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41795 tp_osd_tp ceph NOFD /proc/40468/task/41795/fd (opendir: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41796 tp_osd_tp ceph cwd unknown /proc/40468/task/41796/cwd (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41796 tp_osd_tp ceph rtd unknown /proc/40468/task/41796/root (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41796 tp_osd_tp ceph txt unknown /proc/40468/task/41796/exe (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41796 tp_osd_tp ceph NOFD /proc/40468/task/41796/fd (opendir: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41797 tp_osd_tp ceph cwd unknown /proc/40468/task/41797/cwd (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41797 tp_osd_tp ceph rtd unknown /proc/40468/task/41797/root (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41797 tp_osd_tp ceph txt unknown /proc/40468/task/41797/exe (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41797 tp_osd_tp ceph NOFD /proc/40468/task/41797/fd (opendir: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41798 tp_osd_tp ceph cwd unknown /proc/40468/task/41798/cwd (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41798 tp_osd_tp ceph rtd unknown /proc/40468/task/41798/root (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41798 tp_osd_tp ceph txt unknown /proc/40468/task/41798/exe (readlink: Permission denied) 2024-04-21T09:10:05.396 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41798 tp_osd_tp ceph NOFD /proc/40468/task/41798/fd (opendir: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41799 tp_osd_tp ceph cwd unknown /proc/40468/task/41799/cwd (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41799 tp_osd_tp ceph rtd unknown /proc/40468/task/41799/root (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41799 tp_osd_tp ceph txt unknown /proc/40468/task/41799/exe (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41799 tp_osd_tp ceph NOFD /proc/40468/task/41799/fd (opendir: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41800 tp_osd_tp ceph cwd unknown /proc/40468/task/41800/cwd (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41800 tp_osd_tp ceph rtd unknown /proc/40468/task/41800/root (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41800 tp_osd_tp ceph txt unknown /proc/40468/task/41800/exe (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41800 tp_osd_tp ceph NOFD /proc/40468/task/41800/fd (opendir: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41801 tp_osd_tp ceph cwd unknown /proc/40468/task/41801/cwd (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41801 tp_osd_tp ceph rtd unknown /proc/40468/task/41801/root (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41801 tp_osd_tp ceph txt unknown /proc/40468/task/41801/exe (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41801 tp_osd_tp ceph NOFD /proc/40468/task/41801/fd (opendir: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41802 tp_osd_tp ceph cwd unknown /proc/40468/task/41802/cwd (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41802 tp_osd_tp ceph rtd unknown /proc/40468/task/41802/root (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41802 tp_osd_tp ceph txt unknown /proc/40468/task/41802/exe (readlink: Permission denied) 2024-04-21T09:10:05.397 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41802 tp_osd_tp ceph NOFD /proc/40468/task/41802/fd (opendir: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41803 tp_osd_tp ceph cwd unknown /proc/40468/task/41803/cwd (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41803 tp_osd_tp ceph rtd unknown /proc/40468/task/41803/root (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41803 tp_osd_tp ceph txt unknown /proc/40468/task/41803/exe (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41803 tp_osd_tp ceph NOFD /proc/40468/task/41803/fd (opendir: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41804 tp_osd_tp ceph cwd unknown /proc/40468/task/41804/cwd (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41804 tp_osd_tp ceph rtd unknown /proc/40468/task/41804/root (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41804 tp_osd_tp ceph txt unknown /proc/40468/task/41804/exe (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41804 tp_osd_tp ceph NOFD /proc/40468/task/41804/fd (opendir: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41805 tp_osd_tp ceph cwd unknown /proc/40468/task/41805/cwd (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41805 tp_osd_tp ceph rtd unknown /proc/40468/task/41805/root (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41805 tp_osd_tp ceph txt unknown /proc/40468/task/41805/exe (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41805 tp_osd_tp ceph NOFD /proc/40468/task/41805/fd (opendir: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41806 tp_osd_tp ceph cwd unknown /proc/40468/task/41806/cwd (readlink: Permission denied) 2024-04-21T09:10:05.398 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41806 tp_osd_tp ceph rtd unknown /proc/40468/task/41806/root (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41806 tp_osd_tp ceph txt unknown /proc/40468/task/41806/exe (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41806 tp_osd_tp ceph NOFD /proc/40468/task/41806/fd (opendir: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41807 osd_srv_h ceph cwd unknown /proc/40468/task/41807/cwd (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41807 osd_srv_h ceph rtd unknown /proc/40468/task/41807/root (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41807 osd_srv_h ceph txt unknown /proc/40468/task/41807/exe (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 40468 41807 osd_srv_h ceph NOFD /proc/40468/task/41807/fd (opendir: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 41820 root cwd unknown /proc/41820/cwd (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 41820 root rtd unknown /proc/41820/root (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 41820 root txt unknown /proc/41820/exe (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 41820 root NOFD /proc/41820/fd (opendir: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 41837 root cwd unknown /proc/41837/cwd (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 41837 root rtd unknown /proc/41837/root (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 41837 root txt unknown /proc/41837/exe (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 41837 root NOFD /proc/41837/fd (opendir: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:anacron 44614 root cwd unknown /proc/44614/cwd (readlink: Permission denied) 2024-04-21T09:10:05.399 INFO:tasks.workunit.client.0.smithi066.stdout:anacron 44614 root rtd unknown /proc/44614/root (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:anacron 44614 root txt unknown /proc/44614/exe (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:anacron 44614 root NOFD /proc/44614/fd (opendir: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 44868 root cwd unknown /proc/44868/cwd (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 44868 root rtd unknown /proc/44868/root (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 44868 root txt unknown /proc/44868/exe (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 44868 root NOFD /proc/44868/fd (opendir: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 46584 root cwd unknown /proc/46584/cwd (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 46584 root rtd unknown /proc/46584/root (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 46584 root txt unknown /proc/46584/exe (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 46584 root NOFD /proc/46584/fd (opendir: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 46589 root cwd unknown /proc/46589/cwd (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 46589 root rtd unknown /proc/46589/root (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 46589 root txt unknown /proc/46589/exe (readlink: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 46589 root NOFD /proc/46589/fd (opendir: Permission denied) 2024-04-21T09:10:05.400 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 47339 root cwd unknown /proc/47339/cwd (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 47339 root rtd unknown /proc/47339/root (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 47339 root txt unknown /proc/47339/exe (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 47339 root NOFD /proc/47339/fd (opendir: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 47341 root cwd unknown /proc/47341/cwd (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 47341 root rtd unknown /proc/47341/root (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 47341 root txt unknown /proc/47341/exe (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 47341 root NOFD /proc/47341/fd (opendir: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 ceph cwd unknown /proc/47344/cwd (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 ceph rtd unknown /proc/47344/root (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 ceph txt unknown /proc/47344/exe (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 ceph NOFD /proc/47344/fd (opendir: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47345 log ceph cwd unknown /proc/47344/task/47345/cwd (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47345 log ceph rtd unknown /proc/47344/task/47345/root (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47345 log ceph txt unknown /proc/47344/task/47345/exe (readlink: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47345 log ceph NOFD /proc/47344/task/47345/fd (opendir: Permission denied) 2024-04-21T09:10:05.401 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47347 msgr-work ceph cwd unknown /proc/47344/task/47347/cwd (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47347 msgr-work ceph rtd unknown /proc/47344/task/47347/root (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47347 msgr-work ceph txt unknown /proc/47344/task/47347/exe (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47347 msgr-work ceph NOFD /proc/47344/task/47347/fd (opendir: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47348 msgr-work ceph cwd unknown /proc/47344/task/47348/cwd (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47348 msgr-work ceph rtd unknown /proc/47344/task/47348/root (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47348 msgr-work ceph txt unknown /proc/47344/task/47348/exe (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47348 msgr-work ceph NOFD /proc/47344/task/47348/fd (opendir: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47349 msgr-work ceph cwd unknown /proc/47344/task/47349/cwd (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47349 msgr-work ceph rtd unknown /proc/47344/task/47349/root (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47349 msgr-work ceph txt unknown /proc/47344/task/47349/exe (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47349 msgr-work ceph NOFD /proc/47344/task/47349/fd (opendir: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47353 service ceph cwd unknown /proc/47344/task/47353/cwd (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47353 service ceph rtd unknown /proc/47344/task/47353/root (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47353 service ceph txt unknown /proc/47344/task/47353/exe (readlink: Permission denied) 2024-04-21T09:10:05.402 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47353 service ceph NOFD /proc/47344/task/47353/fd (opendir: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47354 admin_soc ceph cwd unknown /proc/47344/task/47354/cwd (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47354 admin_soc ceph rtd unknown /proc/47344/task/47354/root (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47354 admin_soc ceph txt unknown /proc/47344/task/47354/exe (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47354 admin_soc ceph NOFD /proc/47344/task/47354/fd (opendir: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47496 signal_ha ceph cwd unknown /proc/47344/task/47496/cwd (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47496 signal_ha ceph rtd unknown /proc/47344/task/47496/root (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47496 signal_ha ceph txt unknown /proc/47344/task/47496/exe (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47496 signal_ha ceph NOFD /proc/47344/task/47496/fd (opendir: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47497 io_contex ceph cwd unknown /proc/47344/task/47497/cwd (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47497 io_contex ceph rtd unknown /proc/47344/task/47497/root (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47497 io_contex ceph txt unknown /proc/47344/task/47497/exe (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47497 io_contex ceph NOFD /proc/47344/task/47497/fd (opendir: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47498 io_contex ceph cwd unknown /proc/47344/task/47498/cwd (readlink: Permission denied) 2024-04-21T09:10:05.403 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47498 io_contex ceph rtd unknown /proc/47344/task/47498/root (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47498 io_contex ceph txt unknown /proc/47344/task/47498/exe (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47498 io_contex ceph NOFD /proc/47344/task/47498/fd (opendir: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47614 OpHistory ceph cwd unknown /proc/47344/task/47614/cwd (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47614 OpHistory ceph rtd unknown /proc/47344/task/47614/root (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47614 OpHistory ceph txt unknown /proc/47344/task/47614/exe (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47614 OpHistory ceph NOFD /proc/47344/task/47614/fd (opendir: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47615 ceph_time ceph cwd unknown /proc/47344/task/47615/cwd (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47615 ceph_time ceph rtd unknown /proc/47344/task/47615/root (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47615 ceph_time ceph txt unknown /proc/47344/task/47615/exe (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47615 ceph_time ceph NOFD /proc/47344/task/47615/fd (opendir: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47787 ceph-osd ceph cwd unknown /proc/47344/task/47787/cwd (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47787 ceph-osd ceph rtd unknown /proc/47344/task/47787/root (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47787 ceph-osd ceph txt unknown /proc/47344/task/47787/exe (readlink: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47787 ceph-osd ceph NOFD /proc/47344/task/47787/fd (opendir: Permission denied) 2024-04-21T09:10:05.404 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47788 safe_time ceph cwd unknown /proc/47344/task/47788/cwd (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47788 safe_time ceph rtd unknown /proc/47344/task/47788/root (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47788 safe_time ceph txt unknown /proc/47344/task/47788/exe (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47788 safe_time ceph NOFD /proc/47344/task/47788/fd (opendir: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47789 safe_time ceph cwd unknown /proc/47344/task/47789/cwd (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47789 safe_time ceph rtd unknown /proc/47344/task/47789/root (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47789 safe_time ceph txt unknown /proc/47344/task/47789/exe (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47789 safe_time ceph NOFD /proc/47344/task/47789/fd (opendir: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47790 safe_time ceph cwd unknown /proc/47344/task/47790/cwd (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47790 safe_time ceph rtd unknown /proc/47344/task/47790/root (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47790 safe_time ceph txt unknown /proc/47344/task/47790/exe (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47790 safe_time ceph NOFD /proc/47344/task/47790/fd (opendir: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47791 safe_time ceph cwd unknown /proc/47344/task/47791/cwd (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47791 safe_time ceph rtd unknown /proc/47344/task/47791/root (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47791 safe_time ceph txt unknown /proc/47344/task/47791/exe (readlink: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47791 safe_time ceph NOFD /proc/47344/task/47791/fd (opendir: Permission denied) 2024-04-21T09:10:05.405 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47792 fn_anonym ceph cwd unknown /proc/47344/task/47792/cwd (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47792 fn_anonym ceph rtd unknown /proc/47344/task/47792/root (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47792 fn_anonym ceph txt unknown /proc/47344/task/47792/exe (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47792 fn_anonym ceph NOFD /proc/47344/task/47792/fd (opendir: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47797 rocksdb:l ceph cwd unknown /proc/47344/task/47797/cwd (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47797 rocksdb:l ceph rtd unknown /proc/47344/task/47797/root (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47797 rocksdb:l ceph txt unknown /proc/47344/task/47797/exe (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47797 rocksdb:l ceph NOFD /proc/47344/task/47797/fd (opendir: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47798 rocksdb:l ceph cwd unknown /proc/47344/task/47798/cwd (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47798 rocksdb:l ceph rtd unknown /proc/47344/task/47798/root (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47798 rocksdb:l ceph txt unknown /proc/47344/task/47798/exe (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47798 rocksdb:l ceph NOFD /proc/47344/task/47798/fd (opendir: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47799 rocksdb:l ceph cwd unknown /proc/47344/task/47799/cwd (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47799 rocksdb:l ceph rtd unknown /proc/47344/task/47799/root (readlink: Permission denied) 2024-04-21T09:10:05.406 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47799 rocksdb:l ceph txt unknown /proc/47344/task/47799/exe (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47799 rocksdb:l ceph NOFD /proc/47344/task/47799/fd (opendir: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47800 rocksdb:h ceph cwd unknown /proc/47344/task/47800/cwd (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47800 rocksdb:h ceph rtd unknown /proc/47344/task/47800/root (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47800 rocksdb:h ceph txt unknown /proc/47344/task/47800/exe (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 47800 rocksdb:h ceph NOFD /proc/47344/task/47800/fd (opendir: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48241 bstore_ai ceph cwd unknown /proc/47344/task/48241/cwd (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48241 bstore_ai ceph rtd unknown /proc/47344/task/48241/root (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48241 bstore_ai ceph txt unknown /proc/47344/task/48241/exe (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48241 bstore_ai ceph NOFD /proc/47344/task/48241/fd (opendir: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48472 bstore_ai ceph cwd unknown /proc/47344/task/48472/cwd (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48472 bstore_ai ceph rtd unknown /proc/47344/task/48472/root (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48472 bstore_ai ceph txt unknown /proc/47344/task/48472/exe (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48472 bstore_ai ceph NOFD /proc/47344/task/48472/fd (opendir: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48668 ceph-osd ceph cwd unknown /proc/47344/task/48668/cwd (readlink: Permission denied) 2024-04-21T09:10:05.407 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48668 ceph-osd ceph rtd unknown /proc/47344/task/48668/root (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48668 ceph-osd ceph txt unknown /proc/47344/task/48668/exe (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48668 ceph-osd ceph NOFD /proc/47344/task/48668/fd (opendir: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48669 cfin ceph cwd unknown /proc/47344/task/48669/cwd (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48669 cfin ceph rtd unknown /proc/47344/task/48669/root (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48669 cfin ceph txt unknown /proc/47344/task/48669/exe (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48669 cfin ceph NOFD /proc/47344/task/48669/fd (opendir: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48670 bstore_kv ceph cwd unknown /proc/47344/task/48670/cwd (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48670 bstore_kv ceph rtd unknown /proc/47344/task/48670/root (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48670 bstore_kv ceph txt unknown /proc/47344/task/48670/exe (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48670 bstore_kv ceph NOFD /proc/47344/task/48670/fd (opendir: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48671 bstore_kv ceph cwd unknown /proc/47344/task/48671/cwd (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48671 bstore_kv ceph rtd unknown /proc/47344/task/48671/root (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48671 bstore_kv ceph txt unknown /proc/47344/task/48671/exe (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48671 bstore_kv ceph NOFD /proc/47344/task/48671/fd (opendir: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48672 bstore_me ceph cwd unknown /proc/47344/task/48672/cwd (readlink: Permission denied) 2024-04-21T09:10:05.408 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48672 bstore_me ceph rtd unknown /proc/47344/task/48672/root (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48672 bstore_me ceph txt unknown /proc/47344/task/48672/exe (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48672 bstore_me ceph NOFD /proc/47344/task/48672/fd (opendir: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48682 ms_dispat ceph cwd unknown /proc/47344/task/48682/cwd (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48682 ms_dispat ceph rtd unknown /proc/47344/task/48682/root (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48682 ms_dispat ceph txt unknown /proc/47344/task/48682/exe (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48682 ms_dispat ceph NOFD /proc/47344/task/48682/fd (opendir: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48683 ms_local ceph cwd unknown /proc/47344/task/48683/cwd (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48683 ms_local ceph rtd unknown /proc/47344/task/48683/root (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48683 ms_local ceph txt unknown /proc/47344/task/48683/exe (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48683 ms_local ceph NOFD /proc/47344/task/48683/fd (opendir: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48684 safe_time ceph cwd unknown /proc/47344/task/48684/cwd (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48684 safe_time ceph rtd unknown /proc/47344/task/48684/root (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48684 safe_time ceph txt unknown /proc/47344/task/48684/exe (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48684 safe_time ceph NOFD /proc/47344/task/48684/fd (opendir: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48686 safe_time ceph cwd unknown /proc/47344/task/48686/cwd (readlink: Permission denied) 2024-04-21T09:10:05.409 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48686 safe_time ceph rtd unknown /proc/47344/task/48686/root (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48686 safe_time ceph txt unknown /proc/47344/task/48686/exe (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48686 safe_time ceph NOFD /proc/47344/task/48686/fd (opendir: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48687 ms_dispat ceph cwd unknown /proc/47344/task/48687/cwd (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48687 ms_dispat ceph rtd unknown /proc/47344/task/48687/root (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48687 ms_dispat ceph txt unknown /proc/47344/task/48687/exe (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48687 ms_dispat ceph NOFD /proc/47344/task/48687/fd (opendir: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48688 ms_local ceph cwd unknown /proc/47344/task/48688/cwd (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48688 ms_local ceph rtd unknown /proc/47344/task/48688/root (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48688 ms_local ceph txt unknown /proc/47344/task/48688/exe (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48688 ms_local ceph NOFD /proc/47344/task/48688/fd (opendir: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48689 ms_dispat ceph cwd unknown /proc/47344/task/48689/cwd (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48689 ms_dispat ceph rtd unknown /proc/47344/task/48689/root (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48689 ms_dispat ceph txt unknown /proc/47344/task/48689/exe (readlink: Permission denied) 2024-04-21T09:10:05.410 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48689 ms_dispat ceph NOFD /proc/47344/task/48689/fd (opendir: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48690 ms_local ceph cwd unknown /proc/47344/task/48690/cwd (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48690 ms_local ceph rtd unknown /proc/47344/task/48690/root (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48690 ms_local ceph txt unknown /proc/47344/task/48690/exe (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48690 ms_local ceph NOFD /proc/47344/task/48690/fd (opendir: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48691 ms_dispat ceph cwd unknown /proc/47344/task/48691/cwd (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48691 ms_dispat ceph rtd unknown /proc/47344/task/48691/root (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48691 ms_dispat ceph txt unknown /proc/47344/task/48691/exe (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48691 ms_dispat ceph NOFD /proc/47344/task/48691/fd (opendir: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48692 ms_local ceph cwd unknown /proc/47344/task/48692/cwd (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48692 ms_local ceph rtd unknown /proc/47344/task/48692/root (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48692 ms_local ceph txt unknown /proc/47344/task/48692/exe (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48692 ms_local ceph NOFD /proc/47344/task/48692/fd (opendir: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48693 ms_dispat ceph cwd unknown /proc/47344/task/48693/cwd (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48693 ms_dispat ceph rtd unknown /proc/47344/task/48693/root (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48693 ms_dispat ceph txt unknown /proc/47344/task/48693/exe (readlink: Permission denied) 2024-04-21T09:10:05.411 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48693 ms_dispat ceph NOFD /proc/47344/task/48693/fd (opendir: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48694 ms_local ceph cwd unknown /proc/47344/task/48694/cwd (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48694 ms_local ceph rtd unknown /proc/47344/task/48694/root (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48694 ms_local ceph txt unknown /proc/47344/task/48694/exe (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48694 ms_local ceph NOFD /proc/47344/task/48694/fd (opendir: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48695 ms_dispat ceph cwd unknown /proc/47344/task/48695/cwd (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48695 ms_dispat ceph rtd unknown /proc/47344/task/48695/root (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48695 ms_dispat ceph txt unknown /proc/47344/task/48695/exe (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48695 ms_dispat ceph NOFD /proc/47344/task/48695/fd (opendir: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48696 ms_local ceph cwd unknown /proc/47344/task/48696/cwd (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48696 ms_local ceph rtd unknown /proc/47344/task/48696/root (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48696 ms_local ceph txt unknown /proc/47344/task/48696/exe (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48696 ms_local ceph NOFD /proc/47344/task/48696/fd (opendir: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48697 ms_dispat ceph cwd unknown /proc/47344/task/48697/cwd (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48697 ms_dispat ceph rtd unknown /proc/47344/task/48697/root (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48697 ms_dispat ceph txt unknown /proc/47344/task/48697/exe (readlink: Permission denied) 2024-04-21T09:10:05.412 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48697 ms_dispat ceph NOFD /proc/47344/task/48697/fd (opendir: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48698 ms_local ceph cwd unknown /proc/47344/task/48698/cwd (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48698 ms_local ceph rtd unknown /proc/47344/task/48698/root (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48698 ms_local ceph txt unknown /proc/47344/task/48698/exe (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48698 ms_local ceph NOFD /proc/47344/task/48698/fd (opendir: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48699 fn_anonym ceph cwd unknown /proc/47344/task/48699/cwd (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48699 fn_anonym ceph rtd unknown /proc/47344/task/48699/root (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48699 fn_anonym ceph txt unknown /proc/47344/task/48699/exe (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48699 fn_anonym ceph NOFD /proc/47344/task/48699/fd (opendir: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48700 finisher ceph cwd unknown /proc/47344/task/48700/cwd (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48700 finisher ceph rtd unknown /proc/47344/task/48700/root (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48700 finisher ceph txt unknown /proc/47344/task/48700/exe (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48700 finisher ceph NOFD /proc/47344/task/48700/fd (opendir: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48701 safe_time ceph cwd unknown /proc/47344/task/48701/cwd (readlink: Permission denied) 2024-04-21T09:10:05.413 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48701 safe_time ceph rtd unknown /proc/47344/task/48701/root (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48701 safe_time ceph txt unknown /proc/47344/task/48701/exe (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48701 safe_time ceph NOFD /proc/47344/task/48701/fd (opendir: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48702 safe_time ceph cwd unknown /proc/47344/task/48702/cwd (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48702 safe_time ceph rtd unknown /proc/47344/task/48702/root (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48702 safe_time ceph txt unknown /proc/47344/task/48702/exe (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48702 safe_time ceph NOFD /proc/47344/task/48702/fd (opendir: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48703 ceph-osd ceph cwd unknown /proc/47344/task/48703/cwd (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48703 ceph-osd ceph rtd unknown /proc/47344/task/48703/root (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48703 ceph-osd ceph txt unknown /proc/47344/task/48703/exe (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48703 ceph-osd ceph NOFD /proc/47344/task/48703/fd (opendir: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48704 osd_srv_a ceph cwd unknown /proc/47344/task/48704/cwd (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48704 osd_srv_a ceph rtd unknown /proc/47344/task/48704/root (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48704 osd_srv_a ceph txt unknown /proc/47344/task/48704/exe (readlink: Permission denied) 2024-04-21T09:10:05.414 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48704 osd_srv_a ceph NOFD /proc/47344/task/48704/fd (opendir: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48705 tp_osd_tp ceph cwd unknown /proc/47344/task/48705/cwd (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48705 tp_osd_tp ceph rtd unknown /proc/47344/task/48705/root (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48705 tp_osd_tp ceph txt unknown /proc/47344/task/48705/exe (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48705 tp_osd_tp ceph NOFD /proc/47344/task/48705/fd (opendir: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48706 tp_osd_tp ceph cwd unknown /proc/47344/task/48706/cwd (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48706 tp_osd_tp ceph rtd unknown /proc/47344/task/48706/root (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48706 tp_osd_tp ceph txt unknown /proc/47344/task/48706/exe (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48706 tp_osd_tp ceph NOFD /proc/47344/task/48706/fd (opendir: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48707 tp_osd_tp ceph cwd unknown /proc/47344/task/48707/cwd (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48707 tp_osd_tp ceph rtd unknown /proc/47344/task/48707/root (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48707 tp_osd_tp ceph txt unknown /proc/47344/task/48707/exe (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48707 tp_osd_tp ceph NOFD /proc/47344/task/48707/fd (opendir: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48708 tp_osd_tp ceph cwd unknown /proc/47344/task/48708/cwd (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48708 tp_osd_tp ceph rtd unknown /proc/47344/task/48708/root (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48708 tp_osd_tp ceph txt unknown /proc/47344/task/48708/exe (readlink: Permission denied) 2024-04-21T09:10:05.415 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48708 tp_osd_tp ceph NOFD /proc/47344/task/48708/fd (opendir: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48709 tp_osd_tp ceph cwd unknown /proc/47344/task/48709/cwd (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48709 tp_osd_tp ceph rtd unknown /proc/47344/task/48709/root (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48709 tp_osd_tp ceph txt unknown /proc/47344/task/48709/exe (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48709 tp_osd_tp ceph NOFD /proc/47344/task/48709/fd (opendir: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48710 tp_osd_tp ceph cwd unknown /proc/47344/task/48710/cwd (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48710 tp_osd_tp ceph rtd unknown /proc/47344/task/48710/root (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48710 tp_osd_tp ceph txt unknown /proc/47344/task/48710/exe (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48710 tp_osd_tp ceph NOFD /proc/47344/task/48710/fd (opendir: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48711 tp_osd_tp ceph cwd unknown /proc/47344/task/48711/cwd (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48711 tp_osd_tp ceph rtd unknown /proc/47344/task/48711/root (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48711 tp_osd_tp ceph txt unknown /proc/47344/task/48711/exe (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48711 tp_osd_tp ceph NOFD /proc/47344/task/48711/fd (opendir: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48712 tp_osd_tp ceph cwd unknown /proc/47344/task/48712/cwd (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48712 tp_osd_tp ceph rtd unknown /proc/47344/task/48712/root (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48712 tp_osd_tp ceph txt unknown /proc/47344/task/48712/exe (readlink: Permission denied) 2024-04-21T09:10:05.416 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48712 tp_osd_tp ceph NOFD /proc/47344/task/48712/fd (opendir: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48713 tp_osd_tp ceph cwd unknown /proc/47344/task/48713/cwd (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48713 tp_osd_tp ceph rtd unknown /proc/47344/task/48713/root (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48713 tp_osd_tp ceph txt unknown /proc/47344/task/48713/exe (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48713 tp_osd_tp ceph NOFD /proc/47344/task/48713/fd (opendir: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48714 tp_osd_tp ceph cwd unknown /proc/47344/task/48714/cwd (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48714 tp_osd_tp ceph rtd unknown /proc/47344/task/48714/root (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48714 tp_osd_tp ceph txt unknown /proc/47344/task/48714/exe (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48714 tp_osd_tp ceph NOFD /proc/47344/task/48714/fd (opendir: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48715 tp_osd_tp ceph cwd unknown /proc/47344/task/48715/cwd (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48715 tp_osd_tp ceph rtd unknown /proc/47344/task/48715/root (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48715 tp_osd_tp ceph txt unknown /proc/47344/task/48715/exe (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48715 tp_osd_tp ceph NOFD /proc/47344/task/48715/fd (opendir: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48717 tp_osd_tp ceph cwd unknown /proc/47344/task/48717/cwd (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48717 tp_osd_tp ceph rtd unknown /proc/47344/task/48717/root (readlink: Permission denied) 2024-04-21T09:10:05.417 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48717 tp_osd_tp ceph txt unknown /proc/47344/task/48717/exe (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48717 tp_osd_tp ceph NOFD /proc/47344/task/48717/fd (opendir: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48718 tp_osd_tp ceph cwd unknown /proc/47344/task/48718/cwd (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48718 tp_osd_tp ceph rtd unknown /proc/47344/task/48718/root (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48718 tp_osd_tp ceph txt unknown /proc/47344/task/48718/exe (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48718 tp_osd_tp ceph NOFD /proc/47344/task/48718/fd (opendir: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48719 tp_osd_tp ceph cwd unknown /proc/47344/task/48719/cwd (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48719 tp_osd_tp ceph rtd unknown /proc/47344/task/48719/root (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48719 tp_osd_tp ceph txt unknown /proc/47344/task/48719/exe (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48719 tp_osd_tp ceph NOFD /proc/47344/task/48719/fd (opendir: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48720 tp_osd_tp ceph cwd unknown /proc/47344/task/48720/cwd (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48720 tp_osd_tp ceph rtd unknown /proc/47344/task/48720/root (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48720 tp_osd_tp ceph txt unknown /proc/47344/task/48720/exe (readlink: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48720 tp_osd_tp ceph NOFD /proc/47344/task/48720/fd (opendir: Permission denied) 2024-04-21T09:10:05.418 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48721 tp_osd_tp ceph cwd unknown /proc/47344/task/48721/cwd (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48721 tp_osd_tp ceph rtd unknown /proc/47344/task/48721/root (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48721 tp_osd_tp ceph txt unknown /proc/47344/task/48721/exe (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48721 tp_osd_tp ceph NOFD /proc/47344/task/48721/fd (opendir: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48722 osd_srv_h ceph cwd unknown /proc/47344/task/48722/cwd (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48722 osd_srv_h ceph rtd unknown /proc/47344/task/48722/root (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48722 osd_srv_h ceph txt unknown /proc/47344/task/48722/exe (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 47344 48722 osd_srv_h ceph NOFD /proc/47344/task/48722/fd (opendir: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 48746 root cwd unknown /proc/48746/cwd (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 48746 root rtd unknown /proc/48746/root (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 48746 root txt unknown /proc/48746/exe (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 48746 root NOFD /proc/48746/fd (opendir: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 48765 root cwd unknown /proc/48765/cwd (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 48765 root rtd unknown /proc/48765/root (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 48765 root txt unknown /proc/48765/exe (readlink: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 48765 root NOFD /proc/48765/fd (opendir: Permission denied) 2024-04-21T09:10:05.419 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 53912 root cwd unknown /proc/53912/cwd (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 53912 root rtd unknown /proc/53912/root (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 53912 root txt unknown /proc/53912/exe (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 53912 root NOFD /proc/53912/fd (opendir: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 53914 root cwd unknown /proc/53914/cwd (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 53914 root rtd unknown /proc/53914/root (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 53914 root txt unknown /proc/53914/exe (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 53914 root NOFD /proc/53914/fd (opendir: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 ceph cwd unknown /proc/53916/cwd (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 ceph rtd unknown /proc/53916/root (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 ceph txt unknown /proc/53916/exe (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 ceph NOFD /proc/53916/fd (opendir: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53917 log ceph cwd unknown /proc/53916/task/53917/cwd (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53917 log ceph rtd unknown /proc/53916/task/53917/root (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53917 log ceph txt unknown /proc/53916/task/53917/exe (readlink: Permission denied) 2024-04-21T09:10:05.420 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53917 log ceph NOFD /proc/53916/task/53917/fd (opendir: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53919 msgr-work ceph cwd unknown /proc/53916/task/53919/cwd (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53919 msgr-work ceph rtd unknown /proc/53916/task/53919/root (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53919 msgr-work ceph txt unknown /proc/53916/task/53919/exe (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53919 msgr-work ceph NOFD /proc/53916/task/53919/fd (opendir: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53920 msgr-work ceph cwd unknown /proc/53916/task/53920/cwd (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53920 msgr-work ceph rtd unknown /proc/53916/task/53920/root (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53920 msgr-work ceph txt unknown /proc/53916/task/53920/exe (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53920 msgr-work ceph NOFD /proc/53916/task/53920/fd (opendir: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53921 msgr-work ceph cwd unknown /proc/53916/task/53921/cwd (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53921 msgr-work ceph rtd unknown /proc/53916/task/53921/root (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53921 msgr-work ceph txt unknown /proc/53916/task/53921/exe (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53921 msgr-work ceph NOFD /proc/53916/task/53921/fd (opendir: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53925 service ceph cwd unknown /proc/53916/task/53925/cwd (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53925 service ceph rtd unknown /proc/53916/task/53925/root (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53925 service ceph txt unknown /proc/53916/task/53925/exe (readlink: Permission denied) 2024-04-21T09:10:05.421 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53925 service ceph NOFD /proc/53916/task/53925/fd (opendir: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53926 admin_soc ceph cwd unknown /proc/53916/task/53926/cwd (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53926 admin_soc ceph rtd unknown /proc/53916/task/53926/root (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53926 admin_soc ceph txt unknown /proc/53916/task/53926/exe (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 53926 admin_soc ceph NOFD /proc/53916/task/53926/fd (opendir: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54070 signal_ha ceph cwd unknown /proc/53916/task/54070/cwd (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54070 signal_ha ceph rtd unknown /proc/53916/task/54070/root (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54070 signal_ha ceph txt unknown /proc/53916/task/54070/exe (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54070 signal_ha ceph NOFD /proc/53916/task/54070/fd (opendir: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54071 io_contex ceph cwd unknown /proc/53916/task/54071/cwd (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54071 io_contex ceph rtd unknown /proc/53916/task/54071/root (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54071 io_contex ceph txt unknown /proc/53916/task/54071/exe (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54071 io_contex ceph NOFD /proc/53916/task/54071/fd (opendir: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54072 io_contex ceph cwd unknown /proc/53916/task/54072/cwd (readlink: Permission denied) 2024-04-21T09:10:05.422 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54072 io_contex ceph rtd unknown /proc/53916/task/54072/root (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54072 io_contex ceph txt unknown /proc/53916/task/54072/exe (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54072 io_contex ceph NOFD /proc/53916/task/54072/fd (opendir: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54188 OpHistory ceph cwd unknown /proc/53916/task/54188/cwd (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54188 OpHistory ceph rtd unknown /proc/53916/task/54188/root (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54188 OpHistory ceph txt unknown /proc/53916/task/54188/exe (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54188 OpHistory ceph NOFD /proc/53916/task/54188/fd (opendir: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54189 ceph_time ceph cwd unknown /proc/53916/task/54189/cwd (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54189 ceph_time ceph rtd unknown /proc/53916/task/54189/root (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54189 ceph_time ceph txt unknown /proc/53916/task/54189/exe (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54189 ceph_time ceph NOFD /proc/53916/task/54189/fd (opendir: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54345 ceph-osd ceph cwd unknown /proc/53916/task/54345/cwd (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54345 ceph-osd ceph rtd unknown /proc/53916/task/54345/root (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54345 ceph-osd ceph txt unknown /proc/53916/task/54345/exe (readlink: Permission denied) 2024-04-21T09:10:05.423 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54345 ceph-osd ceph NOFD /proc/53916/task/54345/fd (opendir: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54346 safe_time ceph cwd unknown /proc/53916/task/54346/cwd (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54346 safe_time ceph rtd unknown /proc/53916/task/54346/root (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54346 safe_time ceph txt unknown /proc/53916/task/54346/exe (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54346 safe_time ceph NOFD /proc/53916/task/54346/fd (opendir: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54347 safe_time ceph cwd unknown /proc/53916/task/54347/cwd (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54347 safe_time ceph rtd unknown /proc/53916/task/54347/root (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54347 safe_time ceph txt unknown /proc/53916/task/54347/exe (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54347 safe_time ceph NOFD /proc/53916/task/54347/fd (opendir: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54348 safe_time ceph cwd unknown /proc/53916/task/54348/cwd (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54348 safe_time ceph rtd unknown /proc/53916/task/54348/root (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54348 safe_time ceph txt unknown /proc/53916/task/54348/exe (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54348 safe_time ceph NOFD /proc/53916/task/54348/fd (opendir: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54349 safe_time ceph cwd unknown /proc/53916/task/54349/cwd (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54349 safe_time ceph rtd unknown /proc/53916/task/54349/root (readlink: Permission denied) 2024-04-21T09:10:05.424 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54349 safe_time ceph txt unknown /proc/53916/task/54349/exe (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54349 safe_time ceph NOFD /proc/53916/task/54349/fd (opendir: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54350 fn_anonym ceph cwd unknown /proc/53916/task/54350/cwd (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54350 fn_anonym ceph rtd unknown /proc/53916/task/54350/root (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54350 fn_anonym ceph txt unknown /proc/53916/task/54350/exe (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54350 fn_anonym ceph NOFD /proc/53916/task/54350/fd (opendir: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54353 rocksdb:l ceph cwd unknown /proc/53916/task/54353/cwd (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54353 rocksdb:l ceph rtd unknown /proc/53916/task/54353/root (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54353 rocksdb:l ceph txt unknown /proc/53916/task/54353/exe (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54353 rocksdb:l ceph NOFD /proc/53916/task/54353/fd (opendir: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54354 rocksdb:l ceph cwd unknown /proc/53916/task/54354/cwd (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54354 rocksdb:l ceph rtd unknown /proc/53916/task/54354/root (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54354 rocksdb:l ceph txt unknown /proc/53916/task/54354/exe (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54354 rocksdb:l ceph NOFD /proc/53916/task/54354/fd (opendir: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54355 rocksdb:l ceph cwd unknown /proc/53916/task/54355/cwd (readlink: Permission denied) 2024-04-21T09:10:05.425 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54355 rocksdb:l ceph rtd unknown /proc/53916/task/54355/root (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54355 rocksdb:l ceph txt unknown /proc/53916/task/54355/exe (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54355 rocksdb:l ceph NOFD /proc/53916/task/54355/fd (opendir: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54356 rocksdb:h ceph cwd unknown /proc/53916/task/54356/cwd (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54356 rocksdb:h ceph rtd unknown /proc/53916/task/54356/root (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54356 rocksdb:h ceph txt unknown /proc/53916/task/54356/exe (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54356 rocksdb:h ceph NOFD /proc/53916/task/54356/fd (opendir: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54833 bstore_ai ceph cwd unknown /proc/53916/task/54833/cwd (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54833 bstore_ai ceph rtd unknown /proc/53916/task/54833/root (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54833 bstore_ai ceph txt unknown /proc/53916/task/54833/exe (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 54833 bstore_ai ceph NOFD /proc/53916/task/54833/fd (opendir: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55019 bstore_ai ceph cwd unknown /proc/53916/task/55019/cwd (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55019 bstore_ai ceph rtd unknown /proc/53916/task/55019/root (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55019 bstore_ai ceph txt unknown /proc/53916/task/55019/exe (readlink: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55019 bstore_ai ceph NOFD /proc/53916/task/55019/fd (opendir: Permission denied) 2024-04-21T09:10:05.426 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55204 ceph-osd ceph cwd unknown /proc/53916/task/55204/cwd (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55204 ceph-osd ceph rtd unknown /proc/53916/task/55204/root (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55204 ceph-osd ceph txt unknown /proc/53916/task/55204/exe (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55204 ceph-osd ceph NOFD /proc/53916/task/55204/fd (opendir: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55205 cfin ceph cwd unknown /proc/53916/task/55205/cwd (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55205 cfin ceph rtd unknown /proc/53916/task/55205/root (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55205 cfin ceph txt unknown /proc/53916/task/55205/exe (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55205 cfin ceph NOFD /proc/53916/task/55205/fd (opendir: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55206 bstore_kv ceph cwd unknown /proc/53916/task/55206/cwd (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55206 bstore_kv ceph rtd unknown /proc/53916/task/55206/root (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55206 bstore_kv ceph txt unknown /proc/53916/task/55206/exe (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55206 bstore_kv ceph NOFD /proc/53916/task/55206/fd (opendir: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55208 bstore_kv ceph cwd unknown /proc/53916/task/55208/cwd (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55208 bstore_kv ceph rtd unknown /proc/53916/task/55208/root (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55208 bstore_kv ceph txt unknown /proc/53916/task/55208/exe (readlink: Permission denied) 2024-04-21T09:10:05.427 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55208 bstore_kv ceph NOFD /proc/53916/task/55208/fd (opendir: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55209 bstore_me ceph cwd unknown /proc/53916/task/55209/cwd (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55209 bstore_me ceph rtd unknown /proc/53916/task/55209/root (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55209 bstore_me ceph txt unknown /proc/53916/task/55209/exe (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55209 bstore_me ceph NOFD /proc/53916/task/55209/fd (opendir: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55230 ms_dispat ceph cwd unknown /proc/53916/task/55230/cwd (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55230 ms_dispat ceph rtd unknown /proc/53916/task/55230/root (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55230 ms_dispat ceph txt unknown /proc/53916/task/55230/exe (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55230 ms_dispat ceph NOFD /proc/53916/task/55230/fd (opendir: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55231 ms_local ceph cwd unknown /proc/53916/task/55231/cwd (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55231 ms_local ceph rtd unknown /proc/53916/task/55231/root (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55231 ms_local ceph txt unknown /proc/53916/task/55231/exe (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55231 ms_local ceph NOFD /proc/53916/task/55231/fd (opendir: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55232 safe_time ceph cwd unknown /proc/53916/task/55232/cwd (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55232 safe_time ceph rtd unknown /proc/53916/task/55232/root (readlink: Permission denied) 2024-04-21T09:10:05.428 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55232 safe_time ceph txt unknown /proc/53916/task/55232/exe (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55232 safe_time ceph NOFD /proc/53916/task/55232/fd (opendir: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55233 safe_time ceph cwd unknown /proc/53916/task/55233/cwd (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55233 safe_time ceph rtd unknown /proc/53916/task/55233/root (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55233 safe_time ceph txt unknown /proc/53916/task/55233/exe (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55233 safe_time ceph NOFD /proc/53916/task/55233/fd (opendir: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55235 ms_dispat ceph cwd unknown /proc/53916/task/55235/cwd (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55235 ms_dispat ceph rtd unknown /proc/53916/task/55235/root (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55235 ms_dispat ceph txt unknown /proc/53916/task/55235/exe (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55235 ms_dispat ceph NOFD /proc/53916/task/55235/fd (opendir: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55236 ms_local ceph cwd unknown /proc/53916/task/55236/cwd (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55236 ms_local ceph rtd unknown /proc/53916/task/55236/root (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55236 ms_local ceph txt unknown /proc/53916/task/55236/exe (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55236 ms_local ceph NOFD /proc/53916/task/55236/fd (opendir: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55237 ms_dispat ceph cwd unknown /proc/53916/task/55237/cwd (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55237 ms_dispat ceph rtd unknown /proc/53916/task/55237/root (readlink: Permission denied) 2024-04-21T09:10:05.429 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55237 ms_dispat ceph txt unknown /proc/53916/task/55237/exe (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55237 ms_dispat ceph NOFD /proc/53916/task/55237/fd (opendir: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55238 ms_local ceph cwd unknown /proc/53916/task/55238/cwd (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55238 ms_local ceph rtd unknown /proc/53916/task/55238/root (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55238 ms_local ceph txt unknown /proc/53916/task/55238/exe (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55238 ms_local ceph NOFD /proc/53916/task/55238/fd (opendir: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55239 ms_dispat ceph cwd unknown /proc/53916/task/55239/cwd (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55239 ms_dispat ceph rtd unknown /proc/53916/task/55239/root (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55239 ms_dispat ceph txt unknown /proc/53916/task/55239/exe (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55239 ms_dispat ceph NOFD /proc/53916/task/55239/fd (opendir: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55240 ms_local ceph cwd unknown /proc/53916/task/55240/cwd (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55240 ms_local ceph rtd unknown /proc/53916/task/55240/root (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55240 ms_local ceph txt unknown /proc/53916/task/55240/exe (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55240 ms_local ceph NOFD /proc/53916/task/55240/fd (opendir: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55241 ms_dispat ceph cwd unknown /proc/53916/task/55241/cwd (readlink: Permission denied) 2024-04-21T09:10:05.430 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55241 ms_dispat ceph rtd unknown /proc/53916/task/55241/root (readlink: Permission denied) 2024-04-21T09:10:05.431 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55241 ms_dispat ceph txt unknown /proc/53916/task/55241/exe (readlink: Permission denied) 2024-04-21T09:10:05.431 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55241 ms_dispat ceph NOFD /proc/53916/task/55241/fd (opendir: Permission denied) 2024-04-21T09:10:05.431 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55242 ms_local ceph cwd unknown /proc/53916/task/55242/cwd (readlink: Permission denied) 2024-04-21T09:10:05.431 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55242 ms_local ceph rtd unknown /proc/53916/task/55242/root (readlink: Permission denied) 2024-04-21T09:10:05.431 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55242 ms_local ceph txt unknown /proc/53916/task/55242/exe (readlink: Permission denied) 2024-04-21T09:10:05.431 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55242 ms_local ceph NOFD /proc/53916/task/55242/fd (opendir: Permission denied) 2024-04-21T09:10:05.431 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55243 ms_dispat ceph cwd unknown /proc/53916/task/55243/cwd (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55243 ms_dispat ceph rtd unknown /proc/53916/task/55243/root (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55243 ms_dispat ceph txt unknown /proc/53916/task/55243/exe (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55243 ms_dispat ceph NOFD /proc/53916/task/55243/fd (opendir: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55244 ms_local ceph cwd unknown /proc/53916/task/55244/cwd (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55244 ms_local ceph rtd unknown /proc/53916/task/55244/root (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55244 ms_local ceph txt unknown /proc/53916/task/55244/exe (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55244 ms_local ceph NOFD /proc/53916/task/55244/fd (opendir: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55246 ms_dispat ceph cwd unknown /proc/53916/task/55246/cwd (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55246 ms_dispat ceph rtd unknown /proc/53916/task/55246/root (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55246 ms_dispat ceph txt unknown /proc/53916/task/55246/exe (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55246 ms_dispat ceph NOFD /proc/53916/task/55246/fd (opendir: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55247 ms_local ceph cwd unknown /proc/53916/task/55247/cwd (readlink: Permission denied) 2024-04-21T09:10:05.433 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55247 ms_local ceph rtd unknown /proc/53916/task/55247/root (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55247 ms_local ceph txt unknown /proc/53916/task/55247/exe (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55247 ms_local ceph NOFD /proc/53916/task/55247/fd (opendir: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55248 fn_anonym ceph cwd unknown /proc/53916/task/55248/cwd (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55248 fn_anonym ceph rtd unknown /proc/53916/task/55248/root (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55248 fn_anonym ceph txt unknown /proc/53916/task/55248/exe (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55248 fn_anonym ceph NOFD /proc/53916/task/55248/fd (opendir: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55250 finisher ceph cwd unknown /proc/53916/task/55250/cwd (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55250 finisher ceph rtd unknown /proc/53916/task/55250/root (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55250 finisher ceph txt unknown /proc/53916/task/55250/exe (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55250 finisher ceph NOFD /proc/53916/task/55250/fd (opendir: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55251 safe_time ceph cwd unknown /proc/53916/task/55251/cwd (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55251 safe_time ceph rtd unknown /proc/53916/task/55251/root (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55251 safe_time ceph txt unknown /proc/53916/task/55251/exe (readlink: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55251 safe_time ceph NOFD /proc/53916/task/55251/fd (opendir: Permission denied) 2024-04-21T09:10:05.434 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55252 safe_time ceph cwd unknown /proc/53916/task/55252/cwd (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55252 safe_time ceph rtd unknown /proc/53916/task/55252/root (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55252 safe_time ceph txt unknown /proc/53916/task/55252/exe (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55252 safe_time ceph NOFD /proc/53916/task/55252/fd (opendir: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55253 ceph-osd ceph cwd unknown /proc/53916/task/55253/cwd (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55253 ceph-osd ceph rtd unknown /proc/53916/task/55253/root (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55253 ceph-osd ceph txt unknown /proc/53916/task/55253/exe (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55253 ceph-osd ceph NOFD /proc/53916/task/55253/fd (opendir: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55255 osd_srv_a ceph cwd unknown /proc/53916/task/55255/cwd (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55255 osd_srv_a ceph rtd unknown /proc/53916/task/55255/root (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55255 osd_srv_a ceph txt unknown /proc/53916/task/55255/exe (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55255 osd_srv_a ceph NOFD /proc/53916/task/55255/fd (opendir: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55256 tp_osd_tp ceph cwd unknown /proc/53916/task/55256/cwd (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55256 tp_osd_tp ceph rtd unknown /proc/53916/task/55256/root (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55256 tp_osd_tp ceph txt unknown /proc/53916/task/55256/exe (readlink: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55256 tp_osd_tp ceph NOFD /proc/53916/task/55256/fd (opendir: Permission denied) 2024-04-21T09:10:05.435 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55257 tp_osd_tp ceph cwd unknown /proc/53916/task/55257/cwd (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55257 tp_osd_tp ceph rtd unknown /proc/53916/task/55257/root (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55257 tp_osd_tp ceph txt unknown /proc/53916/task/55257/exe (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55257 tp_osd_tp ceph NOFD /proc/53916/task/55257/fd (opendir: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55258 tp_osd_tp ceph cwd unknown /proc/53916/task/55258/cwd (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55258 tp_osd_tp ceph rtd unknown /proc/53916/task/55258/root (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55258 tp_osd_tp ceph txt unknown /proc/53916/task/55258/exe (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55258 tp_osd_tp ceph NOFD /proc/53916/task/55258/fd (opendir: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55259 tp_osd_tp ceph cwd unknown /proc/53916/task/55259/cwd (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55259 tp_osd_tp ceph rtd unknown /proc/53916/task/55259/root (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55259 tp_osd_tp ceph txt unknown /proc/53916/task/55259/exe (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55259 tp_osd_tp ceph NOFD /proc/53916/task/55259/fd (opendir: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55260 tp_osd_tp ceph cwd unknown /proc/53916/task/55260/cwd (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55260 tp_osd_tp ceph rtd unknown /proc/53916/task/55260/root (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55260 tp_osd_tp ceph txt unknown /proc/53916/task/55260/exe (readlink: Permission denied) 2024-04-21T09:10:05.436 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55260 tp_osd_tp ceph NOFD /proc/53916/task/55260/fd (opendir: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55261 tp_osd_tp ceph cwd unknown /proc/53916/task/55261/cwd (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55261 tp_osd_tp ceph rtd unknown /proc/53916/task/55261/root (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55261 tp_osd_tp ceph txt unknown /proc/53916/task/55261/exe (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55261 tp_osd_tp ceph NOFD /proc/53916/task/55261/fd (opendir: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55262 tp_osd_tp ceph cwd unknown /proc/53916/task/55262/cwd (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55262 tp_osd_tp ceph rtd unknown /proc/53916/task/55262/root (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55262 tp_osd_tp ceph txt unknown /proc/53916/task/55262/exe (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55262 tp_osd_tp ceph NOFD /proc/53916/task/55262/fd (opendir: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55263 tp_osd_tp ceph cwd unknown /proc/53916/task/55263/cwd (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55263 tp_osd_tp ceph rtd unknown /proc/53916/task/55263/root (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55263 tp_osd_tp ceph txt unknown /proc/53916/task/55263/exe (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55263 tp_osd_tp ceph NOFD /proc/53916/task/55263/fd (opendir: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55264 tp_osd_tp ceph cwd unknown /proc/53916/task/55264/cwd (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55264 tp_osd_tp ceph rtd unknown /proc/53916/task/55264/root (readlink: Permission denied) 2024-04-21T09:10:05.437 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55264 tp_osd_tp ceph txt unknown /proc/53916/task/55264/exe (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55264 tp_osd_tp ceph NOFD /proc/53916/task/55264/fd (opendir: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55265 tp_osd_tp ceph cwd unknown /proc/53916/task/55265/cwd (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55265 tp_osd_tp ceph rtd unknown /proc/53916/task/55265/root (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55265 tp_osd_tp ceph txt unknown /proc/53916/task/55265/exe (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55265 tp_osd_tp ceph NOFD /proc/53916/task/55265/fd (opendir: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55266 tp_osd_tp ceph cwd unknown /proc/53916/task/55266/cwd (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55266 tp_osd_tp ceph rtd unknown /proc/53916/task/55266/root (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55266 tp_osd_tp ceph txt unknown /proc/53916/task/55266/exe (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55266 tp_osd_tp ceph NOFD /proc/53916/task/55266/fd (opendir: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55267 tp_osd_tp ceph cwd unknown /proc/53916/task/55267/cwd (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55267 tp_osd_tp ceph rtd unknown /proc/53916/task/55267/root (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55267 tp_osd_tp ceph txt unknown /proc/53916/task/55267/exe (readlink: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55267 tp_osd_tp ceph NOFD /proc/53916/task/55267/fd (opendir: Permission denied) 2024-04-21T09:10:05.438 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55268 tp_osd_tp ceph cwd unknown /proc/53916/task/55268/cwd (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55268 tp_osd_tp ceph rtd unknown /proc/53916/task/55268/root (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55268 tp_osd_tp ceph txt unknown /proc/53916/task/55268/exe (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55268 tp_osd_tp ceph NOFD /proc/53916/task/55268/fd (opendir: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55269 tp_osd_tp ceph cwd unknown /proc/53916/task/55269/cwd (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55269 tp_osd_tp ceph rtd unknown /proc/53916/task/55269/root (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55269 tp_osd_tp ceph txt unknown /proc/53916/task/55269/exe (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55269 tp_osd_tp ceph NOFD /proc/53916/task/55269/fd (opendir: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55270 tp_osd_tp ceph cwd unknown /proc/53916/task/55270/cwd (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55270 tp_osd_tp ceph rtd unknown /proc/53916/task/55270/root (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55270 tp_osd_tp ceph txt unknown /proc/53916/task/55270/exe (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55270 tp_osd_tp ceph NOFD /proc/53916/task/55270/fd (opendir: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55271 tp_osd_tp ceph cwd unknown /proc/53916/task/55271/cwd (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55271 tp_osd_tp ceph rtd unknown /proc/53916/task/55271/root (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55271 tp_osd_tp ceph txt unknown /proc/53916/task/55271/exe (readlink: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55271 tp_osd_tp ceph NOFD /proc/53916/task/55271/fd (opendir: Permission denied) 2024-04-21T09:10:05.439 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55272 osd_srv_h ceph cwd unknown /proc/53916/task/55272/cwd (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55272 osd_srv_h ceph rtd unknown /proc/53916/task/55272/root (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55272 osd_srv_h ceph txt unknown /proc/53916/task/55272/exe (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 53916 55272 osd_srv_h ceph NOFD /proc/53916/task/55272/fd (opendir: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 55347 root cwd unknown /proc/55347/cwd (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 55347 root rtd unknown /proc/55347/root (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 55347 root txt unknown /proc/55347/exe (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 55347 root NOFD /proc/55347/fd (opendir: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 55373 root cwd unknown /proc/55373/cwd (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 55373 root rtd unknown /proc/55373/root (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 55373 root txt unknown /proc/55373/exe (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 55373 root NOFD /proc/55373/fd (opendir: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 57908 root cwd unknown /proc/57908/cwd (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 57908 root rtd unknown /proc/57908/root (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 57908 root txt unknown /proc/57908/exe (readlink: Permission denied) 2024-04-21T09:10:05.440 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 57908 root NOFD /proc/57908/fd (opendir: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 57909 root cwd unknown /proc/57909/cwd (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 57909 root rtd unknown /proc/57909/root (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 57909 root txt unknown /proc/57909/exe (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/7 57909 root NOFD /proc/57909/fd (opendir: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 60811 root cwd unknown /proc/60811/cwd (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 60811 root rtd unknown /proc/60811/root (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 60811 root txt unknown /proc/60811/exe (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 60811 root NOFD /proc/60811/fd (opendir: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 60813 root cwd unknown /proc/60813/cwd (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 60813 root rtd unknown /proc/60813/root (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 60813 root txt unknown /proc/60813/exe (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 60813 root NOFD /proc/60813/fd (opendir: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 ceph cwd unknown /proc/60816/cwd (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 ceph rtd unknown /proc/60816/root (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 ceph txt unknown /proc/60816/exe (readlink: Permission denied) 2024-04-21T09:10:05.441 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 ceph NOFD /proc/60816/fd (opendir: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60817 log ceph cwd unknown /proc/60816/task/60817/cwd (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60817 log ceph rtd unknown /proc/60816/task/60817/root (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60817 log ceph txt unknown /proc/60816/task/60817/exe (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60817 log ceph NOFD /proc/60816/task/60817/fd (opendir: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60819 msgr-work ceph cwd unknown /proc/60816/task/60819/cwd (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60819 msgr-work ceph rtd unknown /proc/60816/task/60819/root (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60819 msgr-work ceph txt unknown /proc/60816/task/60819/exe (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60819 msgr-work ceph NOFD /proc/60816/task/60819/fd (opendir: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60820 msgr-work ceph cwd unknown /proc/60816/task/60820/cwd (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60820 msgr-work ceph rtd unknown /proc/60816/task/60820/root (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60820 msgr-work ceph txt unknown /proc/60816/task/60820/exe (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60820 msgr-work ceph NOFD /proc/60816/task/60820/fd (opendir: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60821 msgr-work ceph cwd unknown /proc/60816/task/60821/cwd (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60821 msgr-work ceph rtd unknown /proc/60816/task/60821/root (readlink: Permission denied) 2024-04-21T09:10:05.442 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60821 msgr-work ceph txt unknown /proc/60816/task/60821/exe (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60821 msgr-work ceph NOFD /proc/60816/task/60821/fd (opendir: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60825 service ceph cwd unknown /proc/60816/task/60825/cwd (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60825 service ceph rtd unknown /proc/60816/task/60825/root (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60825 service ceph txt unknown /proc/60816/task/60825/exe (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60825 service ceph NOFD /proc/60816/task/60825/fd (opendir: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60826 admin_soc ceph cwd unknown /proc/60816/task/60826/cwd (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60826 admin_soc ceph rtd unknown /proc/60816/task/60826/root (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60826 admin_soc ceph txt unknown /proc/60816/task/60826/exe (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60826 admin_soc ceph NOFD /proc/60816/task/60826/fd (opendir: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60968 signal_ha ceph cwd unknown /proc/60816/task/60968/cwd (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60968 signal_ha ceph rtd unknown /proc/60816/task/60968/root (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60968 signal_ha ceph txt unknown /proc/60816/task/60968/exe (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60968 signal_ha ceph NOFD /proc/60816/task/60968/fd (opendir: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60969 io_contex ceph cwd unknown /proc/60816/task/60969/cwd (readlink: Permission denied) 2024-04-21T09:10:05.443 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60969 io_contex ceph rtd unknown /proc/60816/task/60969/root (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60969 io_contex ceph txt unknown /proc/60816/task/60969/exe (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60969 io_contex ceph NOFD /proc/60816/task/60969/fd (opendir: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60970 io_contex ceph cwd unknown /proc/60816/task/60970/cwd (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60970 io_contex ceph rtd unknown /proc/60816/task/60970/root (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60970 io_contex ceph txt unknown /proc/60816/task/60970/exe (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 60970 io_contex ceph NOFD /proc/60816/task/60970/fd (opendir: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61084 OpHistory ceph cwd unknown /proc/60816/task/61084/cwd (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61084 OpHistory ceph rtd unknown /proc/60816/task/61084/root (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61084 OpHistory ceph txt unknown /proc/60816/task/61084/exe (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61084 OpHistory ceph NOFD /proc/60816/task/61084/fd (opendir: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61085 ceph_time ceph cwd unknown /proc/60816/task/61085/cwd (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61085 ceph_time ceph rtd unknown /proc/60816/task/61085/root (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61085 ceph_time ceph txt unknown /proc/60816/task/61085/exe (readlink: Permission denied) 2024-04-21T09:10:05.444 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61085 ceph_time ceph NOFD /proc/60816/task/61085/fd (opendir: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61245 ceph-osd ceph cwd unknown /proc/60816/task/61245/cwd (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61245 ceph-osd ceph rtd unknown /proc/60816/task/61245/root (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61245 ceph-osd ceph txt unknown /proc/60816/task/61245/exe (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61245 ceph-osd ceph NOFD /proc/60816/task/61245/fd (opendir: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61247 safe_time ceph cwd unknown /proc/60816/task/61247/cwd (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61247 safe_time ceph rtd unknown /proc/60816/task/61247/root (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61247 safe_time ceph txt unknown /proc/60816/task/61247/exe (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61247 safe_time ceph NOFD /proc/60816/task/61247/fd (opendir: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61248 safe_time ceph cwd unknown /proc/60816/task/61248/cwd (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61248 safe_time ceph rtd unknown /proc/60816/task/61248/root (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61248 safe_time ceph txt unknown /proc/60816/task/61248/exe (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61248 safe_time ceph NOFD /proc/60816/task/61248/fd (opendir: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61249 safe_time ceph cwd unknown /proc/60816/task/61249/cwd (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61249 safe_time ceph rtd unknown /proc/60816/task/61249/root (readlink: Permission denied) 2024-04-21T09:10:05.445 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61249 safe_time ceph txt unknown /proc/60816/task/61249/exe (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61249 safe_time ceph NOFD /proc/60816/task/61249/fd (opendir: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61250 safe_time ceph cwd unknown /proc/60816/task/61250/cwd (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61250 safe_time ceph rtd unknown /proc/60816/task/61250/root (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61250 safe_time ceph txt unknown /proc/60816/task/61250/exe (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61250 safe_time ceph NOFD /proc/60816/task/61250/fd (opendir: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61251 fn_anonym ceph cwd unknown /proc/60816/task/61251/cwd (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61251 fn_anonym ceph rtd unknown /proc/60816/task/61251/root (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61251 fn_anonym ceph txt unknown /proc/60816/task/61251/exe (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61251 fn_anonym ceph NOFD /proc/60816/task/61251/fd (opendir: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61254 rocksdb:l ceph cwd unknown /proc/60816/task/61254/cwd (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61254 rocksdb:l ceph rtd unknown /proc/60816/task/61254/root (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61254 rocksdb:l ceph txt unknown /proc/60816/task/61254/exe (readlink: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61254 rocksdb:l ceph NOFD /proc/60816/task/61254/fd (opendir: Permission denied) 2024-04-21T09:10:05.446 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61255 rocksdb:l ceph cwd unknown /proc/60816/task/61255/cwd (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61255 rocksdb:l ceph rtd unknown /proc/60816/task/61255/root (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61255 rocksdb:l ceph txt unknown /proc/60816/task/61255/exe (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61255 rocksdb:l ceph NOFD /proc/60816/task/61255/fd (opendir: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61256 rocksdb:l ceph cwd unknown /proc/60816/task/61256/cwd (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61256 rocksdb:l ceph rtd unknown /proc/60816/task/61256/root (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61256 rocksdb:l ceph txt unknown /proc/60816/task/61256/exe (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61256 rocksdb:l ceph NOFD /proc/60816/task/61256/fd (opendir: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61257 rocksdb:h ceph cwd unknown /proc/60816/task/61257/cwd (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61257 rocksdb:h ceph rtd unknown /proc/60816/task/61257/root (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61257 rocksdb:h ceph txt unknown /proc/60816/task/61257/exe (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61257 rocksdb:h ceph NOFD /proc/60816/task/61257/fd (opendir: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61669 bstore_ai ceph cwd unknown /proc/60816/task/61669/cwd (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61669 bstore_ai ceph rtd unknown /proc/60816/task/61669/root (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61669 bstore_ai ceph txt unknown /proc/60816/task/61669/exe (readlink: Permission denied) 2024-04-21T09:10:05.447 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61669 bstore_ai ceph NOFD /proc/60816/task/61669/fd (opendir: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61888 bstore_ai ceph cwd unknown /proc/60816/task/61888/cwd (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61888 bstore_ai ceph rtd unknown /proc/60816/task/61888/root (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61888 bstore_ai ceph txt unknown /proc/60816/task/61888/exe (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 61888 bstore_ai ceph NOFD /proc/60816/task/61888/fd (opendir: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62072 ceph-osd ceph cwd unknown /proc/60816/task/62072/cwd (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62072 ceph-osd ceph rtd unknown /proc/60816/task/62072/root (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62072 ceph-osd ceph txt unknown /proc/60816/task/62072/exe (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62072 ceph-osd ceph NOFD /proc/60816/task/62072/fd (opendir: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62073 cfin ceph cwd unknown /proc/60816/task/62073/cwd (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62073 cfin ceph rtd unknown /proc/60816/task/62073/root (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62073 cfin ceph txt unknown /proc/60816/task/62073/exe (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62073 cfin ceph NOFD /proc/60816/task/62073/fd (opendir: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62074 bstore_kv ceph cwd unknown /proc/60816/task/62074/cwd (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62074 bstore_kv ceph rtd unknown /proc/60816/task/62074/root (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62074 bstore_kv ceph txt unknown /proc/60816/task/62074/exe (readlink: Permission denied) 2024-04-21T09:10:05.448 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62074 bstore_kv ceph NOFD /proc/60816/task/62074/fd (opendir: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62075 bstore_kv ceph cwd unknown /proc/60816/task/62075/cwd (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62075 bstore_kv ceph rtd unknown /proc/60816/task/62075/root (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62075 bstore_kv ceph txt unknown /proc/60816/task/62075/exe (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62075 bstore_kv ceph NOFD /proc/60816/task/62075/fd (opendir: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62076 bstore_me ceph cwd unknown /proc/60816/task/62076/cwd (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62076 bstore_me ceph rtd unknown /proc/60816/task/62076/root (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62076 bstore_me ceph txt unknown /proc/60816/task/62076/exe (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62076 bstore_me ceph NOFD /proc/60816/task/62076/fd (opendir: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62077 ms_dispat ceph cwd unknown /proc/60816/task/62077/cwd (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62077 ms_dispat ceph rtd unknown /proc/60816/task/62077/root (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62077 ms_dispat ceph txt unknown /proc/60816/task/62077/exe (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62077 ms_dispat ceph NOFD /proc/60816/task/62077/fd (opendir: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62078 ms_local ceph cwd unknown /proc/60816/task/62078/cwd (readlink: Permission denied) 2024-04-21T09:10:05.449 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62078 ms_local ceph rtd unknown /proc/60816/task/62078/root (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62078 ms_local ceph txt unknown /proc/60816/task/62078/exe (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62078 ms_local ceph NOFD /proc/60816/task/62078/fd (opendir: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62079 safe_time ceph cwd unknown /proc/60816/task/62079/cwd (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62079 safe_time ceph rtd unknown /proc/60816/task/62079/root (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62079 safe_time ceph txt unknown /proc/60816/task/62079/exe (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62079 safe_time ceph NOFD /proc/60816/task/62079/fd (opendir: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62080 safe_time ceph cwd unknown /proc/60816/task/62080/cwd (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62080 safe_time ceph rtd unknown /proc/60816/task/62080/root (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62080 safe_time ceph txt unknown /proc/60816/task/62080/exe (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62080 safe_time ceph NOFD /proc/60816/task/62080/fd (opendir: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62081 ms_dispat ceph cwd unknown /proc/60816/task/62081/cwd (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62081 ms_dispat ceph rtd unknown /proc/60816/task/62081/root (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62081 ms_dispat ceph txt unknown /proc/60816/task/62081/exe (readlink: Permission denied) 2024-04-21T09:10:05.450 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62081 ms_dispat ceph NOFD /proc/60816/task/62081/fd (opendir: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62082 ms_local ceph cwd unknown /proc/60816/task/62082/cwd (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62082 ms_local ceph rtd unknown /proc/60816/task/62082/root (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62082 ms_local ceph txt unknown /proc/60816/task/62082/exe (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62082 ms_local ceph NOFD /proc/60816/task/62082/fd (opendir: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62083 ms_dispat ceph cwd unknown /proc/60816/task/62083/cwd (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62083 ms_dispat ceph rtd unknown /proc/60816/task/62083/root (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62083 ms_dispat ceph txt unknown /proc/60816/task/62083/exe (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62083 ms_dispat ceph NOFD /proc/60816/task/62083/fd (opendir: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62084 ms_local ceph cwd unknown /proc/60816/task/62084/cwd (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62084 ms_local ceph rtd unknown /proc/60816/task/62084/root (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62084 ms_local ceph txt unknown /proc/60816/task/62084/exe (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62084 ms_local ceph NOFD /proc/60816/task/62084/fd (opendir: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62085 ms_dispat ceph cwd unknown /proc/60816/task/62085/cwd (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62085 ms_dispat ceph rtd unknown /proc/60816/task/62085/root (readlink: Permission denied) 2024-04-21T09:10:05.451 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62085 ms_dispat ceph txt unknown /proc/60816/task/62085/exe (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62085 ms_dispat ceph NOFD /proc/60816/task/62085/fd (opendir: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62086 ms_local ceph cwd unknown /proc/60816/task/62086/cwd (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62086 ms_local ceph rtd unknown /proc/60816/task/62086/root (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62086 ms_local ceph txt unknown /proc/60816/task/62086/exe (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62086 ms_local ceph NOFD /proc/60816/task/62086/fd (opendir: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62087 ms_dispat ceph cwd unknown /proc/60816/task/62087/cwd (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62087 ms_dispat ceph rtd unknown /proc/60816/task/62087/root (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62087 ms_dispat ceph txt unknown /proc/60816/task/62087/exe (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62087 ms_dispat ceph NOFD /proc/60816/task/62087/fd (opendir: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62088 ms_local ceph cwd unknown /proc/60816/task/62088/cwd (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62088 ms_local ceph rtd unknown /proc/60816/task/62088/root (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62088 ms_local ceph txt unknown /proc/60816/task/62088/exe (readlink: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62088 ms_local ceph NOFD /proc/60816/task/62088/fd (opendir: Permission denied) 2024-04-21T09:10:05.452 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62089 ms_dispat ceph cwd unknown /proc/60816/task/62089/cwd (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62089 ms_dispat ceph rtd unknown /proc/60816/task/62089/root (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62089 ms_dispat ceph txt unknown /proc/60816/task/62089/exe (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62089 ms_dispat ceph NOFD /proc/60816/task/62089/fd (opendir: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62090 ms_local ceph cwd unknown /proc/60816/task/62090/cwd (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62090 ms_local ceph rtd unknown /proc/60816/task/62090/root (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62090 ms_local ceph txt unknown /proc/60816/task/62090/exe (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62090 ms_local ceph NOFD /proc/60816/task/62090/fd (opendir: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62091 ms_dispat ceph cwd unknown /proc/60816/task/62091/cwd (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62091 ms_dispat ceph rtd unknown /proc/60816/task/62091/root (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62091 ms_dispat ceph txt unknown /proc/60816/task/62091/exe (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62091 ms_dispat ceph NOFD /proc/60816/task/62091/fd (opendir: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62092 ms_local ceph cwd unknown /proc/60816/task/62092/cwd (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62092 ms_local ceph rtd unknown /proc/60816/task/62092/root (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62092 ms_local ceph txt unknown /proc/60816/task/62092/exe (readlink: Permission denied) 2024-04-21T09:10:05.453 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62092 ms_local ceph NOFD /proc/60816/task/62092/fd (opendir: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62093 fn_anonym ceph cwd unknown /proc/60816/task/62093/cwd (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62093 fn_anonym ceph rtd unknown /proc/60816/task/62093/root (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62093 fn_anonym ceph txt unknown /proc/60816/task/62093/exe (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62093 fn_anonym ceph NOFD /proc/60816/task/62093/fd (opendir: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62094 finisher ceph cwd unknown /proc/60816/task/62094/cwd (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62094 finisher ceph rtd unknown /proc/60816/task/62094/root (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62094 finisher ceph txt unknown /proc/60816/task/62094/exe (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62094 finisher ceph NOFD /proc/60816/task/62094/fd (opendir: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62095 safe_time ceph cwd unknown /proc/60816/task/62095/cwd (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62095 safe_time ceph rtd unknown /proc/60816/task/62095/root (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62095 safe_time ceph txt unknown /proc/60816/task/62095/exe (readlink: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62095 safe_time ceph NOFD /proc/60816/task/62095/fd (opendir: Permission denied) 2024-04-21T09:10:05.454 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62096 safe_time ceph cwd unknown /proc/60816/task/62096/cwd (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62096 safe_time ceph rtd unknown /proc/60816/task/62096/root (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62096 safe_time ceph txt unknown /proc/60816/task/62096/exe (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62096 safe_time ceph NOFD /proc/60816/task/62096/fd (opendir: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62097 ceph-osd ceph cwd unknown /proc/60816/task/62097/cwd (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62097 ceph-osd ceph rtd unknown /proc/60816/task/62097/root (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62097 ceph-osd ceph txt unknown /proc/60816/task/62097/exe (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62097 ceph-osd ceph NOFD /proc/60816/task/62097/fd (opendir: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62098 osd_srv_a ceph cwd unknown /proc/60816/task/62098/cwd (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62098 osd_srv_a ceph rtd unknown /proc/60816/task/62098/root (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62098 osd_srv_a ceph txt unknown /proc/60816/task/62098/exe (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62098 osd_srv_a ceph NOFD /proc/60816/task/62098/fd (opendir: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62099 tp_osd_tp ceph cwd unknown /proc/60816/task/62099/cwd (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62099 tp_osd_tp ceph rtd unknown /proc/60816/task/62099/root (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62099 tp_osd_tp ceph txt unknown /proc/60816/task/62099/exe (readlink: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62099 tp_osd_tp ceph NOFD /proc/60816/task/62099/fd (opendir: Permission denied) 2024-04-21T09:10:05.455 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62100 tp_osd_tp ceph cwd unknown /proc/60816/task/62100/cwd (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62100 tp_osd_tp ceph rtd unknown /proc/60816/task/62100/root (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62100 tp_osd_tp ceph txt unknown /proc/60816/task/62100/exe (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62100 tp_osd_tp ceph NOFD /proc/60816/task/62100/fd (opendir: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62101 tp_osd_tp ceph cwd unknown /proc/60816/task/62101/cwd (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62101 tp_osd_tp ceph rtd unknown /proc/60816/task/62101/root (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62101 tp_osd_tp ceph txt unknown /proc/60816/task/62101/exe (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62101 tp_osd_tp ceph NOFD /proc/60816/task/62101/fd (opendir: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62102 tp_osd_tp ceph cwd unknown /proc/60816/task/62102/cwd (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62102 tp_osd_tp ceph rtd unknown /proc/60816/task/62102/root (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62102 tp_osd_tp ceph txt unknown /proc/60816/task/62102/exe (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62102 tp_osd_tp ceph NOFD /proc/60816/task/62102/fd (opendir: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62103 tp_osd_tp ceph cwd unknown /proc/60816/task/62103/cwd (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62103 tp_osd_tp ceph rtd unknown /proc/60816/task/62103/root (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62103 tp_osd_tp ceph txt unknown /proc/60816/task/62103/exe (readlink: Permission denied) 2024-04-21T09:10:05.456 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62103 tp_osd_tp ceph NOFD /proc/60816/task/62103/fd (opendir: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62104 tp_osd_tp ceph cwd unknown /proc/60816/task/62104/cwd (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62104 tp_osd_tp ceph rtd unknown /proc/60816/task/62104/root (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62104 tp_osd_tp ceph txt unknown /proc/60816/task/62104/exe (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62104 tp_osd_tp ceph NOFD /proc/60816/task/62104/fd (opendir: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62105 tp_osd_tp ceph cwd unknown /proc/60816/task/62105/cwd (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62105 tp_osd_tp ceph rtd unknown /proc/60816/task/62105/root (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62105 tp_osd_tp ceph txt unknown /proc/60816/task/62105/exe (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62105 tp_osd_tp ceph NOFD /proc/60816/task/62105/fd (opendir: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62106 tp_osd_tp ceph cwd unknown /proc/60816/task/62106/cwd (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62106 tp_osd_tp ceph rtd unknown /proc/60816/task/62106/root (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62106 tp_osd_tp ceph txt unknown /proc/60816/task/62106/exe (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62106 tp_osd_tp ceph NOFD /proc/60816/task/62106/fd (opendir: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62107 tp_osd_tp ceph cwd unknown /proc/60816/task/62107/cwd (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62107 tp_osd_tp ceph rtd unknown /proc/60816/task/62107/root (readlink: Permission denied) 2024-04-21T09:10:05.457 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62107 tp_osd_tp ceph txt unknown /proc/60816/task/62107/exe (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62107 tp_osd_tp ceph NOFD /proc/60816/task/62107/fd (opendir: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62108 tp_osd_tp ceph cwd unknown /proc/60816/task/62108/cwd (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62108 tp_osd_tp ceph rtd unknown /proc/60816/task/62108/root (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62108 tp_osd_tp ceph txt unknown /proc/60816/task/62108/exe (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62108 tp_osd_tp ceph NOFD /proc/60816/task/62108/fd (opendir: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62109 tp_osd_tp ceph cwd unknown /proc/60816/task/62109/cwd (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62109 tp_osd_tp ceph rtd unknown /proc/60816/task/62109/root (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62109 tp_osd_tp ceph txt unknown /proc/60816/task/62109/exe (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62109 tp_osd_tp ceph NOFD /proc/60816/task/62109/fd (opendir: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62110 tp_osd_tp ceph cwd unknown /proc/60816/task/62110/cwd (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62110 tp_osd_tp ceph rtd unknown /proc/60816/task/62110/root (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62110 tp_osd_tp ceph txt unknown /proc/60816/task/62110/exe (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62110 tp_osd_tp ceph NOFD /proc/60816/task/62110/fd (opendir: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62111 tp_osd_tp ceph cwd unknown /proc/60816/task/62111/cwd (readlink: Permission denied) 2024-04-21T09:10:05.458 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62111 tp_osd_tp ceph rtd unknown /proc/60816/task/62111/root (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62111 tp_osd_tp ceph txt unknown /proc/60816/task/62111/exe (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62111 tp_osd_tp ceph NOFD /proc/60816/task/62111/fd (opendir: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62112 tp_osd_tp ceph cwd unknown /proc/60816/task/62112/cwd (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62112 tp_osd_tp ceph rtd unknown /proc/60816/task/62112/root (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62112 tp_osd_tp ceph txt unknown /proc/60816/task/62112/exe (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62112 tp_osd_tp ceph NOFD /proc/60816/task/62112/fd (opendir: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62113 tp_osd_tp ceph cwd unknown /proc/60816/task/62113/cwd (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62113 tp_osd_tp ceph rtd unknown /proc/60816/task/62113/root (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62113 tp_osd_tp ceph txt unknown /proc/60816/task/62113/exe (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62113 tp_osd_tp ceph NOFD /proc/60816/task/62113/fd (opendir: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62114 tp_osd_tp ceph cwd unknown /proc/60816/task/62114/cwd (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62114 tp_osd_tp ceph rtd unknown /proc/60816/task/62114/root (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62114 tp_osd_tp ceph txt unknown /proc/60816/task/62114/exe (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62114 tp_osd_tp ceph NOFD /proc/60816/task/62114/fd (opendir: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62115 osd_srv_h ceph cwd unknown /proc/60816/task/62115/cwd (readlink: Permission denied) 2024-04-21T09:10:05.459 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62115 osd_srv_h ceph rtd unknown /proc/60816/task/62115/root (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62115 osd_srv_h ceph txt unknown /proc/60816/task/62115/exe (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-osd 60816 62115 osd_srv_h ceph NOFD /proc/60816/task/62115/fd (opendir: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 62137 root cwd unknown /proc/62137/cwd (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 62137 root rtd unknown /proc/62137/root (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 62137 root txt unknown /proc/62137/exe (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 62137 root NOFD /proc/62137/fd (opendir: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 62155 root cwd unknown /proc/62155/cwd (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 62155 root rtd unknown /proc/62155/root (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 62155 root txt unknown /proc/62155/exe (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 62155 root NOFD /proc/62155/fd (opendir: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 64572 root cwd unknown /proc/64572/cwd (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 64572 root rtd unknown /proc/64572/root (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 64572 root txt unknown /proc/64572/exe (readlink: Permission denied) 2024-04-21T09:10:05.460 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/2 64572 root NOFD /proc/64572/fd (opendir: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65301 root cwd unknown /proc/65301/cwd (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65301 root rtd unknown /proc/65301/root (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65301 root txt unknown /proc/65301/exe (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65301 root NOFD /proc/65301/fd (opendir: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65318 root cwd unknown /proc/65318/cwd (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65318 root rtd unknown /proc/65318/root (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65318 root txt unknown /proc/65318/exe (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65318 root NOFD /proc/65318/fd (opendir: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65319 root cwd unknown /proc/65319/cwd (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65319 root rtd unknown /proc/65319/root (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65319 root txt unknown /proc/65319/exe (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65319 root NOFD /proc/65319/fd (opendir: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65336 root cwd unknown /proc/65336/cwd (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65336 root rtd unknown /proc/65336/root (readlink: Permission denied) 2024-04-21T09:10:05.461 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65336 root txt unknown /proc/65336/exe (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65336 root NOFD /proc/65336/fd (opendir: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65337 root cwd unknown /proc/65337/cwd (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65337 root rtd unknown /proc/65337/root (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65337 root txt unknown /proc/65337/exe (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65337 root NOFD /proc/65337/fd (opendir: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65354 root cwd unknown /proc/65354/cwd (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65354 root rtd unknown /proc/65354/root (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65354 root txt unknown /proc/65354/exe (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65354 root NOFD /proc/65354/fd (opendir: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65355 root cwd unknown /proc/65355/cwd (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65355 root rtd unknown /proc/65355/root (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65355 root txt unknown /proc/65355/exe (readlink: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 65355 root NOFD /proc/65355/fd (opendir: Permission denied) 2024-04-21T09:10:05.462 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65372 root cwd unknown /proc/65372/cwd (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65372 root rtd unknown /proc/65372/root (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65372 root txt unknown /proc/65372/exe (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:journalct 65372 root NOFD /proc/65372/fd (opendir: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 65537 root cwd unknown /proc/65537/cwd (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 65537 root rtd unknown /proc/65537/root (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 65537 root txt unknown /proc/65537/exe (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/6 65537 root NOFD /proc/65537/fd (opendir: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 65895 root cwd unknown /proc/65895/cwd (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 65895 root rtd unknown /proc/65895/root (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 65895 root txt unknown /proc/65895/exe (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 65895 root NOFD /proc/65895/fd (opendir: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 65897 root cwd unknown /proc/65897/cwd (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 65897 root rtd unknown /proc/65897/root (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 65897 root txt unknown /proc/65897/exe (readlink: Permission denied) 2024-04-21T09:10:05.463 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 65897 root NOFD /proc/65897/fd (opendir: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 ceph cwd unknown /proc/65899/cwd (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 ceph rtd unknown /proc/65899/root (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 ceph txt unknown /proc/65899/exe (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 ceph NOFD /proc/65899/fd (opendir: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65900 log ceph cwd unknown /proc/65899/task/65900/cwd (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65900 log ceph rtd unknown /proc/65899/task/65900/root (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65900 log ceph txt unknown /proc/65899/task/65900/exe (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65900 log ceph NOFD /proc/65899/task/65900/fd (opendir: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65902 msgr-work ceph cwd unknown /proc/65899/task/65902/cwd (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65902 msgr-work ceph rtd unknown /proc/65899/task/65902/root (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65902 msgr-work ceph txt unknown /proc/65899/task/65902/exe (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65902 msgr-work ceph NOFD /proc/65899/task/65902/fd (opendir: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65903 msgr-work ceph cwd unknown /proc/65899/task/65903/cwd (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65903 msgr-work ceph rtd unknown /proc/65899/task/65903/root (readlink: Permission denied) 2024-04-21T09:10:05.464 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65903 msgr-work ceph txt unknown /proc/65899/task/65903/exe (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65903 msgr-work ceph NOFD /proc/65899/task/65903/fd (opendir: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65904 msgr-work ceph cwd unknown /proc/65899/task/65904/cwd (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65904 msgr-work ceph rtd unknown /proc/65899/task/65904/root (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65904 msgr-work ceph txt unknown /proc/65899/task/65904/exe (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65904 msgr-work ceph NOFD /proc/65899/task/65904/fd (opendir: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65908 service ceph cwd unknown /proc/65899/task/65908/cwd (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65908 service ceph rtd unknown /proc/65899/task/65908/root (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65908 service ceph txt unknown /proc/65899/task/65908/exe (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65908 service ceph NOFD /proc/65899/task/65908/fd (opendir: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65909 admin_soc ceph cwd unknown /proc/65899/task/65909/cwd (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65909 admin_soc ceph rtd unknown /proc/65899/task/65909/root (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65909 admin_soc ceph txt unknown /proc/65899/task/65909/exe (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65909 admin_soc ceph NOFD /proc/65899/task/65909/fd (opendir: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65910 signal_ha ceph cwd unknown /proc/65899/task/65910/cwd (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65910 signal_ha ceph rtd unknown /proc/65899/task/65910/root (readlink: Permission denied) 2024-04-21T09:10:05.465 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65910 signal_ha ceph txt unknown /proc/65899/task/65910/exe (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65910 signal_ha ceph NOFD /proc/65899/task/65910/fd (opendir: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65911 io_contex ceph cwd unknown /proc/65899/task/65911/cwd (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65911 io_contex ceph rtd unknown /proc/65899/task/65911/root (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65911 io_contex ceph txt unknown /proc/65899/task/65911/exe (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65911 io_contex ceph NOFD /proc/65899/task/65911/fd (opendir: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65912 io_contex ceph cwd unknown /proc/65899/task/65912/cwd (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65912 io_contex ceph rtd unknown /proc/65899/task/65912/root (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65912 io_contex ceph txt unknown /proc/65899/task/65912/exe (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65912 io_contex ceph NOFD /proc/65899/task/65912/fd (opendir: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65913 ms_dispat ceph cwd unknown /proc/65899/task/65913/cwd (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65913 ms_dispat ceph rtd unknown /proc/65899/task/65913/root (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65913 ms_dispat ceph txt unknown /proc/65899/task/65913/exe (readlink: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65913 ms_dispat ceph NOFD /proc/65899/task/65913/fd (opendir: Permission denied) 2024-04-21T09:10:05.466 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65914 ms_local ceph cwd unknown /proc/65899/task/65914/cwd (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65914 ms_local ceph rtd unknown /proc/65899/task/65914/root (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65914 ms_local ceph txt unknown /proc/65899/task/65914/exe (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65914 ms_local ceph NOFD /proc/65899/task/65914/fd (opendir: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65915 safe_time ceph cwd unknown /proc/65899/task/65915/cwd (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65915 safe_time ceph rtd unknown /proc/65899/task/65915/root (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65915 safe_time ceph txt unknown /proc/65899/task/65915/exe (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65915 safe_time ceph NOFD /proc/65899/task/65915/fd (opendir: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65916 safe_time ceph cwd unknown /proc/65899/task/65916/cwd (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65916 safe_time ceph rtd unknown /proc/65899/task/65916/root (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65916 safe_time ceph txt unknown /proc/65899/task/65916/exe (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65916 safe_time ceph NOFD /proc/65899/task/65916/fd (opendir: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65917 ceph-mds ceph cwd unknown /proc/65899/task/65917/cwd (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65917 ceph-mds ceph rtd unknown /proc/65899/task/65917/root (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65917 ceph-mds ceph txt unknown /proc/65899/task/65917/exe (readlink: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65917 ceph-mds ceph NOFD /proc/65899/task/65917/fd (opendir: Permission denied) 2024-04-21T09:10:05.467 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65922 safe_time ceph cwd unknown /proc/65899/task/65922/cwd (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65922 safe_time ceph rtd unknown /proc/65899/task/65922/root (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65922 safe_time ceph txt unknown /proc/65899/task/65922/exe (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 65899 65922 safe_time ceph NOFD /proc/65899/task/65922/fd (opendir: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66251 root cwd unknown /proc/66251/cwd (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66251 root rtd unknown /proc/66251/root (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66251 root txt unknown /proc/66251/exe (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66251 root NOFD /proc/66251/fd (opendir: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66253 root cwd unknown /proc/66253/cwd (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66253 root rtd unknown /proc/66253/root (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66253 root txt unknown /proc/66253/exe (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66253 root NOFD /proc/66253/fd (opendir: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 ceph cwd unknown /proc/66255/cwd (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 ceph rtd unknown /proc/66255/root (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 ceph txt unknown /proc/66255/exe (readlink: Permission denied) 2024-04-21T09:10:05.468 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 ceph NOFD /proc/66255/fd (opendir: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66256 log ceph cwd unknown /proc/66255/task/66256/cwd (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66256 log ceph rtd unknown /proc/66255/task/66256/root (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66256 log ceph txt unknown /proc/66255/task/66256/exe (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66256 log ceph NOFD /proc/66255/task/66256/fd (opendir: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66258 msgr-work ceph cwd unknown /proc/66255/task/66258/cwd (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66258 msgr-work ceph rtd unknown /proc/66255/task/66258/root (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66258 msgr-work ceph txt unknown /proc/66255/task/66258/exe (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66258 msgr-work ceph NOFD /proc/66255/task/66258/fd (opendir: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66259 msgr-work ceph cwd unknown /proc/66255/task/66259/cwd (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66259 msgr-work ceph rtd unknown /proc/66255/task/66259/root (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66259 msgr-work ceph txt unknown /proc/66255/task/66259/exe (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66259 msgr-work ceph NOFD /proc/66255/task/66259/fd (opendir: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66260 msgr-work ceph cwd unknown /proc/66255/task/66260/cwd (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66260 msgr-work ceph rtd unknown /proc/66255/task/66260/root (readlink: Permission denied) 2024-04-21T09:10:05.469 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66260 msgr-work ceph txt unknown /proc/66255/task/66260/exe (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66260 msgr-work ceph NOFD /proc/66255/task/66260/fd (opendir: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66264 service ceph cwd unknown /proc/66255/task/66264/cwd (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66264 service ceph rtd unknown /proc/66255/task/66264/root (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66264 service ceph txt unknown /proc/66255/task/66264/exe (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66264 service ceph NOFD /proc/66255/task/66264/fd (opendir: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66265 admin_soc ceph cwd unknown /proc/66255/task/66265/cwd (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66265 admin_soc ceph rtd unknown /proc/66255/task/66265/root (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66265 admin_soc ceph txt unknown /proc/66255/task/66265/exe (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66265 admin_soc ceph NOFD /proc/66255/task/66265/fd (opendir: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66266 signal_ha ceph cwd unknown /proc/66255/task/66266/cwd (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66266 signal_ha ceph rtd unknown /proc/66255/task/66266/root (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66266 signal_ha ceph txt unknown /proc/66255/task/66266/exe (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66266 signal_ha ceph NOFD /proc/66255/task/66266/fd (opendir: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66267 io_contex ceph cwd unknown /proc/66255/task/66267/cwd (readlink: Permission denied) 2024-04-21T09:10:05.470 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66267 io_contex ceph rtd unknown /proc/66255/task/66267/root (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66267 io_contex ceph txt unknown /proc/66255/task/66267/exe (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66267 io_contex ceph NOFD /proc/66255/task/66267/fd (opendir: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66268 io_contex ceph cwd unknown /proc/66255/task/66268/cwd (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66268 io_contex ceph rtd unknown /proc/66255/task/66268/root (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66268 io_contex ceph txt unknown /proc/66255/task/66268/exe (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66268 io_contex ceph NOFD /proc/66255/task/66268/fd (opendir: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66269 ms_dispat ceph cwd unknown /proc/66255/task/66269/cwd (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66269 ms_dispat ceph rtd unknown /proc/66255/task/66269/root (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66269 ms_dispat ceph txt unknown /proc/66255/task/66269/exe (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66269 ms_dispat ceph NOFD /proc/66255/task/66269/fd (opendir: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66270 ms_local ceph cwd unknown /proc/66255/task/66270/cwd (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66270 ms_local ceph rtd unknown /proc/66255/task/66270/root (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66270 ms_local ceph txt unknown /proc/66255/task/66270/exe (readlink: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66270 ms_local ceph NOFD /proc/66255/task/66270/fd (opendir: Permission denied) 2024-04-21T09:10:05.471 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66271 safe_time ceph cwd unknown /proc/66255/task/66271/cwd (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66271 safe_time ceph rtd unknown /proc/66255/task/66271/root (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66271 safe_time ceph txt unknown /proc/66255/task/66271/exe (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66271 safe_time ceph NOFD /proc/66255/task/66271/fd (opendir: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66272 safe_time ceph cwd unknown /proc/66255/task/66272/cwd (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66272 safe_time ceph rtd unknown /proc/66255/task/66272/root (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66272 safe_time ceph txt unknown /proc/66255/task/66272/exe (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66272 safe_time ceph NOFD /proc/66255/task/66272/fd (opendir: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66273 ceph-mds ceph cwd unknown /proc/66255/task/66273/cwd (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66273 ceph-mds ceph rtd unknown /proc/66255/task/66273/root (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66273 ceph-mds ceph txt unknown /proc/66255/task/66273/exe (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66273 ceph-mds ceph NOFD /proc/66255/task/66273/fd (opendir: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66278 safe_time ceph cwd unknown /proc/66255/task/66278/cwd (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66278 safe_time ceph rtd unknown /proc/66255/task/66278/root (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66278 safe_time ceph txt unknown /proc/66255/task/66278/exe (readlink: Permission denied) 2024-04-21T09:10:05.472 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66255 66278 safe_time ceph NOFD /proc/66255/task/66278/fd (opendir: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66862 root cwd unknown /proc/66862/cwd (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66862 root rtd unknown /proc/66862/root (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66862 root txt unknown /proc/66862/exe (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 66862 root NOFD /proc/66862/fd (opendir: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66864 root cwd unknown /proc/66864/cwd (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66864 root rtd unknown /proc/66864/root (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66864 root txt unknown /proc/66864/exe (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 66864 root NOFD /proc/66864/fd (opendir: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 ceph cwd unknown /proc/66866/cwd (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 ceph rtd unknown /proc/66866/root (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 ceph txt unknown /proc/66866/exe (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 ceph NOFD /proc/66866/fd (opendir: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66867 log ceph cwd unknown /proc/66866/task/66867/cwd (readlink: Permission denied) 2024-04-21T09:10:05.473 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66867 log ceph rtd unknown /proc/66866/task/66867/root (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66867 log ceph txt unknown /proc/66866/task/66867/exe (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66867 log ceph NOFD /proc/66866/task/66867/fd (opendir: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66869 msgr-work ceph cwd unknown /proc/66866/task/66869/cwd (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66869 msgr-work ceph rtd unknown /proc/66866/task/66869/root (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66869 msgr-work ceph txt unknown /proc/66866/task/66869/exe (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66869 msgr-work ceph NOFD /proc/66866/task/66869/fd (opendir: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66870 msgr-work ceph cwd unknown /proc/66866/task/66870/cwd (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66870 msgr-work ceph rtd unknown /proc/66866/task/66870/root (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66870 msgr-work ceph txt unknown /proc/66866/task/66870/exe (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66870 msgr-work ceph NOFD /proc/66866/task/66870/fd (opendir: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66871 msgr-work ceph cwd unknown /proc/66866/task/66871/cwd (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66871 msgr-work ceph rtd unknown /proc/66866/task/66871/root (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66871 msgr-work ceph txt unknown /proc/66866/task/66871/exe (readlink: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66871 msgr-work ceph NOFD /proc/66866/task/66871/fd (opendir: Permission denied) 2024-04-21T09:10:05.474 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66875 service ceph cwd unknown /proc/66866/task/66875/cwd (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66875 service ceph rtd unknown /proc/66866/task/66875/root (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66875 service ceph txt unknown /proc/66866/task/66875/exe (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66875 service ceph NOFD /proc/66866/task/66875/fd (opendir: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66876 admin_soc ceph cwd unknown /proc/66866/task/66876/cwd (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66876 admin_soc ceph rtd unknown /proc/66866/task/66876/root (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66876 admin_soc ceph txt unknown /proc/66866/task/66876/exe (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66876 admin_soc ceph NOFD /proc/66866/task/66876/fd (opendir: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66877 signal_ha ceph cwd unknown /proc/66866/task/66877/cwd (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66877 signal_ha ceph rtd unknown /proc/66866/task/66877/root (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66877 signal_ha ceph txt unknown /proc/66866/task/66877/exe (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66877 signal_ha ceph NOFD /proc/66866/task/66877/fd (opendir: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66878 io_contex ceph cwd unknown /proc/66866/task/66878/cwd (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66878 io_contex ceph rtd unknown /proc/66866/task/66878/root (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66878 io_contex ceph txt unknown /proc/66866/task/66878/exe (readlink: Permission denied) 2024-04-21T09:10:05.475 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66878 io_contex ceph NOFD /proc/66866/task/66878/fd (opendir: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66879 io_contex ceph cwd unknown /proc/66866/task/66879/cwd (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66879 io_contex ceph rtd unknown /proc/66866/task/66879/root (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66879 io_contex ceph txt unknown /proc/66866/task/66879/exe (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66879 io_contex ceph NOFD /proc/66866/task/66879/fd (opendir: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66880 ms_dispat ceph cwd unknown /proc/66866/task/66880/cwd (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66880 ms_dispat ceph rtd unknown /proc/66866/task/66880/root (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66880 ms_dispat ceph txt unknown /proc/66866/task/66880/exe (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66880 ms_dispat ceph NOFD /proc/66866/task/66880/fd (opendir: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66881 ms_local ceph cwd unknown /proc/66866/task/66881/cwd (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66881 ms_local ceph rtd unknown /proc/66866/task/66881/root (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66881 ms_local ceph txt unknown /proc/66866/task/66881/exe (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66881 ms_local ceph NOFD /proc/66866/task/66881/fd (opendir: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66882 safe_time ceph cwd unknown /proc/66866/task/66882/cwd (readlink: Permission denied) 2024-04-21T09:10:05.476 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66882 safe_time ceph rtd unknown /proc/66866/task/66882/root (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66882 safe_time ceph txt unknown /proc/66866/task/66882/exe (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66882 safe_time ceph NOFD /proc/66866/task/66882/fd (opendir: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66883 safe_time ceph cwd unknown /proc/66866/task/66883/cwd (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66883 safe_time ceph rtd unknown /proc/66866/task/66883/root (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66883 safe_time ceph txt unknown /proc/66866/task/66883/exe (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66883 safe_time ceph NOFD /proc/66866/task/66883/fd (opendir: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66884 ceph-mds ceph cwd unknown /proc/66866/task/66884/cwd (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66884 ceph-mds ceph rtd unknown /proc/66866/task/66884/root (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66884 ceph-mds ceph txt unknown /proc/66866/task/66884/exe (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66884 ceph-mds ceph NOFD /proc/66866/task/66884/fd (opendir: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66889 safe_time ceph cwd unknown /proc/66866/task/66889/cwd (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66889 safe_time ceph rtd unknown /proc/66866/task/66889/root (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66889 safe_time ceph txt unknown /proc/66866/task/66889/exe (readlink: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 66866 66889 safe_time ceph NOFD /proc/66866/task/66889/fd (opendir: Permission denied) 2024-04-21T09:10:05.477 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 67210 root cwd unknown /proc/67210/cwd (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 67210 root rtd unknown /proc/67210/root (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 67210 root txt unknown /proc/67210/exe (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:conmon 67210 root NOFD /proc/67210/fd (opendir: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 67212 root cwd unknown /proc/67212/cwd (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 67212 root rtd unknown /proc/67212/root (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 67212 root txt unknown /proc/67212/exe (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:podman-in 67212 root NOFD /proc/67212/fd (opendir: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 ceph cwd unknown /proc/67214/cwd (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 ceph rtd unknown /proc/67214/root (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 ceph txt unknown /proc/67214/exe (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 ceph NOFD /proc/67214/fd (opendir: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67215 log ceph cwd unknown /proc/67214/task/67215/cwd (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67215 log ceph rtd unknown /proc/67214/task/67215/root (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67215 log ceph txt unknown /proc/67214/task/67215/exe (readlink: Permission denied) 2024-04-21T09:10:05.478 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67215 log ceph NOFD /proc/67214/task/67215/fd (opendir: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67217 msgr-work ceph cwd unknown /proc/67214/task/67217/cwd (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67217 msgr-work ceph rtd unknown /proc/67214/task/67217/root (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67217 msgr-work ceph txt unknown /proc/67214/task/67217/exe (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67217 msgr-work ceph NOFD /proc/67214/task/67217/fd (opendir: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67218 msgr-work ceph cwd unknown /proc/67214/task/67218/cwd (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67218 msgr-work ceph rtd unknown /proc/67214/task/67218/root (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67218 msgr-work ceph txt unknown /proc/67214/task/67218/exe (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67218 msgr-work ceph NOFD /proc/67214/task/67218/fd (opendir: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67219 msgr-work ceph cwd unknown /proc/67214/task/67219/cwd (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67219 msgr-work ceph rtd unknown /proc/67214/task/67219/root (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67219 msgr-work ceph txt unknown /proc/67214/task/67219/exe (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67219 msgr-work ceph NOFD /proc/67214/task/67219/fd (opendir: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67223 service ceph cwd unknown /proc/67214/task/67223/cwd (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67223 service ceph rtd unknown /proc/67214/task/67223/root (readlink: Permission denied) 2024-04-21T09:10:05.479 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67223 service ceph txt unknown /proc/67214/task/67223/exe (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67223 service ceph NOFD /proc/67214/task/67223/fd (opendir: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67224 admin_soc ceph cwd unknown /proc/67214/task/67224/cwd (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67224 admin_soc ceph rtd unknown /proc/67214/task/67224/root (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67224 admin_soc ceph txt unknown /proc/67214/task/67224/exe (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67224 admin_soc ceph NOFD /proc/67214/task/67224/fd (opendir: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67225 signal_ha ceph cwd unknown /proc/67214/task/67225/cwd (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67225 signal_ha ceph rtd unknown /proc/67214/task/67225/root (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67225 signal_ha ceph txt unknown /proc/67214/task/67225/exe (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67225 signal_ha ceph NOFD /proc/67214/task/67225/fd (opendir: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67226 io_contex ceph cwd unknown /proc/67214/task/67226/cwd (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67226 io_contex ceph rtd unknown /proc/67214/task/67226/root (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67226 io_contex ceph txt unknown /proc/67214/task/67226/exe (readlink: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67226 io_contex ceph NOFD /proc/67214/task/67226/fd (opendir: Permission denied) 2024-04-21T09:10:05.480 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67227 io_contex ceph cwd unknown /proc/67214/task/67227/cwd (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67227 io_contex ceph rtd unknown /proc/67214/task/67227/root (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67227 io_contex ceph txt unknown /proc/67214/task/67227/exe (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67227 io_contex ceph NOFD /proc/67214/task/67227/fd (opendir: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67228 ms_dispat ceph cwd unknown /proc/67214/task/67228/cwd (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67228 ms_dispat ceph rtd unknown /proc/67214/task/67228/root (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67228 ms_dispat ceph txt unknown /proc/67214/task/67228/exe (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67228 ms_dispat ceph NOFD /proc/67214/task/67228/fd (opendir: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67229 ms_local ceph cwd unknown /proc/67214/task/67229/cwd (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67229 ms_local ceph rtd unknown /proc/67214/task/67229/root (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67229 ms_local ceph txt unknown /proc/67214/task/67229/exe (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67229 ms_local ceph NOFD /proc/67214/task/67229/fd (opendir: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67230 safe_time ceph cwd unknown /proc/67214/task/67230/cwd (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67230 safe_time ceph rtd unknown /proc/67214/task/67230/root (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67230 safe_time ceph txt unknown /proc/67214/task/67230/exe (readlink: Permission denied) 2024-04-21T09:10:05.481 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67230 safe_time ceph NOFD /proc/67214/task/67230/fd (opendir: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67231 safe_time ceph cwd unknown /proc/67214/task/67231/cwd (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67231 safe_time ceph rtd unknown /proc/67214/task/67231/root (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67231 safe_time ceph txt unknown /proc/67214/task/67231/exe (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67231 safe_time ceph NOFD /proc/67214/task/67231/fd (opendir: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67232 ceph-mds ceph cwd unknown /proc/67214/task/67232/cwd (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67232 ceph-mds ceph rtd unknown /proc/67214/task/67232/root (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67232 ceph-mds ceph txt unknown /proc/67214/task/67232/exe (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67232 ceph-mds ceph NOFD /proc/67214/task/67232/fd (opendir: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67257 safe_time ceph cwd unknown /proc/67214/task/67257/cwd (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67257 safe_time ceph rtd unknown /proc/67214/task/67257/root (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67257 safe_time ceph txt unknown /proc/67214/task/67257/exe (readlink: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-mds 67214 67257 safe_time ceph NOFD /proc/67214/task/67257/fd (opendir: Permission denied) 2024-04-21T09:10:05.482 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 71321 root cwd unknown /proc/71321/cwd (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 71321 root rtd unknown /proc/71321/root (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 71321 root txt unknown /proc/71321/exe (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/3 71321 root NOFD /proc/71321/fd (opendir: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 72618 root cwd unknown /proc/72618/cwd (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 72618 root rtd unknown /proc/72618/root (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 72618 root txt unknown /proc/72618/exe (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/5 72618 root NOFD /proc/72618/fd (opendir: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 72928 root cwd unknown /proc/72928/cwd (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 72928 root rtd unknown /proc/72928/root (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 72928 root txt unknown /proc/72928/exe (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 72928 root NOFD /proc/72928/fd (opendir: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 73472 root cwd unknown /proc/73472/cwd (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 73472 root rtd unknown /proc/73472/root (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 73472 root txt unknown /proc/73472/exe (readlink: Permission denied) 2024-04-21T09:10:05.483 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/u 73472 root NOFD /proc/73472/fd (opendir: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 73928 root cwd unknown /proc/73928/cwd (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 73928 root rtd unknown /proc/73928/root (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 73928 root txt unknown /proc/73928/exe (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 73928 root NOFD /proc/73928/fd (opendir: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 75920 root cwd unknown /proc/75920/cwd (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 75920 root rtd unknown /proc/75920/root (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 75920 root txt unknown /proc/75920/exe (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 75920 root NOFD /proc/75920/fd (opendir: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 77010 root cwd unknown /proc/77010/cwd (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 77010 root rtd unknown /proc/77010/root (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 77010 root txt unknown /proc/77010/exe (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/4 77010 root NOFD /proc/77010/fd (opendir: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-msgr 79475 root cwd unknown /proc/79475/cwd (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-msgr 79475 root rtd unknown /proc/79475/root (readlink: Permission denied) 2024-04-21T09:10:05.484 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-msgr 79475 root txt unknown /proc/79475/exe (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-msgr 79475 root NOFD /proc/79475/fd (opendir: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-watc 79476 root cwd unknown /proc/79476/cwd (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-watc 79476 root rtd unknown /proc/79476/root (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-watc 79476 root txt unknown /proc/79476/exe (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-watc 79476 root NOFD /proc/79476/fd (opendir: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-comp 79477 root cwd unknown /proc/79477/cwd (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-comp 79477 root rtd unknown /proc/79477/root (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-comp 79477 root txt unknown /proc/79477/exe (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:ceph-comp 79477 root NOFD /proc/79477/fd (opendir: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 79671 root cwd unknown /proc/79671/cwd (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 79671 root rtd unknown /proc/79671/root (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 79671 root txt unknown /proc/79671/exe (readlink: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/0 79671 root NOFD /proc/79671/fd (opendir: Permission denied) 2024-04-21T09:10:05.485 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 79985 root cwd unknown /proc/79985/cwd (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 79985 root rtd unknown /proc/79985/root (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 79985 root txt unknown /proc/79985/exe (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 79985 root NOFD /proc/79985/fd (opendir: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 80086 root cwd unknown /proc/80086/cwd (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 80086 root rtd unknown /proc/80086/root (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 80086 root txt unknown /proc/80086/exe (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sshd 80086 root NOFD /proc/80086/fd (opendir: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 82427 root cwd unknown /proc/82427/cwd (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 82427 root rtd unknown /proc/82427/root (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 82427 root txt unknown /proc/82427/exe (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:sudo 82427 root NOFD /proc/82427/fd (opendir: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:daemon-he 82444 root cwd unknown /proc/82444/cwd (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:daemon-he 82444 root rtd unknown /proc/82444/root (readlink: Permission denied) 2024-04-21T09:10:05.486 INFO:tasks.workunit.client.0.smithi066.stdout:daemon-he 82444 root txt unknown /proc/82444/exe (readlink: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:daemon-he 82444 root NOFD /proc/82444/fd (opendir: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:bash 82445 root cwd unknown /proc/82445/cwd (readlink: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:bash 82445 root rtd unknown /proc/82445/root (readlink: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:bash 82445 root txt unknown /proc/82445/exe (readlink: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:bash 82445 root NOFD /proc/82445/fd (opendir: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 83664 root cwd unknown /proc/83664/cwd (readlink: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 83664 root rtd unknown /proc/83664/root (readlink: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 83664 root txt unknown /proc/83664/exe (readlink: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:kworker/1 83664 root NOFD /proc/83664/fd (opendir: Permission denied) 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu rtd DIR 8,1 4096 2 / 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-21T09:10:05.487 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu 0r FIFO 0,13 0t0 235748 pipe 2024-04-21T09:10:05.488 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu 1w FIFO 0,13 0t0 235749 pipe 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:timeout 84303 ubuntu 2w FIFO 0,13 0t0 235750 pipe 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu rtd DIR 8,1 4096 2 / 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-21T09:10:05.489 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu 0r FIFO 0,13 0t0 235748 pipe 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu 1w FIFO 0,13 0t0 235749 pipe 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu 2w FIFO 0,13 0t0 235750 pipe 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:fsync-tes 84321 ubuntu 255r REG 8,1 207 530566 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:sleep 85645 root cwd unknown /proc/85645/cwd (readlink: Permission denied) 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:sleep 85645 root rtd unknown /proc/85645/root (readlink: Permission denied) 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:sleep 85645 root txt unknown /proc/85645/exe (readlink: Permission denied) 2024-04-21T09:10:05.490 INFO:tasks.workunit.client.0.smithi066.stdout:sleep 85645 root NOFD /proc/85645/fd (opendir: Permission denied) 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu rtd DIR 8,1 4096 2 / 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-21T09:10:05.491 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu 0r FIFO 0,13 0t0 235748 pipe 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu 1w FIFO 0,13 0t0 235749 pipe 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu 2w FIFO 0,13 0t0 235750 pipe 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu 3r DIR 0,20 0 1 /proc 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu 4r DIR 0,20 0 239663 /proc/85647/fd 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu 5w FIFO 0,13 0t0 239668 pipe 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85647 ubuntu 6r FIFO 0,13 0t0 239669 pipe 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu rtd DIR 8,1 4096 2 / 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-21T09:10:05.492 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-21T09:10:05.493 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-21T09:10:05.494 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu 4r FIFO 0,13 0t0 239668 pipe 2024-04-21T09:10:05.494 INFO:tasks.workunit.client.0.smithi066.stdout:lsof 85648 ubuntu 7w FIFO 0,13 0t0 239669 pipe 2024-04-21T09:10:05.494 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-21T09:10:05.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T09:10:05.516 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:05 smithi066 ceph-mon[27953]: pgmap v59: 129 pgs: 129 active+clean; 260 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-21T09:10:05.535 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-04-21T09:10:05.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-21T09:10:05.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:05 smithi186 ceph-mon[31505]: pgmap v59: 129 pgs: 129 active+clean; 260 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-21T09:10:05.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:05 smithi083 ceph-mon[31413]: pgmap v59: 129 pgs: 129 active+clean; 260 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-21T09:10:05.871 DEBUG:teuthology.parallel:result is None 2024-04-21T09:10:05.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T09:10:05.900 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T09:10:05.900 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-21T09:10:05.911 INFO:tasks.quiescer:joining Quiescers 2024-04-21T09:10:05.911 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:10:06.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 -- 172.21.15.66:0/795142963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118101b60 msgr2=0x7fb11810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/795142963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118101b60 0x7fb11810dc90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb108009cd0 tx=0x7fb10802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 -- 172.21.15.66:0/795142963 shutdown_connections 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/795142963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118101b60 0x7fb11810dc90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/795142963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1181011a0 0x7fb118101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/795142963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118105e30 0x7fb118106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 -- 172.21.15.66:0/795142963 >> 172.21.15.66:0/795142963 conn(0x7fb118077e30 msgr2=0x7fb118078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 -- 172.21.15.66:0/795142963 shutdown_connections 2024-04-21T09:10:06.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.043+0000 7fb11f6e4640 1 -- 172.21.15.66:0/795142963 wait complete. 2024-04-21T09:10:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 Processor -- start 2024-04-21T09:10:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 -- start start 2024-04-21T09:10:06.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1181011a0 0x7fb1181975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118101b60 0x7fb118197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118105e30 0x7fb11819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181110c0 con 0x7fb118105e30 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb118110f40 con 0x7fb118101b60 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11f6e4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb118111240 con 0x7fb1181011a0 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118105e30 0x7fb11819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118105e30 0x7fb11819be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43872/0 (socket says 172.21.15.66:43872) 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 -- 172.21.15.66:0/3325664620 learned_addr learned my addr 172.21.15.66:0/3325664620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11cc7f640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118101b60 0x7fb118197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11d480640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1181011a0 0x7fb1181975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 -- 172.21.15.66:0/3325664620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1181011a0 msgr2=0x7fb1181975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1181011a0 0x7fb1181975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 -- 172.21.15.66:0/3325664620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118101b60 msgr2=0x7fb118197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118101b60 0x7fb118197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.044+0000 7fb11dc81640 1 -- 172.21.15.66:0/3325664620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11819c5b0 con 0x7fb118105e30 2024-04-21T09:10:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.045+0000 7fb11dc81640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118105e30 0x7fb11819be80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb108004af0 tx=0x7fb108034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.045+0000 7fb1167fc640 1 -- 172.21.15.66:0/3325664620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108041020 con 0x7fb118105e30 2024-04-21T09:10:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.045+0000 7fb1167fc640 1 -- 172.21.15.66:0/3325664620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb108034ca0 con 0x7fb118105e30 2024-04-21T09:10:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.045+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11819c840 con 0x7fb118105e30 2024-04-21T09:10:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.045+0000 7fb1167fc640 1 -- 172.21.15.66:0/3325664620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108039040 con 0x7fb118105e30 2024-04-21T09:10:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.045+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1181a4130 con 0x7fb118105e30 2024-04-21T09:10:06.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.046+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb118068de0 con 0x7fb118105e30 2024-04-21T09:10:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.047+0000 7fb1167fc640 1 -- 172.21.15.66:0/3325664620 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb108044020 con 0x7fb118105e30 2024-04-21T09:10:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.048+0000 7fb1167fc640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb1000a3d20 0x7fb1000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.048+0000 7fb11d480640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb1000a3d20 0x7fb1000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.048+0000 7fb1167fc640 1 -- 172.21.15.66:0/3325664620 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb1080f2780 con 0x7fb118105e30 2024-04-21T09:10:06.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.048+0000 7fb11d480640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb1000a3d20 0x7fb1000a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb11000bb60 tx=0x7fb11000b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.052+0000 7fb1167fc640 1 -- 172.21.15.66:0/3325664620 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb10804b020 con 0x7fb118105e30 2024-04-21T09:10:06.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.278+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb118109350 con 0x7fb118105e30 2024-04-21T09:10:06.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.279+0000 7fb1167fc640 1 -- 172.21.15.66:0/3325664620 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7fb1080bebc0 con 0x7fb118105e30 2024-04-21T09:10:06.280 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:10:06.280 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:10:06.280 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:10:06.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb1000a3d20 msgr2=0x7fb1000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb1000a3d20 0x7fb1000a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb11000bb60 tx=0x7fb11000b390 comp rx=0 tx=0).stop 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118105e30 msgr2=0x7fb11819be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118105e30 0x7fb11819be80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb108004af0 tx=0x7fb108034670 comp rx=0 tx=0).stop 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 shutdown_connections 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb1000a3d20 0x7fb1000a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb118105e30 0x7fb11819be80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118101b60 0x7fb118197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 --2- 172.21.15.66:0/3325664620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1181011a0 0x7fb1181975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 >> 172.21.15.66:0/3325664620 conn(0x7fb118077e30 msgr2=0x7fb1180fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 shutdown_connections 2024-04-21T09:10:06.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.281+0000 7fb11f6e4640 1 -- 172.21.15.66:0/3325664620 wait complete. 2024-04-21T09:10:06.293 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24512 quiesce db --cancel --all' 2024-04-21T09:10:06.293 DEBUG:teuthology.orchestra.run.smithi066:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24512 quiesce db --cancel --all 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 -- 172.21.15.66:0/816628270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4410c520 msgr2=0x7f4f4410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 --2- 172.21.15.66:0/816628270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4410c520 0x7f4f4410e9c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009ca0 tx=0x7f4f3c02c230 comp rx=0 tx=0).stop 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 -- 172.21.15.66:0/816628270 shutdown_connections 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 --2- 172.21.15.66:0/816628270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4410c520 0x7f4f4410e9c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 --2- 172.21.15.66:0/816628270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f44103b00 0x7f4f4410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 --2- 172.21.15.66:0/816628270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f441031e0 0x7f4f441035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 -- 172.21.15.66:0/816628270 >> 172.21.15.66:0/816628270 conn(0x7f4f440771d0 msgr2=0x7f4f440775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 -- 172.21.15.66:0/816628270 shutdown_connections 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.400+0000 7f4f4b515640 1 -- 172.21.15.66:0/816628270 wait complete. 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 Processor -- start 2024-04-21T09:10:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 -- start start 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f441031e0 0x7f4f44100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f44103b00 0x7f4f44101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4410c520 0x7f4f441018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441121b0 con 0x7f4f441031e0 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44112030 con 0x7f4f44103b00 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f4b515640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44112330 con 0x7f4f4410c520 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f492b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f441031e0 0x7f4f44100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f492b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f441031e0 0x7f4f44100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43894/0 (socket says 172.21.15.66:43894) 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f492b1640 1 -- 172.21.15.66:0/2625808670 learned_addr learned my addr 172.21.15.66:0/2625808670 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.401+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f44103b00 0x7f4f44101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4410c520 0x7f4f441018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f492b1640 1 -- 172.21.15.66:0/2625808670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4410c520 msgr2=0x7f4f441018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f492b1640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4410c520 0x7f4f441018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f492b1640 1 -- 172.21.15.66:0/2625808670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f44103b00 msgr2=0x7f4f44101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f492b1640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f44103b00 0x7f4f44101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f492b1640 1 -- 172.21.15.66:0/2625808670 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f441a6c30 con 0x7f4f441031e0 2024-04-21T09:10:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f44103b00 0x7f4f44101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f492b1640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f441031e0 0x7f4f44100e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c970 tx=0x7f4f4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f3a7fc640 1 -- 172.21.15.66:0/2625808670 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40007bb0 con 0x7f4f441031e0 2024-04-21T09:10:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f3a7fc640 1 -- 172.21.15.66:0/2625808670 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f40007d50 con 0x7f4f441031e0 2024-04-21T09:10:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f441a6f20 con 0x7f4f441031e0 2024-04-21T09:10:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f3a7fc640 1 -- 172.21.15.66:0/2625808670 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000f450 con 0x7f4f441031e0 2024-04-21T09:10:06.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.402+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f441a7390 con 0x7f4f441031e0 2024-04-21T09:10:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.404+0000 7f4f3a7fc640 1 -- 172.21.15.66:0/2625808670 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4f40004f20 con 0x7f4f441031e0 2024-04-21T09:10:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 Processor -- start 2024-04-21T09:10:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 -- start start 2024-04-21T09:10:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1ad10 0x5588bbf1b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf23c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf26690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5588bbe80000 con 0x5588bbf24240 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588bbe1cb20 con 0x5588bbf1ad10 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5588bbe7bfc0 con 0x5588bbf1b6c0 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f3a7fc640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f300a3d20 0x7f4f300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f48ab0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf26690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f48ab0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf26690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43896/0 (socket says 172.21.15.66:43896) 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f48ab0640 1 -- 172.21.15.66:0/3581508962 learned_addr learned my addr 172.21.15.66:0/3581508962 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f492b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf23c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1ad10 0x5588bbf1b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.405+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f300a3d20 0x7f4f300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f3a7fc640 1 -- 172.21.15.66:0/2625808670 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4f400d09c0 con 0x7f4f441031e0 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f48ab0640 1 -- 172.21.15.66:0/3581508962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 msgr2=0x5588bbf23c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf23c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f48ab0640 1 -- 172.21.15.66:0/3581508962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1ad10 msgr2=0x5588bbf1b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1ad10 0x5588bbf1b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f48ab0640 1 -- 172.21.15.66:0/3581508962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588bbe36ea0 con 0x5588bbf24240 2024-04-21T09:10:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf26690 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400bcb0 tx=0x7f4f3400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19bf02f82ff75c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f300a3d20 0x7f4f300a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0325e0 tx=0x7f4f3c0095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f2f7fe640 1 -- 172.21.15.66:0/3581508962 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34015070 con 0x5588bbf24240 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f2f7fe640 1 -- 172.21.15.66:0/3581508962 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f340118f0 con 0x5588bbf24240 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.406+0000 7f4f2f7fe640 1 -- 172.21.15.66:0/3581508962 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340108b0 con 0x5588bbf24240 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3581508962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 msgr2=0x5588bbf26690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf26690 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400bcb0 tx=0x7f4f3400f730 comp rx=0 tx=0).stop 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3581508962 shutdown_connections 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf26690 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf23c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3581508962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1ad10 0x5588bbf1b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3581508962 >> 172.21.15.66:0/3581508962 conn(0x5588bbf153e0 msgr2=0x5588bbf157c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3581508962 shutdown_connections 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3581508962 wait complete. 2024-04-21T09:10:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 -- start start 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.407+0000 7f4f4e1e1740 1 Processor -- start 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 0x5588bbf21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5588bbf27120 con 0x5588bbf1ad10 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588bbe1a510 con 0x5588bbf24240 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5588bbf272a0 con 0x5588bbf1b6c0 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 10 client.0 _ll_register_callbacks cb 0x5588bbe19f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf21090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43910/0 (socket says 172.21.15.66:43910) 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 -- 172.21.15.66:0/1953336203 learned_addr learned my addr 172.21.15.66:0/1953336203 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 -- 172.21.15.66:0/1953336203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 msgr2=0x5588bbf215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 -- 172.21.15.66:0/1953336203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 msgr2=0x5588bbf21b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 0x5588bbf21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 -- 172.21.15.66:0/1953336203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588bbc5ea20 con 0x5588bbf1ad10 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f492b1640 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf21090 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4f40098f80 tx=0x7f4f4006be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4ed4dd59b7a3bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f2cff9640 1 -- 172.21.15.66:0/1953336203 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4001d050 con 0x5588bbf1ad10 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f2cff9640 1 -- 172.21.15.66:0/1953336203 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4f40005210 con 0x5588bbf1ad10 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.408+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f340118f0 con 0x5588bbf1ad10 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f2cff9640 1 -- 172.21.15.66:0/1953336203 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f400ce1e0 con 0x5588bbf1ad10 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 10 client.16389 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588bbcbf1b0 con 0x5588bbf1ad10 2024-04-21T09:10:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f2cff9640 1 -- 172.21.15.66:0/1953336203 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f340118f0 con 0x5588bbf1ad10 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f2cff9640 1 -- 172.21.15.66:0/1953336203 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f40085af0 con 0x5588bbf1ad10 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 10 client.16389 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 10 client.16389 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 20 client.16389 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf34670 0x5588bbf36ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 4 client.16389 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588bbe8da20 con 0x5588bbf34670 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.409+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf34670 0x5588bbf36ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.410+0000 7f4f2cff9640 10 client.16389 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.410+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf34670 0x5588bbf36ae0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2edeaa917dda84a server_cookie=6e6bf0c8276bab52 in_seq=0 out_seq=0 2024-04-21T09:10:06.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.410+0000 7f4f2cff9640 1 -- 172.21.15.66:0/1953336203 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588bbe8da20 con 0x5588bbf34670 2024-04-21T09:10:06.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.410+0000 7f4f2cff9640 10 client.16389 handle_command_reply: tid=0 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.410+0000 7f4f4e1e1740 1 client.16389 shutdown 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 msgr2=0x5588bbf21090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf21090 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4f40098f80 tx=0x7f4f4006be90 comp rx=0 tx=0).stop 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 shutdown_connections 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf34670 0x5588bbf36ae0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 0x5588bbf21b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f2cff9640 0 client.16389 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/1953336203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf21090 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 >> 172.21.15.66:0/1953336203 conn(0x5588bbf153e0 msgr2=0x5588bbf14680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 shutdown_connections 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/1953336203 wait complete. 2024-04-21T09:10:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.411+0000 7f4f4e1e1740 20 client.16389 trim_cache size 0 max 16384 2024-04-21T09:10:06.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.428+0000 7f4f4e1e1740 1 Processor -- start 2024-04-21T09:10:06.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.428+0000 7f4f4e1e1740 1 -- start start 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1ad10 0x5588bbf237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1b6c0 0x5588bbf300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf3c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b3e20 con 0x5588bbf24240 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b16f0 con 0x5588bbf1b6c0 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441af520 con 0x5588bbf1ad10 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf3c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf3c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43924/0 (socket says 172.21.15.66:43924) 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 -- 172.21.15.66:0/3198474821 learned_addr learned my addr 172.21.15.66:0/3198474821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1b6c0 0x5588bbf300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f492b1640 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1ad10 0x5588bbf237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 -- 172.21.15.66:0/3198474821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1ad10 msgr2=0x5588bbf237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1ad10 0x5588bbf237c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 -- 172.21.15.66:0/3198474821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1b6c0 msgr2=0x5588bbf300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1b6c0 0x5588bbf300d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 -- 172.21.15.66:0/3198474821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588bbb52310 con 0x5588bbf24240 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf3c3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009ca0 tx=0x7f4f3c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea2bd3ae1e3bb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f2effd640 1 -- 172.21.15.66:0/3198474821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c053070 con 0x5588bbf24240 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f2effd640 1 -- 172.21.15.66:0/3198474821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c03a040 con 0x5588bbf24240 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.429+0000 7f4f2effd640 1 -- 172.21.15.66:0/3198474821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c039070 con 0x5588bbf24240 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3198474821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 msgr2=0x5588bbf3c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf3c3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009ca0 tx=0x7f4f3c040000 comp rx=0 tx=0).stop 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3198474821 shutdown_connections 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf24240 0x5588bbf3c3b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf1b6c0 0x5588bbf300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/3198474821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1ad10 0x5588bbf237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3198474821 >> 172.21.15.66:0/3198474821 conn(0x5588bbf153e0 msgr2=0x5588bbf14680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3198474821 shutdown_connections 2024-04-21T09:10:06.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/3198474821 wait complete. 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 -- start start 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.430+0000 7f4f4e1e1740 1 Processor -- start 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf2cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 0x5588bbf295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441afbb0 con 0x5588bbf1ad10 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b1020 con 0x5588bbf24240 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b0280 con 0x5588bbf1b6c0 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 10 client.0 _ll_register_callbacks cb 0x7f4f4009c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf2c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43934/0 (socket says 172.21.15.66:43934) 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 -- 172.21.15.66:0/687895706 learned_addr learned my addr 172.21.15.66:0/687895706 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf2cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f49ab2640 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 0x5588bbf295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 -- 172.21.15.66:0/687895706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 msgr2=0x5588bbf2cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf2cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 -- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 msgr2=0x5588bbf295b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 0x5588bbf295b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 -- 172.21.15.66:0/687895706 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588bbf29ba0 con 0x5588bbf1ad10 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f492b1640 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf2c5e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4f400d0d80 tx=0x7f4f40098770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec07fc27505ade2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f2cff9640 1 -- 172.21.15.66:0/687895706 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f400ba060 con 0x5588bbf1ad10 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f2cff9640 1 -- 172.21.15.66:0/687895706 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4f40005210 con 0x5588bbf1ad10 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3c03a040 con 0x5588bbf1ad10 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.431+0000 7f4f2cff9640 1 -- 172.21.15.66:0/687895706 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f400ce1f0 con 0x5588bbf1ad10 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 10 client.16395 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588bbf9b530 con 0x5588bbf1ad10 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f2cff9640 1 -- 172.21.15.66:0/687895706 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c03a040 con 0x5588bbf1ad10 2024-04-21T09:10:06.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f2cff9640 1 -- 172.21.15.66:0/687895706 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f400bb250 con 0x5588bbf1ad10 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 10 client.16395 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 10 client.16395 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 20 client.16395 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf9bc30 0x5588bbf9e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 4 client.16395 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5588bbe87890 con 0x5588bbf9bc30 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf9bc30 0x5588bbf9e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f48ab0640 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf9bc30 0x5588bbf9e050 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b04fcddfcd86831 server_cookie=c37811a771c0e843 in_seq=0 out_seq=0 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.432+0000 7f4f2cff9640 10 client.16395 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f2cff9640 1 -- 172.21.15.66:0/687895706 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x5588bbe87890 con 0x5588bbf9bc30 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f2cff9640 10 client.16395 handle_command_reply: tid=0 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 client.16395 shutdown 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 msgr2=0x5588bbf2c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf2c5e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4f400d0d80 tx=0x7f4f40098770 comp rx=0 tx=0).stop 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 shutdown_connections 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5588bbf9bc30 0x5588bbf9e050 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bbf24240 0x5588bbf295b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bbf1b6c0 0x5588bbf2cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 --2- 172.21.15.66:0/687895706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bbf1ad10 0x5588bbf2c5e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f2cff9640 0 client.16395 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 >> 172.21.15.66:0/687895706 conn(0x5588bbf153e0 msgr2=0x5588bbf167f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 shutdown_connections 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 1 -- 172.21.15.66:0/687895706 wait complete. 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.433+0000 7f4f4e1e1740 20 client.16395 trim_cache size 0 max 16384 2024-04-21T09:10:06.440 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:10:06.441 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 25, 2024-04-21T09:10:06.441 INFO:teuthology.orchestra.run.smithi066.stdout: "leader": 24512, 2024-04-21T09:10:06.441 INFO:teuthology.orchestra.run.smithi066.stdout: "set_version": 0, 2024-04-21T09:10:06.441 INFO:teuthology.orchestra.run.smithi066.stdout: "sets": {} 2024-04-21T09:10:06.441 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f300a3d20 msgr2=0x7f4f300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f300a3d20 0x7f4f300a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0325e0 tx=0x7f4f3c0095c0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f441031e0 msgr2=0x7f4f44100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f441031e0 0x7f4f44100e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c970 tx=0x7f4f4000ce30 comp rx=0 tx=0).stop 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 shutdown_connections 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f300a3d20 0x7f4f300a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4410c520 0x7f4f441018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f44103b00 0x7f4f44101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 --2- 172.21.15.66:0/2625808670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f441031e0 0x7f4f44100e30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 >> 172.21.15.66:0/2625808670 conn(0x7f4f440771d0 msgr2=0x7f4f440fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:06.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 shutdown_connections 2024-04-21T09:10:06.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:06.434+0000 7f4f4b515640 1 -- 172.21.15.66:0/2625808670 wait complete. 2024-04-21T09:10:06.445 INFO:tasks.quiescer:done joining Quiescers 2024-04-21T09:10:06.445 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-21T09:10:06.456 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:10:06.464 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-21T09:10:06.496 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-21T09:10:06.496 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T09:10:06.496 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:10:06.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-21T09:10:06.529 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-21T09:10:07.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:07 smithi186 ceph-mon[31505]: pgmap v60: 129 pgs: 129 active+clean; 260 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-04-21T09:10:07.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:07 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3325664620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:07.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:07 smithi083 ceph-mon[31413]: pgmap v60: 129 pgs: 129 active+clean; 260 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-04-21T09:10:07.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:07 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3325664620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:07 smithi066 ceph-mon[27953]: pgmap v60: 129 pgs: 129 active+clean; 260 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-04-21T09:10:07.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:07 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3325664620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:09.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:09 smithi186 ceph-mon[31505]: pgmap v61: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-04-21T09:10:09.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:09 smithi083 ceph-mon[31413]: pgmap v61: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-04-21T09:10:09.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:09 smithi066 ceph-mon[27953]: pgmap v61: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-04-21T09:10:11.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:11 smithi186 ceph-mon[31505]: pgmap v62: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-21T09:10:11.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:11 smithi083 ceph-mon[31413]: pgmap v62: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-21T09:10:11.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:11 smithi066 ceph-mon[27953]: pgmap v62: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-21T09:10:13.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:13 smithi186 ceph-mon[31505]: pgmap v63: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-21T09:10:13.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:13 smithi083 ceph-mon[31413]: pgmap v63: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-21T09:10:13.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:13 smithi066 ceph-mon[27953]: pgmap v63: 129 pgs: 129 active+clean; 260 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-21T09:10:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:15 smithi186 ceph-mon[31505]: pgmap v64: 129 pgs: 129 active+clean; 259 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 344 op/s 2024-04-21T09:10:15.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:15 smithi083 ceph-mon[31413]: pgmap v64: 129 pgs: 129 active+clean; 259 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 344 op/s 2024-04-21T09:10:15.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:15 smithi066 ceph-mon[27953]: pgmap v64: 129 pgs: 129 active+clean; 259 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 344 op/s 2024-04-21T09:10:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:17 smithi186 ceph-mon[31505]: pgmap v65: 129 pgs: 129 active+clean; 259 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 228 op/s 2024-04-21T09:10:17.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:17 smithi083 ceph-mon[31413]: pgmap v65: 129 pgs: 129 active+clean; 259 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 228 op/s 2024-04-21T09:10:17.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:17 smithi066 ceph-mon[27953]: pgmap v65: 129 pgs: 129 active+clean; 259 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 228 op/s 2024-04-21T09:10:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:19 smithi186 ceph-mon[31505]: pgmap v66: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 460 op/s 2024-04-21T09:10:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:19 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:10:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:19 smithi083 ceph-mon[31413]: pgmap v66: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 460 op/s 2024-04-21T09:10:19.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:19 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:10:19.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:19 smithi066 ceph-mon[27953]: pgmap v66: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 460 op/s 2024-04-21T09:10:19.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:19 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:10:20.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:20 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:10:20.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:20 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:10:20.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:20 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:10:20.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:20 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:10:20.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:20 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:10:20.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:20 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:10:20.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:20 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:10:20.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:20 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:10:20.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:20 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:10:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:21 smithi186 ceph-mon[31505]: pgmap v67: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 367 op/s 2024-04-21T09:10:21.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:21 smithi083 ceph-mon[31413]: pgmap v67: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 367 op/s 2024-04-21T09:10:21.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:21 smithi066 ceph-mon[27953]: pgmap v67: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 367 op/s 2024-04-21T09:10:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:23 smithi186 ceph-mon[31505]: pgmap v68: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 367 op/s 2024-04-21T09:10:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:23 smithi083 ceph-mon[31413]: pgmap v68: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 367 op/s 2024-04-21T09:10:23.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:23 smithi066 ceph-mon[27953]: pgmap v68: 129 pgs: 129 active+clean; 268 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 367 op/s 2024-04-21T09:10:25.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:10:25.118 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:10:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 -- 172.21.15.66:0/3003129469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98107270 msgr2=0x7fec9810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 --2- 172.21.15.66:0/3003129469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98107270 0x7fec9810ee40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fec90009900 tx=0x7fec9002bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 -- 172.21.15.66:0/3003129469 shutdown_connections 2024-04-21T09:10:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 --2- 172.21.15.66:0/3003129469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98107270 0x7fec9810ee40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 --2- 172.21.15.66:0/3003129469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec98102b50 0x7fec98106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 --2- 172.21.15.66:0/3003129469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec981021a0 0x7fec98102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 -- 172.21.15.66:0/3003129469 >> 172.21.15.66:0/3003129469 conn(0x7fec980fda40 msgr2=0x7fec980ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 -- 172.21.15.66:0/3003129469 shutdown_connections 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 -- 172.21.15.66:0/3003129469 wait complete. 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.236+0000 7fec9de1f640 1 Processor -- start 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec9de1f640 1 -- start start 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec9de1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec981021a0 0x7fec98074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec9de1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec98107270 0x7fec98074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec9de1f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98074cd0 0x7fec981a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec9de1f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec981121c0 con 0x7fec98074cd0 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec9de1f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec98112040 con 0x7fec981021a0 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec9de1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec98112340 con 0x7fec98107270 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98074cd0 0x7fec981a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98074cd0 0x7fec981a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58734/0 (socket says 172.21.15.66:58734) 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 -- 172.21.15.66:0/942035673 learned_addr learned my addr 172.21.15.66:0/942035673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec977fe640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec981021a0 0x7fec98074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec96ffd640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec98107270 0x7fec98074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 -- 172.21.15.66:0/942035673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec98107270 msgr2=0x7fec98074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec98107270 0x7fec98074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 -- 172.21.15.66:0/942035673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec981021a0 msgr2=0x7fec98074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec981021a0 0x7fec98074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.237+0000 7fec97fff640 1 -- 172.21.15.66:0/942035673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec981a6f30 con 0x7fec98074cd0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.238+0000 7fec97fff640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98074cd0 0x7fec981a6830 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fec90009230 tx=0x7fec900045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.238+0000 7fec94ff9640 1 -- 172.21.15.66:0/942035673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90043020 con 0x7fec98074cd0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.238+0000 7fec94ff9640 1 -- 172.21.15.66:0/942035673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9002cb70 con 0x7fec98074cd0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.238+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec981a71c0 con 0x7fec98074cd0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.238+0000 7fec94ff9640 1 -- 172.21.15.66:0/942035673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90003ac0 con 0x7fec98074cd0 2024-04-21T09:10:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.238+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a7710 con 0x7fec98074cd0 2024-04-21T09:10:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.240+0000 7fec94ff9640 1 -- 172.21.15.66:0/942035673 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fec90046020 con 0x7fec98074cd0 2024-04-21T09:10:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.240+0000 7feca0a0d740 1 Processor -- start 2024-04-21T09:10:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.240+0000 7feca0a0d740 1 -- start start 2024-04-21T09:10:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b954960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7feca0a0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56253b8b91e0 con 0x56253b954ea0 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7feca0a0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56253b856390 con 0x56253b954580 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7feca0a0d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56253b8b51a0 con 0x56253b95da80 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58740/0 (socket says 172.21.15.66:58740) 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 -- 172.21.15.66:0/2844737690 learned_addr learned my addr 172.21.15.66:0/2844737690 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec94ff9640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec840a3d50 0x7fec840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec96ffd640 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec97fff640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec840a3d50 0x7fec840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec96ffd640 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b954960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 -- 172.21.15.66:0/2844737690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 msgr2=0x56253b95fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 -- 172.21.15.66:0/2844737690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 msgr2=0x56253b954960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b954960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 -- 172.21.15.66:0/2844737690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253b6957d0 con 0x56253b954ea0 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec94ff9640 1 -- 172.21.15.66:0/942035673 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fec900f2c00 con 0x7fec98074cd0 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.241+0000 7fec977fe640 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95d540 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fec88007f90 tx=0x7fec8800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d2c9f64c48f3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7fec97fff640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec840a3d50 0x7fec840a6210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec900be640 tx=0x7fec90090380 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7fec8dffb640 1 -- 172.21.15.66:0/2844737690 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88004d20 con 0x56253b954ea0 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7fec8dffb640 1 -- 172.21.15.66:0/2844737690 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec88010bb0 con 0x56253b954ea0 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7fec8dffb640 1 -- 172.21.15.66:0/2844737690 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec880056c0 con 0x56253b954ea0 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 -- 172.21.15.66:0/2844737690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 msgr2=0x56253b95d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95d540 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fec88007f90 tx=0x7fec8800f850 comp rx=0 tx=0).stop 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 -- 172.21.15.66:0/2844737690 shutdown_connections 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95d540 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 --2- 172.21.15.66:0/2844737690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b954960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 -- 172.21.15.66:0/2844737690 >> 172.21.15.66:0/2844737690 conn(0x56253b94ecb0 msgr2=0x56253b94f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 -- 172.21.15.66:0/2844737690 shutdown_connections 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.242+0000 7feca0a0d740 1 -- 172.21.15.66:0/2844737690 wait complete. 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 -- start start 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 Processor -- start 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b95aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56253b960870 con 0x56253b954ea0 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56253b8722f0 con 0x56253b954580 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56253b9609f0 con 0x56253b95da80 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.243+0000 7feca0a0d740 10 client.0 _ll_register_callbacks cb 0x56253b87f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58744/0 (socket says 172.21.15.66:58744) 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 -- 172.21.15.66:0/3175537164 learned_addr learned my addr 172.21.15.66:0/3175537164 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec96ffd640 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b95aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec96ffd640 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 -- 172.21.15.66:0/3175537164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 msgr2=0x56253b95b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 -- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 msgr2=0x56253b95aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b95aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 -- 172.21.15.66:0/3175537164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253b5cb3f0 con 0x56253b954ea0 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec977fe640 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95af40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fec88007b60 tx=0x7fec88010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9f73d8be63f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:25.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec76ffd640 1 -- 172.21.15.66:0/3175537164 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88026020 con 0x56253b954ea0 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec76ffd640 1 -- 172.21.15.66:0/3175537164 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fec88005740 con 0x56253b954ea0 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec88010bb0 con 0x56253b954ea0 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7fec76ffd640 1 -- 172.21.15.66:0/3175537164 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec88031050 con 0x56253b954ea0 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7feca0a0d740 10 client.16407 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.244+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56253b58b0f0 con 0x56253b954ea0 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7fec76ffd640 1 -- 172.21.15.66:0/3175537164 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec88010bb0 con 0x56253b954ea0 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7fec76ffd640 1 -- 172.21.15.66:0/3175537164 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec88022740 con 0x56253b954ea0 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7feca0a0d740 10 client.16407 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7feca0a0d740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7feca0a0d740 20 client.16407 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7feca0a0d740 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b96d980 0x56253b96fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7feca0a0d740 4 client.16407 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56253b8c6c00 con 0x56253b96d980 2024-04-21T09:10:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7fec96ffd640 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b96d980 0x56253b96fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7fec96ffd640 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b96d980 0x56253b96fd70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54bbe030a232c9c server_cookie=524d15a93c5f5a84 in_seq=0 out_seq=0 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.245+0000 7fec76ffd640 10 client.16407 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7fec76ffd640 1 -- 172.21.15.66:0/3175537164 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56253b8c6c00 con 0x56253b96d980 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7fec76ffd640 10 client.16407 handle_command_reply: tid=0 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 client.16407 shutdown 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 msgr2=0x56253b95af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95af40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fec88007b60 tx=0x7fec88010f90 comp rx=0 tx=0).stop 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 shutdown_connections 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b96d980 0x56253b96fd70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b95b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7fec76ffd640 0 client.16407 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b95af40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 --2- 172.21.15.66:0/3175537164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b95aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.246+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 >> 172.21.15.66:0/3175537164 conn(0x56253b94ecb0 msgr2=0x56253b950a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.247+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 shutdown_connections 2024-04-21T09:10:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.247+0000 7feca0a0d740 1 -- 172.21.15.66:0/3175537164 wait complete. 2024-04-21T09:10:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.247+0000 7feca0a0d740 20 client.16407 trim_cache size 0 max 16384 2024-04-21T09:10:25.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 Processor -- start 2024-04-21T09:10:25.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 -- start start 2024-04-21T09:10:25.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954580 0x56253b95d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b954ea0 0x56253b969760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b95da80 0x56253b95b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b3d60 con 0x56253b954580 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b1630 con 0x56253b95da80 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7feca0a0d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec981af460 con 0x56253b954ea0 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.264+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954580 0x56253b95d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954580 0x56253b95d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58758/0 (socket says 172.21.15.66:58758) 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 -- 172.21.15.66:0/1363359053 learned_addr learned my addr 172.21.15.66:0/1363359053 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b95da80 0x56253b95b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec977fe640 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b954ea0 0x56253b969760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 -- 172.21.15.66:0/1363359053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b954ea0 msgr2=0x56253b969760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b954ea0 0x56253b969760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 -- 172.21.15.66:0/1363359053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b95da80 msgr2=0x56253b95b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b95da80 0x56253b95b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 -- 172.21.15.66:0/1363359053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253b94eae0 con 0x56253b954580 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec96ffd640 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954580 0x56253b95d130 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fec80005fd0 tx=0x7fec800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff59d716490c8811 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec8d7fa640 1 -- 172.21.15.66:0/1363359053 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80029070 con 0x56253b954580 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec8d7fa640 1 -- 172.21.15.66:0/1363359053 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec80011c20 con 0x56253b954580 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.265+0000 7fec8d7fa640 1 -- 172.21.15.66:0/1363359053 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80005620 con 0x56253b954580 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 -- 172.21.15.66:0/1363359053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954580 msgr2=0x56253b95d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954580 0x56253b95d130 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fec80005fd0 tx=0x7fec800046c0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 -- 172.21.15.66:0/1363359053 shutdown_connections 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b95da80 0x56253b95b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b954ea0 0x56253b969760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 --2- 172.21.15.66:0/1363359053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954580 0x56253b95d130 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 -- 172.21.15.66:0/1363359053 >> 172.21.15.66:0/1363359053 conn(0x56253b94ecb0 msgr2=0x56253b95a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 -- 172.21.15.66:0/1363359053 shutdown_connections 2024-04-21T09:10:25.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 -- 172.21.15.66:0/1363359053 wait complete. 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 -- start start 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.266+0000 7feca0a0d740 1 Processor -- start 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b957770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b957cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b9581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec981afaf0 con 0x56253b954ea0 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b0f60 con 0x56253b954580 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b01c0 con 0x56253b95da80 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 10 client.0 _ll_register_callbacks cb 0x7fec8000ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b957cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b957cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58772/0 (socket says 172.21.15.66:58772) 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 -- 172.21.15.66:0/376884568 learned_addr learned my addr 172.21.15.66:0/376884568 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec96ffd640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b957770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec96ffd640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b9581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 -- 172.21.15.66:0/376884568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 msgr2=0x56253b9581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b9581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 -- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 msgr2=0x56253b957770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b957770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 -- 172.21.15.66:0/376884568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253b94e4f0 con 0x56253b954ea0 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec96ffd640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b9581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec977fe640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b957cb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fec88000f80 tx=0x7fec88022da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d094d1d27266c4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec76ffd640 1 -- 172.21.15.66:0/376884568 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8803a060 con 0x56253b954ea0 2024-04-21T09:10:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7fec76ffd640 1 -- 172.21.15.66:0/376884568 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fec88010a80 con 0x56253b954ea0 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.267+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec80011c20 con 0x56253b954ea0 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7fec76ffd640 1 -- 172.21.15.66:0/376884568 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec88041050 con 0x56253b954ea0 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 10 client.16413 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56253b95fe70 con 0x56253b954ea0 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7fec76ffd640 1 -- 172.21.15.66:0/376884568 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec80011c20 con 0x56253b954ea0 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7fec76ffd640 1 -- 172.21.15.66:0/376884568 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec880185c0 con 0x56253b954ea0 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 10 client.16413 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 20 client.16413 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b9d4410 0x56253b9d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 4 client.16413 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56253b8c0a70 con 0x56253b9d4410 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.268+0000 7fec96ffd640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b9d4410 0x56253b9d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7fec76ffd640 10 client.16413 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7fec96ffd640 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b9d4410 0x56253b9d6830 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ef3998199fc1fc server_cookie=59af01f5fa98d5b3 in_seq=0 out_seq=0 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7fec76ffd640 1 -- 172.21.15.66:0/376884568 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56253b8c0a70 con 0x56253b9d4410 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7fec76ffd640 10 client.16413 handle_command_reply: tid=0 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 client.16413 shutdown 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 msgr2=0x56253b957cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b957cb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fec88000f80 tx=0x7fec88022da0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 shutdown_connections 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253b9d4410 0x56253b9d6830 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253b95da80 0x56253b9581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253b954ea0 0x56253b957cb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 --2- 172.21.15.66:0/376884568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253b954580 0x56253b957770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7fec76ffd640 0 client.16413 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 >> 172.21.15.66:0/376884568 conn(0x56253b94ecb0 msgr2=0x56253b94dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 shutdown_connections 2024-04-21T09:10:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 1 -- 172.21.15.66:0/376884568 wait complete. 2024-04-21T09:10:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.269+0000 7feca0a0d740 20 client.16413 trim_cache size 0 max 16384 2024-04-21T09:10:25.277 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:10:25.278 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:10:25.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:10:25.278 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec840a3d50 msgr2=0x7fec840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec840a3d50 0x7fec840a6210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec900be640 tx=0x7fec90090380 comp rx=0 tx=0).stop 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98074cd0 msgr2=0x7fec981a6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98074cd0 0x7fec981a6830 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fec90009230 tx=0x7fec900045e0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 shutdown_connections 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec840a3d50 0x7fec840a6210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec98074cd0 0x7fec981a6830 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec98107270 0x7fec98074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 --2- 172.21.15.66:0/942035673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec981021a0 0x7fec98074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 >> 172.21.15.66:0/942035673 conn(0x7fec980fda40 msgr2=0x7fec980ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 shutdown_connections 2024-04-21T09:10:25.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.270+0000 7fec9de1f640 1 -- 172.21.15.66:0/942035673 wait complete. 2024-04-21T09:10:25.283 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:10:25.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.384+0000 7f1aca19f640 1 -- 172.21.15.66:0/1883253948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac40690e0 msgr2=0x7f1ac4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.384+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1883253948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac40690e0 0x7f1ac4069560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009900 tx=0x7f1ab802bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.384+0000 7f1aca19f640 1 -- 172.21.15.66:0/1883253948 shutdown_connections 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.384+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1883253948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac4069aa0 0x7f1ac410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1883253948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac40690e0 0x7f1ac4069560 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1883253948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac4106f50 0x7f1ac4107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 -- 172.21.15.66:0/1883253948 >> 172.21.15.66:0/1883253948 conn(0x7f1ac4077e30 msgr2=0x7f1ac4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 -- 172.21.15.66:0/1883253948 shutdown_connections 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 -- 172.21.15.66:0/1883253948 wait complete. 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 Processor -- start 2024-04-21T09:10:25.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 -- start start 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac40690e0 0x7f1ac4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac4069aa0 0x7f1ac4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac4106f50 0x7f1ac419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4112170 con 0x7f1ac4106f50 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4111ff0 con 0x7f1ac40690e0 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.385+0000 7f1aca19f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac41122f0 con 0x7f1ac4069aa0 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac4106f50 0x7f1ac419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac4106f50 0x7f1ac419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58790/0 (socket says 172.21.15.66:58790) 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 -- 172.21.15.66:0/1331389221 learned_addr learned my addr 172.21.15.66:0/1331389221 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac37fe640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac40690e0 0x7f1ac4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac2ffd640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac4069aa0 0x7f1ac4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 -- 172.21.15.66:0/1331389221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac4069aa0 msgr2=0x7f1ac4198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac4069aa0 0x7f1ac4198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 -- 172.21.15.66:0/1331389221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac40690e0 msgr2=0x7f1ac4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac40690e0 0x7f1ac4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 -- 172.21.15.66:0/1331389221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac419d640 con 0x7f1ac4106f50 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac3fff640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac4106f50 0x7f1ac419cf10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00d750 tx=0x7f1abc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac0ff9640 1 -- 172.21.15.66:0/1331389221 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc017070 con 0x7f1ac4106f50 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac0ff9640 1 -- 172.21.15.66:0/1331389221 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc0054d0 con 0x7f1ac4106f50 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac419d930 con 0x7f1ac4106f50 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1ac0ff9640 1 -- 172.21.15.66:0/1331389221 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc004c60 con 0x7f1ac4106f50 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.386+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41a51c0 con 0x7f1ac4106f50 2024-04-21T09:10:25.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.387+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac4068de0 con 0x7f1ac4106f50 2024-04-21T09:10:25.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.389+0000 7f1ac0ff9640 1 -- 172.21.15.66:0/1331389221 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1abc009bd0 con 0x7f1ac4106f50 2024-04-21T09:10:25.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.389+0000 7f1ac0ff9640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.389+0000 7f1ac37fe640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.389+0000 7f1ac0ff9640 1 -- 172.21.15.66:0/1331389221 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1abc0d08e0 con 0x7f1ac4106f50 2024-04-21T09:10:25.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.390+0000 7f1ac37fe640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005fd0 tx=0x7f1ab4009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.393+0000 7f1ac0ff9640 1 -- 172.21.15.66:0/1331389221 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1abc09cc10 con 0x7f1ac4106f50 2024-04-21T09:10:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:25 smithi186 ceph-mon[31505]: pgmap v69: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 21 MiB/s wr, 376 op/s 2024-04-21T09:10:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:25 smithi083 ceph-mon[31413]: pgmap v69: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 21 MiB/s wr, 376 op/s 2024-04-21T09:10:25.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:25 smithi066 ceph-mon[27953]: pgmap v69: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 21 MiB/s wr, 376 op/s 2024-04-21T09:10:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.620+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ac402d5c0 con 0x7f1ac4106f50 2024-04-21T09:10:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.621+0000 7f1ac0ff9640 1 -- 172.21.15.66:0/1331389221 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f1abc09c510 con 0x7f1ac4106f50 2024-04-21T09:10:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:10:25.622 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:10:25.622 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:10:25.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.622+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1aac0a3d20 msgr2=0x7f1aac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.622+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005fd0 tx=0x7f1ab4009500 comp rx=0 tx=0).stop 2024-04-21T09:10:25.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.622+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac4106f50 msgr2=0x7f1ac419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.622+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac4106f50 0x7f1ac419cf10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00d750 tx=0x7f1abc00dc10 comp rx=0 tx=0).stop 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 shutdown_connections 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac4106f50 0x7f1ac419cf10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac4069aa0 0x7f1ac4198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 --2- 172.21.15.66:0/1331389221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac40690e0 0x7f1ac4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 >> 172.21.15.66:0/1331389221 conn(0x7f1ac4077e30 msgr2=0x7f1ac40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 shutdown_connections 2024-04-21T09:10:25.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.623+0000 7f1aca19f640 1 -- 172.21.15.66:0/1331389221 wait complete. 2024-04-21T09:10:25.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:10:25.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 -- 172.21.15.66:0/2808347033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006b1f0 msgr2=0x7fcd90106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 --2- 172.21.15.66:0/2808347033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006b1f0 0x7fcd90106b90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009900 tx=0x7fcd8402bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:25.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 -- 172.21.15.66:0/2808347033 shutdown_connections 2024-04-21T09:10:25.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 --2- 172.21.15.66:0/2808347033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd90107270 0x7fcd9010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 --2- 172.21.15.66:0/2808347033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006b1f0 0x7fcd90106b90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 --2- 172.21.15.66:0/2808347033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd9006a840 0x7fcd9006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 -- 172.21.15.66:0/2808347033 >> 172.21.15.66:0/2808347033 conn(0x7fcd90077eb0 msgr2=0x7fcd900782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 -- 172.21.15.66:0/2808347033 shutdown_connections 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.738+0000 7fcd97d52640 1 -- 172.21.15.66:0/2808347033 wait complete. 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 Processor -- start 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 -- start start 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006a840 0x7fcd9019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd90107270 0x7fcd9019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd901985d0 0x7fcd90198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112210 con 0x7fcd9006a840 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112090 con 0x7fcd901985d0 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd97d52640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112390 con 0x7fcd90107270 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd95aee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006a840 0x7fcd9019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd95aee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006a840 0x7fcd9019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58802/0 (socket says 172.21.15.66:58802) 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd95aee640 1 -- 172.21.15.66:0/368934852 learned_addr learned my addr 172.21.15.66:0/368934852 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.739+0000 7fcd962ef640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd901985d0 0x7fcd90198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd95aee640 1 -- 172.21.15.66:0/368934852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd90107270 msgr2=0x7fcd9019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd95aee640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd90107270 0x7fcd9019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd95aee640 1 -- 172.21.15.66:0/368934852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd901985d0 msgr2=0x7fcd90198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd95aee640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd901985d0 0x7fcd90198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd95aee640 1 -- 172.21.15.66:0/368934852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd901990f0 con 0x7fcd9006a840 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd95aee640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006a840 0x7fcd9019e420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800ba50 tx=0x7fcd8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd8effd640 1 -- 172.21.15.66:0/368934852 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88019070 con 0x7fcd9006a840 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd8effd640 1 -- 172.21.15.66:0/368934852 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd8800cbe0 con 0x7fcd9006a840 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd901993e0 con 0x7fcd9006a840 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd8effd640 1 -- 172.21.15.66:0/368934852 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880075d0 con 0x7fcd9006a840 2024-04-21T09:10:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.740+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901a5630 con 0x7fcd9006a840 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.742+0000 7fcd8effd640 1 -- 172.21.15.66:0/368934852 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fcd8801c070 con 0x7fcd9006a840 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.742+0000 7fcd9aa27740 1 Processor -- start 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd9aa27740 1 -- start start 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9192131f0 con 0x55a9192ae3b0 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9191b04b0 con 0x55a9192aed60 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a91920f1b0 con 0x55a9192b7e60 2024-04-21T09:10:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58806/0 (socket says 172.21.15.66:58806) 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 -- 172.21.15.66:0/4071040068 learned_addr learned my addr 172.21.15.66:0/4071040068 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd8effd640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcd780a3d20 0x7fcd780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd95aee640 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd952ed640 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 -- 172.21.15.66:0/4071040068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 msgr2=0x55a9192b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 -- 172.21.15.66:0/4071040068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 msgr2=0x55a9192b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd962ef640 1 -- 172.21.15.66:0/4071040068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9191a21b0 con 0x55a9192ae3b0 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.743+0000 7fcd8effd640 1 -- 172.21.15.66:0/368934852 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fcd880d00f0 con 0x7fcd9006a840 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.744+0000 7fcd962ef640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcd780a3d20 0x7fcd780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.744+0000 7fcd962ef640 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192ae790 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000cd60 tx=0x7fcd8000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4594b3aeb4ef89e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.744+0000 7fcd77fff640 1 -- 172.21.15.66:0/4071040068 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80022070 con 0x55a9192ae3b0 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.744+0000 7fcd77fff640 1 -- 172.21.15.66:0/4071040068 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd80018070 con 0x55a9192ae3b0 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.744+0000 7fcd77fff640 1 -- 172.21.15.66:0/4071040068 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80005670 con 0x55a9192ae3b0 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 -- 172.21.15.66:0/4071040068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 msgr2=0x55a9192ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd962ef640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcd780a3d20 0x7fcd780a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd8001f000 tx=0x7fcd80020000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192ae790 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000cd60 tx=0x7fcd8000e720 comp rx=0 tx=0).stop 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 -- 172.21.15.66:0/4071040068 shutdown_connections 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/4071040068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192ae790 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 -- 172.21.15.66:0/4071040068 >> 172.21.15.66:0/4071040068 conn(0x55a9192a8ba0 msgr2=0x55a9192a8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 -- 172.21.15.66:0/4071040068 shutdown_connections 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 -- 172.21.15.66:0/4071040068 wait complete. 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 -- start start 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.745+0000 7fcd9aa27740 1 Processor -- start 2024-04-21T09:10:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9192beda0 con 0x55a9192aed60 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9191cc500 con 0x55a9192ae3b0 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a9192bef20 con 0x55a9192b7e60 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 10 client.0 _ll_register_callbacks cb 0x55a91919ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58822/0 (socket says 172.21.15.66:58822) 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 -- 172.21.15.66:0/2491821548 learned_addr learned my addr 172.21.15.66:0/2491821548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd962ef640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 -- 172.21.15.66:0/2491821548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 msgr2=0x55a9192bbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 -- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 msgr2=0x55a9192be270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192be270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 -- 172.21.15.66:0/2491821548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a918f253f0 con 0x55a9192aed60 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd962ef640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bb600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcd9006a420 tx=0x7fcd8402c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a531585f807f20be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd757fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd757fa640 1 -- 172.21.15.66:0/2491821548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84042070 con 0x55a9192aed60 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd80018070 con 0x55a9192aed60 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.746+0000 7fcd757fa640 1 -- 172.21.15.66:0/2491821548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fcd84005240 con 0x55a9192aed60 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd757fa640 1 -- 172.21.15.66:0/2491821548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd8404e050 con 0x55a9192aed60 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 10 client.16431 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a918fef7d0 con 0x55a9192aed60 2024-04-21T09:10:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd757fa640 1 -- 172.21.15.66:0/2491821548 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd80018070 con 0x55a9192aed60 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd757fa640 1 -- 172.21.15.66:0/2491821548 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd840035f0 con 0x55a9192aed60 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 20 client.16431 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a9192cbfc0 0x55a9192ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 4 client.16431 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9191b2930 con 0x55a9192cbfc0 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.747+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a9192cbfc0 0x55a9192ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.748+0000 7fcd757fa640 10 client.16431 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.748+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a9192cbfc0 0x55a9192ce420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a28d56e350702fe server_cookie=fefa2393b486179 in_seq=0 out_seq=0 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd757fa640 1 -- 172.21.15.66:0/2491821548 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9191b2930 con 0x55a9192cbfc0 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd757fa640 10 client.16431 handle_command_reply: tid=0 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 client.16431 shutdown 2024-04-21T09:10:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 msgr2=0x55a9192bb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bb600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcd9006a420 tx=0x7fcd8402c9f0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 shutdown_connections 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a9192cbfc0 0x55a9192ce420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bb600 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2491821548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 >> 172.21.15.66:0/2491821548 conn(0x55a9192a8ba0 msgr2=0x55a9192aa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd757fa640 0 client.16431 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 shutdown_connections 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2491821548 wait complete. 2024-04-21T09:10:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.749+0000 7fcd9aa27740 20 client.16431 trim_cache size 0 max 16384 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 Processor -- start 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 -- start start 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901af950 con 0x55a9192ae3b0 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901ad220 con 0x55a9192aed60 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.762+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901ab050 con 0x55a9192b7e60 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58838/0 (socket says 172.21.15.66:58838) 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 -- 172.21.15.66:0/2780962594 learned_addr learned my addr 172.21.15.66:0/2780962594 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd962ef640 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 -- 172.21.15.66:0/2780962594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 msgr2=0x55a9192a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 -- 172.21.15.66:0/2780962594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 msgr2=0x55a9192b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 -- 172.21.15.66:0/2780962594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9192a86d0 con 0x55a9192ae3b0 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd962ef640 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192c7ca0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcd88098800 tx=0x7fcd880c4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0e73abeb602bd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd777fe640 1 -- 172.21.15.66:0/2780962594 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8809a130 con 0x55a9192ae3b0 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd777fe640 1 -- 172.21.15.66:0/2780962594 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd880984a0 con 0x55a9192ae3b0 2024-04-21T09:10:25.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.763+0000 7fcd777fe640 1 -- 172.21.15.66:0/2780962594 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880a4870 con 0x55a9192ae3b0 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2780962594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 msgr2=0x55a9192c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192c7ca0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcd88098800 tx=0x7fcd880c4040 comp rx=0 tx=0).stop 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2780962594 shutdown_connections 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192aed60 0x55a9192b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2780962594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192ae3b0 0x55a9192c7ca0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2780962594 >> 172.21.15.66:0/2780962594 conn(0x55a9192a8ba0 msgr2=0x55a9192a7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2780962594 shutdown_connections 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2780962594 wait complete. 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 -- start start 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.764+0000 7fcd9aa27740 1 Processor -- start 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901ab6e0 con 0x55a9192aed60 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901acb50 con 0x55a9192ae3b0 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901abdb0 con 0x55a9192b7e60 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 10 client.0 _ll_register_callbacks cb 0x7fcd84004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58852/0 (socket says 172.21.15.66:58852) 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 -- 172.21.15.66:0/2878690520 learned_addr learned my addr 172.21.15.66:0/2878690520 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 -- 172.21.15.66:0/2878690520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 msgr2=0x55a9192bdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 -- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 msgr2=0x55a9192bd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192bd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 -- 172.21.15.66:0/2878690520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a919331560 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd952ed640 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bd7f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcd84001eb0 tx=0x7fcd84003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac526cb6fd77cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd757fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd757fa640 1 -- 172.21.15.66:0/2878690520 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84057060 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd880984a0 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd757fa640 1 -- 172.21.15.66:0/2878690520 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fcd84005240 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd757fa640 1 -- 172.21.15.66:0/2878690520 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd8405e050 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 10 client.16437 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.765+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a919332810 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd757fa640 1 -- 172.21.15.66:0/2878690520 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd880984a0 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd757fa640 1 -- 172.21.15.66:0/2878690520 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd84004b70 con 0x55a9192aed60 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd9aa27740 10 client.16437 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:25.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd9aa27740 10 client.16437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd9aa27740 20 client.16437 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a919332fc0 0x55a9193353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd9aa27740 4 client.16437 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9192bbed0 con 0x55a919332fc0 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a919332fc0 0x55a9193353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd95aee640 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a919332fc0 0x55a9193353e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b1e2196c1e8654e server_cookie=16443c86dab2ec84 in_seq=0 out_seq=0 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.766+0000 7fcd757fa640 10 client.16437 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd757fa640 1 -- 172.21.15.66:0/2878690520 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9192bbed0 con 0x55a919332fc0 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd757fa640 10 client.16437 handle_command_reply: tid=0 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 client.16437 shutdown 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 msgr2=0x55a9192bd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bd7f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcd84001eb0 tx=0x7fcd84003800 comp rx=0 tx=0).stop 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 shutdown_connections 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a919332fc0 0x55a9193353e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9192b7e60 0x55a9192bdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9192aed60 0x55a9192bd7f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 --2- 172.21.15.66:0/2878690520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9192ae3b0 0x55a9192bd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd757fa640 0 client.16437 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 >> 172.21.15.66:0/2878690520 conn(0x55a9192a8ba0 msgr2=0x55a9192a7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 shutdown_connections 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 1 -- 172.21.15.66:0/2878690520 wait complete. 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd9aa27740 20 client.16437 trim_cache size 0 max 16384 2024-04-21T09:10:25.774 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcd780a3d20 msgr2=0x7fcd780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcd780a3d20 0x7fcd780a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd8001f000 tx=0x7fcd80020000 comp rx=0 tx=0).stop 2024-04-21T09:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006a840 msgr2=0x7fcd9019e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006a840 0x7fcd9019e420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800ba50 tx=0x7fcd8800bf10 comp rx=0 tx=0).stop 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 shutdown_connections 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcd780a3d20 0x7fcd780a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd901985d0 0x7fcd90198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd90107270 0x7fcd9019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.767+0000 7fcd97d52640 1 --2- 172.21.15.66:0/368934852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd9006a840 0x7fcd9019e420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.768+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 >> 172.21.15.66:0/368934852 conn(0x7fcd90077eb0 msgr2=0x7fcd90100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.768+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 shutdown_connections 2024-04-21T09:10:25.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.768+0000 7fcd97d52640 1 -- 172.21.15.66:0/368934852 wait complete. 2024-04-21T09:10:25.779 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 -- 172.21.15.66:0/1349465145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09041010a0 msgr2=0x7f0904101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 --2- 172.21.15.66:0/1349465145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09041010a0 0x7f0904101480 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0900009900 tx=0x7f090002bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 -- 172.21.15.66:0/1349465145 shutdown_connections 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 --2- 172.21.15.66:0/1349465145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0904106170 0x7f090410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 --2- 172.21.15.66:0/1349465145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0904101a50 0x7f0904105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 --2- 172.21.15.66:0/1349465145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09041010a0 0x7f0904101480 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 -- 172.21.15.66:0/1349465145 >> 172.21.15.66:0/1349465145 conn(0x7f0904077eb0 msgr2=0x7f09040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 -- 172.21.15.66:0/1349465145 shutdown_connections 2024-04-21T09:10:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.882+0000 7f090b6f1640 1 -- 172.21.15.66:0/1349465145 wait complete. 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 Processor -- start 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 -- start start 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09041010a0 0x7f0904198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0904106170 0x7f0904198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f090419cfd0 0x7f090419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09041121f0 con 0x7f090419cfd0 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0904112070 con 0x7f09041010a0 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090b6f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0904112370 con 0x7f0904106170 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f090419cfd0 0x7f090419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f090419cfd0 0x7f090419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58874/0 (socket says 172.21.15.66:58874) 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 -- 172.21.15.66:0/3060642553 learned_addr learned my addr 172.21.15.66:0/3060642553 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f090948d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09041010a0 0x7f0904198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0904106170 0x7f0904198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 -- 172.21.15.66:0/3060642553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0904106170 msgr2=0x7f0904198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0904106170 0x7f0904198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 -- 172.21.15.66:0/3060642553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09041010a0 msgr2=0x7f0904198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09041010a0 0x7f0904198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.883+0000 7f0909c8e640 1 -- 172.21.15.66:0/3060642553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090419dac0 con 0x7f090419cfd0 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.884+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0904106170 0x7f0904198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.884+0000 7f0909c8e640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f090419cfd0 0x7f090419d480 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009fd0 tx=0x7f08fc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.884+0000 7f08fa7fc640 1 -- 172.21.15.66:0/3060642553 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc017070 con 0x7f090419cfd0 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.884+0000 7f08fa7fc640 1 -- 172.21.15.66:0/3060642553 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08fc003e20 con 0x7f090419cfd0 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.884+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090419ddb0 con 0x7f090419cfd0 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.884+0000 7f08fa7fc640 1 -- 172.21.15.66:0/3060642553 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc010460 con 0x7f090419cfd0 2024-04-21T09:10:25.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.884+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09041a5550 con 0x7f090419cfd0 2024-04-21T09:10:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.886+0000 7f08fa7fc640 1 -- 172.21.15.66:0/3060642553 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f08fc003970 con 0x7f090419cfd0 2024-04-21T09:10:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 Processor -- start 2024-04-21T09:10:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 -- start start 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab4833b0 0x5622ab483790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab48d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5622ab3e81f0 con 0x5622ab4833b0 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622ab3854b0 con 0x5622ab48ce60 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090e3b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5622ab3e41b0 con 0x5622ab483d60 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f08fa7fc640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08f00a3d20 0x7f08f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab4833b0 0x5622ab483790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab4833b0 0x5622ab483790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58890/0 (socket says 172.21.15.66:58890) 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 -- 172.21.15.66:0/3506137947 learned_addr learned my addr 172.21.15.66:0/3506137947 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0909c8e640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08f00a3d20 0x7f08f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f090948d640 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab48d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f08fa7fc640 1 -- 172.21.15.66:0/3060642553 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f08fc0d08b0 con 0x7f090419cfd0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 -- 172.21.15.66:0/3506137947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 msgr2=0x5622ab48c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 -- 172.21.15.66:0/3506137947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 msgr2=0x5622ab48d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab48d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.887+0000 7f0908c8c640 1 -- 172.21.15.66:0/3506137947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ab3771b0 con 0x5622ab4833b0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab4833b0 0x5622ab483790 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f08f40155a0 tx=0x7f08f40095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38d75ae4a201be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f0909c8e640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08f00a3d20 0x7f08f00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08fc098940 tx=0x7f08fc004020 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f08ef7fe640 1 -- 172.21.15.66:0/3506137947 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40150f0 con 0x5622ab4833b0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f08ef7fe640 1 -- 172.21.15.66:0/3506137947 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f4005020 con 0x5622ab4833b0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f08ef7fe640 1 -- 172.21.15.66:0/3506137947 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004a00 con 0x5622ab4833b0 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f090e3b6740 1 -- 172.21.15.66:0/3506137947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab4833b0 msgr2=0x5622ab483790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab4833b0 0x5622ab483790 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f08f40155a0 tx=0x7f08f40095d0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f090e3b6740 1 -- 172.21.15.66:0/3506137947 shutdown_connections 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.888+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab48d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3506137947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab4833b0 0x5622ab483790 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 -- 172.21.15.66:0/3506137947 >> 172.21.15.66:0/3506137947 conn(0x5622ab47dba0 msgr2=0x5622ab47df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 -- 172.21.15.66:0/3506137947 shutdown_connections 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 -- 172.21.15.66:0/3506137947 wait complete. 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 -- start start 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 Processor -- start 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab493200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab490600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab48ce60 0x5622ab490b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5622ab493da0 con 0x5622ab483d60 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622ab3a1500 con 0x5622ab4833b0 2024-04-21T09:10:25.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5622ab493f20 con 0x5622ab48ce60 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090e3b6740 10 client.0 _ll_register_callbacks cb 0x5622ab36fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.889+0000 7f090948d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab490600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab490600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58904/0 (socket says 172.21.15.66:58904) 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 -- 172.21.15.66:0/1271217322 learned_addr learned my addr 172.21.15.66:0/1271217322 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f0908c8c640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab493200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f0908c8c640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab48ce60 0x5622ab490b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 -- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab48ce60 msgr2=0x5622ab490b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab48ce60 0x5622ab490b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 -- 172.21.15.66:0/1271217322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 msgr2=0x5622ab493200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab493200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 -- 172.21.15.66:0/1271217322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ab0fa3f0 con 0x5622ab483d60 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f0908c8c640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab48ce60 0x5622ab490b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:25.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090948d640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab490600 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0900009900 tx=0x7f090002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd592258989356f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f08ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f08ecff9640 1 -- 172.21.15.66:0/1271217322 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900005030 con 0x5622ab483d60 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f08ecff9640 1 -- 172.21.15.66:0/1271217322 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f09000051d0 con 0x5622ab483d60 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f4005020 con 0x5622ab483d60 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f08ecff9640 1 -- 172.21.15.66:0/1271217322 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0900052050 con 0x5622ab483d60 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090e3b6740 10 client.16449 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.890+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622ab1c47d0 con 0x5622ab483d60 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f08ecff9640 1 -- 172.21.15.66:0/1271217322 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f4005020 con 0x5622ab483d60 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f08ecff9640 1 -- 172.21.15.66:0/1271217322 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f09000035f0 con 0x5622ab483d60 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f090e3b6740 10 client.16449 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f090e3b6740 10 client.16449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f090e3b6740 20 client.16449 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f090e3b6740 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab4a1050 0x5622ab4a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f090e3b6740 4 client.16449 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622ab387930 con 0x5622ab4a1050 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.891+0000 7f0908c8c640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab4a1050 0x5622ab4a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f0908c8c640 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab4a1050 0x5622ab4a34e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a44b39457c0fd44e server_cookie=422027e791b849d8 in_seq=0 out_seq=0 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f08ecff9640 10 client.16449 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f08ecff9640 1 -- 172.21.15.66:0/1271217322 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622ab387930 con 0x5622ab4a1050 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f08ecff9640 10 client.16449 handle_command_reply: tid=0 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 client.16449 shutdown 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 msgr2=0x5622ab490600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab490600 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0900009900 tx=0x7f090002c9f0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 shutdown_connections 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab48ce60 0x5622ab490b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab4a1050 0x5622ab4a34e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab490600 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 --2- 172.21.15.66:0/1271217322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab493200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 >> 172.21.15.66:0/1271217322 conn(0x5622ab47dba0 msgr2=0x5622ab47f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.892+0000 7f08ecff9640 0 client.16449 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:25.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.893+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 shutdown_connections 2024-04-21T09:10:25.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.893+0000 7f090e3b6740 1 -- 172.21.15.66:0/1271217322 wait complete. 2024-04-21T09:10:25.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.893+0000 7f090e3b6740 20 client.16449 trim_cache size 0 max 16384 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 Processor -- start 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 -- start start 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab4833b0 0x5622ab49cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab48ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab4a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09041af8a0 con 0x5622ab483d60 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ad170 con 0x5622ab48ce60 2024-04-21T09:10:25.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090e3b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f09041aafa0 con 0x5622ab4833b0 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090948d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab48ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090948d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab48ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58920/0 (socket says 172.21.15.66:58920) 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090948d640 1 -- 172.21.15.66:0/3417167874 learned_addr learned my addr 172.21.15.66:0/3417167874 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab4a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090948d640 1 -- 172.21.15.66:0/3417167874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab4833b0 msgr2=0x5622ab49cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.906+0000 7f090948d640 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab4833b0 0x5622ab49cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090948d640 1 -- 172.21.15.66:0/3417167874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 msgr2=0x5622ab4a8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090948d640 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab4a8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090948d640 1 -- 172.21.15.66:0/3417167874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ab482fe0 con 0x5622ab483d60 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090948d640 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab48ff00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f090002c170 tx=0x7f090002cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426844158cfbf96e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f08eeffd640 1 -- 172.21.15.66:0/3417167874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900003c30 con 0x5622ab483d60 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f08eeffd640 1 -- 172.21.15.66:0/3417167874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f090005b070 con 0x5622ab483d60 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f08eeffd640 1 -- 172.21.15.66:0/3417167874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900005000 con 0x5622ab483d60 2024-04-21T09:10:25.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090e3b6740 1 -- 172.21.15.66:0/3417167874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 msgr2=0x5622ab48ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab48ff00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f090002c170 tx=0x7f090002cd70 comp rx=0 tx=0).stop 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090e3b6740 1 -- 172.21.15.66:0/3417167874 shutdown_connections 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.907+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab48ce60 0x5622ab4a8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab483d60 0x5622ab48ff00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3417167874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab4833b0 0x5622ab49cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 -- 172.21.15.66:0/3417167874 >> 172.21.15.66:0/3417167874 conn(0x5622ab47dba0 msgr2=0x5622ab47f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 -- 172.21.15.66:0/3417167874 shutdown_connections 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 -- 172.21.15.66:0/3417167874 wait complete. 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 -- start start 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 Processor -- start 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab48d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab48ce60 0x5622ab48e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ab630 con 0x5622ab48ce60 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09041acaa0 con 0x5622ab4833b0 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f09041abd00 con 0x5622ab483d60 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f090e3b6740 10 client.0 _ll_register_callbacks cb 0x7f0900004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.908+0000 7f0908c8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab48ce60 0x5622ab48e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab48ce60 0x5622ab48e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58924/0 (socket says 172.21.15.66:58924) 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 -- 172.21.15.66:0/3019072997 learned_addr learned my addr 172.21.15.66:0/3019072997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f090948d640 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab48d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 -- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 msgr2=0x5622ab48dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 -- 172.21.15.66:0/3019072997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 msgr2=0x5622ab48d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab48d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 -- 172.21.15.66:0/3019072997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ab506620 con 0x5622ab48ce60 2024-04-21T09:10:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f0908c8c640 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab48ce60 0x5622ab48e280 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f08f40153f0 tx=0x7f08f4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3f994ab3d422cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f08ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f08ecff9640 1 -- 172.21.15.66:0/3019072997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40134c0 con 0x5622ab48ce60 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f08ecff9640 1 -- 172.21.15.66:0/3019072997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f08f4013660 con 0x5622ab48ce60 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f090005b070 con 0x5622ab48ce60 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f08ecff9640 1 -- 172.21.15.66:0/3019072997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f08f4039050 con 0x5622ab48ce60 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f090e3b6740 10 client.16455 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622ab5078d0 con 0x5622ab48ce60 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.909+0000 7f08ecff9640 1 -- 172.21.15.66:0/3019072997 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f090005b070 con 0x5622ab48ce60 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f08ecff9640 1 -- 172.21.15.66:0/3019072997 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f08f4021040 con 0x5622ab48ce60 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090e3b6740 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090e3b6740 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090e3b6740 20 client.16455 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab507cc0 0x5622ab50a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090e3b6740 4 client.16455 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622ab3f5c10 con 0x5622ab507cc0 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090948d640 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab507cc0 0x5622ab50a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f08ecff9640 10 client.16455 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.910+0000 7f090948d640 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab507cc0 0x5622ab50a0e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e488929e728539df server_cookie=31f4381fe9514fca in_seq=0 out_seq=0 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f08ecff9640 1 -- 172.21.15.66:0/3019072997 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622ab3f5c10 con 0x5622ab507cc0 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f08ecff9640 10 client.16455 handle_command_reply: tid=0 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 client.16455 shutdown 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab48ce60 msgr2=0x5622ab48e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab48ce60 0x5622ab48e280 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f08f40153f0 tx=0x7f08f4009360 comp rx=0 tx=0).stop 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 shutdown_connections 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5622ab507cc0 0x5622ab50a0e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622ab48ce60 0x5622ab48e280 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622ab483d60 0x5622ab48dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 --2- 172.21.15.66:0/3019072997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622ab4833b0 0x5622ab48d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f08ecff9640 0 client.16455 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:25.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 >> 172.21.15.66:0/3019072997 conn(0x5622ab47dba0 msgr2=0x5622ab48a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 shutdown_connections 2024-04-21T09:10:25.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 1 -- 172.21.15.66:0/3019072997 wait complete. 2024-04-21T09:10:25.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090e3b6740 20 client.16455 trim_cache size 0 max 16384 2024-04-21T09:10:25.918 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08f00a3d20 msgr2=0x7f08f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08f00a3d20 0x7f08f00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08fc098940 tx=0x7f08fc004020 comp rx=0 tx=0).stop 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f090419cfd0 msgr2=0x7f090419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f090419cfd0 0x7f090419d480 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009fd0 tx=0x7f08fc00b260 comp rx=0 tx=0).stop 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 shutdown_connections 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f08f00a3d20 0x7f08f00a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f090419cfd0 0x7f090419d480 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0904106170 0x7f0904198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.911+0000 7f090b6f1640 1 --2- 172.21.15.66:0/3060642553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09041010a0 0x7f0904198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.912+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 >> 172.21.15.66:0/3060642553 conn(0x7f0904077eb0 msgr2=0x7f09040ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.912+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 shutdown_connections 2024-04-21T09:10:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:25.912+0000 7f090b6f1640 1 -- 172.21.15.66:0/3060642553 wait complete. 2024-04-21T09:10:25.923 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 -- 172.21.15.66:0/1137849286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1988106160 msgr2=0x7f198810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1137849286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1988106160 0x7f198810dd30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1980009ca0 tx=0x7f198002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 -- 172.21.15.66:0/1137849286 shutdown_connections 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1137849286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1988106160 0x7f198810dd30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1137849286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1988101a40 0x7f1988105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1137849286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1988101090 0x7f1988101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 -- 172.21.15.66:0/1137849286 >> 172.21.15.66:0/1137849286 conn(0x7f1988077f50 msgr2=0x7f1988078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 -- 172.21.15.66:0/1137849286 shutdown_connections 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 -- 172.21.15.66:0/1137849286 wait complete. 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 Processor -- start 2024-04-21T09:10:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.028+0000 7f198cbf4640 1 -- start start 2024-04-21T09:10:26.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198cbf4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1988101090 0x7f1988198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198cbf4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1988106160 0x7f1988198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198cbf4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f198819cf00 0x7f198819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198cbf4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19881121e0 con 0x7f198819cf00 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198cbf4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1988112060 con 0x7f1988101090 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198cbf4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1988112360 con 0x7f1988106160 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f198819cf00 0x7f198819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f198819cf00 0x7f198819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58938/0 (socket says 172.21.15.66:58938) 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 -- 172.21.15.66:0/1348823236 learned_addr learned my addr 172.21.15.66:0/1348823236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f197ffff640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1988106160 0x7f1988198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198659c640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1988101090 0x7f1988198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 -- 172.21.15.66:0/1348823236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1988106160 msgr2=0x7f1988198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1988106160 0x7f1988198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 -- 172.21.15.66:0/1348823236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1988101090 msgr2=0x7f1988198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1988101090 0x7f1988198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 -- 172.21.15.66:0/1348823236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198819d9f0 con 0x7f198819cf00 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f198659c640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1988101090 0x7f1988198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:26.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f1986d9d640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f198819cf00 0x7f198819d3b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1980005e30 tx=0x7f1980034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:26.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.029+0000 7f197f7fe640 1 -- 172.21.15.66:0/1348823236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980039040 con 0x7f198819cf00 2024-04-21T09:10:26.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.030+0000 7f197f7fe640 1 -- 172.21.15.66:0/1348823236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1980044070 con 0x7f198819cf00 2024-04-21T09:10:26.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.030+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198819dc80 con 0x7f198819cf00 2024-04-21T09:10:26.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.030+0000 7f197f7fe640 1 -- 172.21.15.66:0/1348823236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980004040 con 0x7f198819cf00 2024-04-21T09:10:26.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.030+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a5440 con 0x7f198819cf00 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.032+0000 7f197f7fe640 1 -- 172.21.15.66:0/1348823236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f19800324c0 con 0x7f198819cf00 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.032+0000 7f198f7e5740 1 Processor -- start 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.032+0000 7f198f7e5740 1 -- start start 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a2c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2ce80 0x559638a35430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a35a00 0x559638a37e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f198f7e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5596389911f0 con 0x559638a35a00 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f198f7e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55963892e4b0 con 0x559638a2c4d0 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f198f7e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55963898d1b0 con 0x559638a2ce80 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197f7fe640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19740a3d20 0x7f19740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a35a00 0x559638a37e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a35a00 0x559638a37e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58952/0 (socket says 172.21.15.66:58952) 2024-04-21T09:10:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 -- 172.21.15.66:0/4241775203 learned_addr learned my addr 172.21.15.66:0/4241775203 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f198659c640 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2ce80 0x559638a35430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f1986d9d640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19740a3d20 0x7f19740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a2c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197f7fe640 1 -- 172.21.15.66:0/1348823236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f19800f2cf0 con 0x7f198819cf00 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 -- 172.21.15.66:0/4241775203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2ce80 msgr2=0x559638a35430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2ce80 0x559638a35430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 -- 172.21.15.66:0/4241775203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 msgr2=0x559638a2c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a2c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 -- 172.21.15.66:0/4241775203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596389201b0 con 0x559638a35a00 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f197ffff640 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a35a00 0x559638a37e50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1970011730 tx=0x7f197000a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcce4ea1e5e313cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.033+0000 7f1986d9d640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19740a3d20 0x7f19740a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f19800be730 tx=0x7f198008d3e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f197cff9640 1 -- 172.21.15.66:0/4241775203 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970005020 con 0x559638a35a00 2024-04-21T09:10:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f197cff9640 1 -- 172.21.15.66:0/4241775203 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19700047a0 con 0x559638a35a00 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f197cff9640 1 -- 172.21.15.66:0/4241775203 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000f9c0 con 0x559638a35a00 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 -- 172.21.15.66:0/4241775203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a35a00 msgr2=0x559638a37e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a35a00 0x559638a37e50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1970011730 tx=0x7f197000a6a0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 -- 172.21.15.66:0/4241775203 shutdown_connections 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a35a00 0x559638a37e50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2ce80 0x559638a35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4241775203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a2c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 -- 172.21.15.66:0/4241775203 >> 172.21.15.66:0/4241775203 conn(0x559638a26ba0 msgr2=0x559638a26f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 -- 172.21.15.66:0/4241775203 shutdown_connections 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.034+0000 7f198f7e5740 1 -- 172.21.15.66:0/4241775203 wait complete. 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 -- start start 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 Processor -- start 2024-04-21T09:10:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2c4d0 0x559638a35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2ce80 0x559638a326c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 0x559638a32cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559638a38970 con 0x559638a2c4d0 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55963894a500 con 0x559638a2ce80 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559638a38af0 con 0x559638a35a00 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198f7e5740 10 client.0 _ll_register_callbacks cb 0x559638918b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2c4d0 0x559638a35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2c4d0 0x559638a35330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58960/0 (socket says 172.21.15.66:58960) 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 -- 172.21.15.66:0/1829725480 learned_addr learned my addr 172.21.15.66:0/1829725480 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198659c640 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2ce80 0x559638a326c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 -- 172.21.15.66:0/1829725480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 msgr2=0x559638a32cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 0x559638a32cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 -- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2ce80 msgr2=0x559638a326c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2ce80 0x559638a326c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f197ffff640 1 -- 172.21.15.66:0/1829725480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596386a33f0 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.035+0000 7f198659c640 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2ce80 0x559638a326c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f197ffff640 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2c4d0 0x559638a35330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1970005bd0 tx=0x7f19700046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a7011a77d79b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f196dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f196dffb640 1 -- 172.21.15.66:0/1829725480 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000f7c0 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f196dffb640 1 -- 172.21.15.66:0/1829725480 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f197000f960 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19700047a0 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f196dffb640 1 -- 172.21.15.66:0/1829725480 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1970030050 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 10 client.16467 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55963876d7d0 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f196dffb640 1 -- 172.21.15.66:0/1829725480 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19700047a0 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f196dffb640 1 -- 172.21.15.66:0/1829725480 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f197001eab0 con 0x559638a2c4d0 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 10 client.16467 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:26.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 20 client.16467 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638a45af0 0x559638a47ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 4 client.16467 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.036+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559638930930 con 0x559638a45af0 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.037+0000 7f198659c640 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638a45af0 0x559638a47ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.037+0000 7f198659c640 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638a45af0 0x559638a47ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bb065e2652211ae server_cookie=6b73e0235b33b1e in_seq=0 out_seq=0 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.037+0000 7f196dffb640 10 client.16467 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f196dffb640 1 -- 172.21.15.66:0/1829725480 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559638930930 con 0x559638a45af0 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f196dffb640 10 client.16467 handle_command_reply: tid=0 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 client.16467 shutdown 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2c4d0 msgr2=0x559638a35330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2c4d0 0x559638a35330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1970005bd0 tx=0x7f19700046a0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 shutdown_connections 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638a45af0 0x559638a47ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 0x559638a32cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f196dffb640 0 client.16467 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2ce80 0x559638a326c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1829725480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2c4d0 0x559638a35330 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 >> 172.21.15.66:0/1829725480 conn(0x559638a26ba0 msgr2=0x559638a288f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 shutdown_connections 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 1 -- 172.21.15.66:0/1829725480 wait complete. 2024-04-21T09:10:26.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.038+0000 7f198f7e5740 20 client.16467 trim_cache size 0 max 16384 2024-04-21T09:10:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 Processor -- start 2024-04-21T09:10:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 -- start start 2024-04-21T09:10:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a27ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 0x559638a34df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b19f0 con 0x559638a2ce80 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19881af2c0 con 0x559638a2c4d0 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.051+0000 7f198f7e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ad0f0 con 0x559638a35a00 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a27ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a27ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58974/0 (socket says 172.21.15.66:58974) 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 -- 172.21.15.66:0/1142897845 learned_addr learned my addr 172.21.15.66:0/1142897845 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 -- 172.21.15.66:0/1142897845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 msgr2=0x559638a34df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 0x559638a34df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f197ffff640 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 -- 172.21.15.66:0/1142897845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 msgr2=0x559638a41860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a41860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 -- 172.21.15.66:0/1142897845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559638a48470 con 0x559638a2ce80 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f197ffff640 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a41860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f198659c640 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a27ef0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x559638a2bfa0 tx=0x7f19780039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f86f784adae0408 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f196ffff640 1 -- 172.21.15.66:0/1142897845 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800bd00 con 0x559638a2ce80 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f196ffff640 1 -- 172.21.15.66:0/1142897845 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197801d8d0 con 0x559638a2ce80 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.052+0000 7f196ffff640 1 -- 172.21.15.66:0/1142897845 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19780047a0 con 0x559638a2ce80 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- 172.21.15.66:0/1142897845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 msgr2=0x559638a27ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a27ef0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x559638a2bfa0 tx=0x7f19780039c0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- 172.21.15.66:0/1142897845 shutdown_connections 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a35a00 0x559638a34df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a27ef0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 --2- 172.21.15.66:0/1142897845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a2c4d0 0x559638a41860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- 172.21.15.66:0/1142897845 >> 172.21.15.66:0/1142897845 conn(0x559638a26ba0 msgr2=0x559638a2db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- 172.21.15.66:0/1142897845 shutdown_connections 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- 172.21.15.66:0/1142897845 wait complete. 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- start start 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 Processor -- start 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2c4d0 0x559638a3dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a3e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a35a00 0x559638a394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ad780 con 0x559638a2ce80 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19881aebf0 con 0x559638a35a00 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ade50 con 0x559638a2c4d0 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198f7e5740 10 client.0 _ll_register_callbacks cb 0x7f1970011600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a3e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.053+0000 7f198659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a3e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58978/0 (socket says 172.21.15.66:58978) 2024-04-21T09:10:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198659c640 1 -- 172.21.15.66:0/4071679478 learned_addr learned my addr 172.21.15.66:0/4071679478 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f197ffff640 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2c4d0 0x559638a3dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198659c640 1 -- 172.21.15.66:0/4071679478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2c4d0 msgr2=0x559638a3dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198659c640 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2c4d0 0x559638a3dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198659c640 1 -- 172.21.15.66:0/4071679478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a35a00 msgr2=0x559638a394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198659c640 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a35a00 0x559638a394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198659c640 1 -- 172.21.15.66:0/4071679478 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559638a33660 con 0x559638a2ce80 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f197ffff640 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2c4d0 0x559638a3dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198659c640 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a3e2e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1978000c00 tx=0x7f197800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64efb311d627df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f196dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f196dffb640 1 -- 172.21.15.66:0/4071679478 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197802a070 con 0x559638a2ce80 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f197801d8d0 con 0x559638a2ce80 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f196dffb640 1 -- 172.21.15.66:0/4071679478 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f197801d630 con 0x559638a2ce80 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f196dffb640 1 -- 172.21.15.66:0/4071679478 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1978036050 con 0x559638a2ce80 2024-04-21T09:10:26.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198f7e5740 10 client.16473 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559638aab960 con 0x559638a2ce80 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.054+0000 7f196dffb640 1 -- 172.21.15.66:0/4071679478 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197801d8d0 con 0x559638a2ce80 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f196dffb640 1 -- 172.21.15.66:0/4071679478 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f19780114f0 con 0x559638a2ce80 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f198f7e5740 10 client.16473 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f198f7e5740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f198f7e5740 20 client.16473 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638aac030 0x559638aae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f198f7e5740 4 client.16473 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55963899ec10 con 0x559638aac030 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f197ffff640 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638aac030 0x559638aae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f196dffb640 10 client.16473 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.055+0000 7f197ffff640 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638aac030 0x559638aae450 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3281285a30fc6f1e server_cookie=57d9cfa9da3b65da in_seq=0 out_seq=0 2024-04-21T09:10:26.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f196dffb640 1 -- 172.21.15.66:0/4071679478 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55963899ec10 con 0x559638aac030 2024-04-21T09:10:26.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f196dffb640 10 client.16473 handle_command_reply: tid=0 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 client.16473 shutdown 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 msgr2=0x559638a3e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a3e2e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1978000c00 tx=0x7f197800bee0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 shutdown_connections 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559638aac030 0x559638aae450 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559638a35a00 0x559638a394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559638a2ce80 0x559638a3e2e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 --2- 172.21.15.66:0/4071679478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559638a2c4d0 0x559638a3dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f196dffb640 0 client.16473 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:26.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 >> 172.21.15.66:0/4071679478 conn(0x559638a26ba0 msgr2=0x559638a25370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:26.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 shutdown_connections 2024-04-21T09:10:26.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 1 -- 172.21.15.66:0/4071679478 wait complete. 2024-04-21T09:10:26.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198f7e5740 20 client.16473 trim_cache size 0 max 16384 2024-04-21T09:10:26.066 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19740a3d20 msgr2=0x7f19740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19740a3d20 0x7f19740a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f19800be730 tx=0x7f198008d3e0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f198819cf00 msgr2=0x7f198819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f198819cf00 0x7f198819d3b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1980005e30 tx=0x7f1980034760 comp rx=0 tx=0).stop 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 shutdown_connections 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19740a3d20 0x7f19740a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f198819cf00 0x7f198819d3b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1988106160 0x7f1988198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 --2- 172.21.15.66:0/1348823236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1988101090 0x7f1988198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:26.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 >> 172.21.15.66:0/1348823236 conn(0x7f1988077f50 msgr2=0x7f19880feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:26.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 shutdown_connections 2024-04-21T09:10:26.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:26.056+0000 7f198cbf4640 1 -- 172.21.15.66:0/1348823236 wait complete. 2024-04-21T09:10:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:26 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1331389221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:26.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:26 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1331389221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:26.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:26 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1331389221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:27.070 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:10:27.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:10:27.070 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:10:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 -- 172.21.15.66:0/2552062354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c107270 msgr2=0x7f377c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 --2- 172.21.15.66:0/2552062354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c107270 0x7f377c10ee40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3774009900 tx=0x7f377402bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 -- 172.21.15.66:0/2552062354 shutdown_connections 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 --2- 172.21.15.66:0/2552062354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c107270 0x7f377c10ee40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 --2- 172.21.15.66:0/2552062354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f377c06b1f0 0x7f377c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 --2- 172.21.15.66:0/2552062354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377c06a840 0x7f377c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 -- 172.21.15.66:0/2552062354 >> 172.21.15.66:0/2552062354 conn(0x7f377c077eb0 msgr2=0x7f377c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 -- 172.21.15.66:0/2552062354 shutdown_connections 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 -- 172.21.15.66:0/2552062354 wait complete. 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 Processor -- start 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.188+0000 7f378447a640 1 -- start start 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f378447a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f377c06a840 0x7f377c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f378447a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377c107270 0x7f377c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f378447a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c19cf70 0x7f377c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f378447a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1121d0 con 0x7f377c19cf70 2024-04-21T09:10:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f378447a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f377c112050 con 0x7f377c06a840 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f378447a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f377c112350 con 0x7f377c107270 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c19cf70 0x7f377c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c19cf70 0x7f377c19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58994/0 (socket says 172.21.15.66:58994) 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 -- 172.21.15.66:0/2124224211 learned_addr learned my addr 172.21.15.66:0/2124224211 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3781a15640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377c107270 0x7f377c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782216640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f377c06a840 0x7f377c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 -- 172.21.15.66:0/2124224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377c107270 msgr2=0x7f377c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377c107270 0x7f377c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 -- 172.21.15.66:0/2124224211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f377c06a840 msgr2=0x7f377c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f377c06a840 0x7f377c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 -- 172.21.15.66:0/2124224211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c19da60 con 0x7f377c19cf70 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f3782a17640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c19cf70 0x7f377c19d420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3774009230 tx=0x7f37740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f377b7fe640 1 -- 172.21.15.66:0/2124224211 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774043020 con 0x7f377c19cf70 2024-04-21T09:10:27.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f377b7fe640 1 -- 172.21.15.66:0/2124224211 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377402cb70 con 0x7f377c19cf70 2024-04-21T09:10:27.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c19dcf0 con 0x7f377c19cf70 2024-04-21T09:10:27.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.189+0000 7f377b7fe640 1 -- 172.21.15.66:0/2124224211 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774003ac0 con 0x7f377c19cf70 2024-04-21T09:10:27.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.190+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1a5540 con 0x7f377c19cf70 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.192+0000 7f377b7fe640 1 -- 172.21.15.66:0/2124224211 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3774046020 con 0x7f377c19cf70 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.192+0000 7f378706a740 1 Processor -- start 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.192+0000 7f378706a740 1 -- start start 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f378706a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d661cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f378706a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d66a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f378706a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d66ae10 0x55ee2d66d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f378706a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ee2d5c5f00 con 0x55ee2d662290 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f378706a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee2d563590 con 0x55ee2d6618e0 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f378706a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ee2d5c1ec0 con 0x55ee2d66ae10 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f377b7fe640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37640a3da0 0x7f37640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d66a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d66a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59008/0 (socket says 172.21.15.66:59008) 2024-04-21T09:10:27.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 -- 172.21.15.66:0/2724827627 learned_addr learned my addr 172.21.15.66:0/2724827627 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3781a15640 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d66ae10 0x55ee2d66d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782a17640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37640a3da0 0x7f37640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3781a15640 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d661cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f377b7fe640 1 -- 172.21.15.66:0/2124224211 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f37740f2c70 con 0x7f377c19cf70 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 -- 172.21.15.66:0/2724827627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d66ae10 msgr2=0x55ee2d66d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d66ae10 0x55ee2d66d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 -- 172.21.15.66:0/2724827627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 msgr2=0x55ee2d661cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d661cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 -- 172.21.15.66:0/2724827627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2d3a07e0 con 0x55ee2d662290 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782216640 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d66a840 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f376c009f60 tx=0x7f376c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c7da519c7e7ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.193+0000 7f3782a17640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37640a3da0 0x7f37640a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37740be6b0 tx=0x7f37740a9040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f3778bfe640 1 -- 172.21.15.66:0/2724827627 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c016070 con 0x55ee2d662290 2024-04-21T09:10:27.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f3778bfe640 1 -- 172.21.15.66:0/2724827627 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f376c004d20 con 0x55ee2d662290 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f3778bfe640 1 -- 172.21.15.66:0/2724827627 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c012780 con 0x55ee2d662290 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f378706a740 1 -- 172.21.15.66:0/2724827627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 msgr2=0x55ee2d66a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f378706a740 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d66a840 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f376c009f60 tx=0x7f376c009f90 comp rx=0 tx=0).stop 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f378706a740 1 -- 172.21.15.66:0/2724827627 shutdown_connections 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f378706a740 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d66ae10 0x55ee2d66d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f378706a740 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d66a840 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f378706a740 1 --2- 172.21.15.66:0/2724827627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d661cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.194+0000 7f378706a740 1 -- 172.21.15.66:0/2724827627 >> 172.21.15.66:0/2724827627 conn(0x55ee2d65bfb0 msgr2=0x55ee2d65c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 -- 172.21.15.66:0/2724827627 shutdown_connections 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 -- 172.21.15.66:0/2724827627 wait complete. 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 -- start start 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 Processor -- start 2024-04-21T09:10:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d667da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d6682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d668820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ee2d66dd80 con 0x55ee2d66ae10 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee2d57f580 con 0x55ee2d6618e0 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ee2d66df00 con 0x55ee2d662290 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f378706a740 10 client.0 _ll_register_callbacks cb 0x55ee2d585650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f3781a15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d668820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f3781a15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d668820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59014/0 (socket says 172.21.15.66:59014) 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f3781a15640 1 -- 172.21.15.66:0/1580602865 learned_addr learned my addr 172.21.15.66:0/1580602865 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f3781a15640 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d667da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.195+0000 7f3782216640 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d6682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3781a15640 1 -- 172.21.15.66:0/1580602865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 msgr2=0x55ee2d6682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3781a15640 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d6682e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3781a15640 1 -- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 msgr2=0x55ee2d667da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3781a15640 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d667da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3781a15640 1 -- 172.21.15.66:0/1580602865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2d400f70 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3781a15640 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d668820 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3770010f80 tx=0x7f37700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c255fbf22cec582 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3761ffb640 1 -- 172.21.15.66:0/1580602865 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004020 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3761ffb640 1 -- 172.21.15.66:0/1580602865 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f37700041c0 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f376c004d20 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3761ffb640 1 -- 172.21.15.66:0/1580602865 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37700043c0 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f378706a740 10 client.16485 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee2d298170 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3761ffb640 1 -- 172.21.15.66:0/1580602865 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f376c004d20 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.196+0000 7f3761ffb640 1 -- 172.21.15.66:0/1580602865 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3770004bb0 con 0x55ee2d66ae10 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f378706a740 10 client.16485 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f378706a740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f378706a740 20 client.16485 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f378706a740 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d67b3d0 0x55ee2d67d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f378706a740 4 client.16485 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee2d5d3920 con 0x55ee2d67b3d0 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f3782216640 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d67b3d0 0x55ee2d67d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f3761ffb640 10 client.16485 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.197+0000 7f3782216640 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d67b3d0 0x55ee2d67d7e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47df294a78a45f5b server_cookie=c57e048f92c61b0c in_seq=0 out_seq=0 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f3761ffb640 1 -- 172.21.15.66:0/1580602865 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee2d5d3920 con 0x55ee2d67b3d0 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f3761ffb640 10 client.16485 handle_command_reply: tid=0 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 client.16485 shutdown 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 msgr2=0x55ee2d668820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d668820 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3770010f80 tx=0x7f37700046c0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 shutdown_connections 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d67b3d0 0x55ee2d67d7e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d668820 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d6682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f3761ffb640 0 client.16485 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 --2- 172.21.15.66:0/1580602865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d667da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 >> 172.21.15.66:0/1580602865 conn(0x55ee2d65bfb0 msgr2=0x55ee2d65dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 shutdown_connections 2024-04-21T09:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 1 -- 172.21.15.66:0/1580602865 wait complete. 2024-04-21T09:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.198+0000 7f378706a740 20 client.16485 trim_cache size 0 max 16384 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 Processor -- start 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 -- start start 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d66a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d676c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d666ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1b1a90 con 0x55ee2d66ae10 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1af360 con 0x55ee2d6618e0 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f378706a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1ad190 con 0x55ee2d662290 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f3781a15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d666ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f3781a15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d666ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59030/0 (socket says 172.21.15.66:59030) 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.216+0000 7f3781a15640 1 -- 172.21.15.66:0/3349015704 learned_addr learned my addr 172.21.15.66:0/3349015704 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3781a15640 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d66a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3782216640 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d676c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3781a15640 1 -- 172.21.15.66:0/3349015704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 msgr2=0x55ee2d676c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3781a15640 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d676c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3781a15640 1 -- 172.21.15.66:0/3349015704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 msgr2=0x55ee2d66a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3781a15640 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d66a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3781a15640 1 -- 172.21.15.66:0/3349015704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2d679f70 con 0x55ee2d66ae10 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3781a15640 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d666ce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3770010be0 tx=0x7f3770005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fd1f2a8e218e31 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3763fff640 1 -- 172.21.15.66:0/3349015704 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004fe0 con 0x55ee2d66ae10 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3763fff640 1 -- 172.21.15.66:0/3349015704 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377000ae00 con 0x55ee2d66ae10 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.217+0000 7f3763fff640 1 -- 172.21.15.66:0/3349015704 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004c70 con 0x55ee2d66ae10 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- 172.21.15.66:0/3349015704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 msgr2=0x55ee2d666ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d666ce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3770010be0 tx=0x7f3770005c40 comp rx=0 tx=0).stop 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- 172.21.15.66:0/3349015704 shutdown_connections 2024-04-21T09:10:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d66ae10 0x55ee2d666ce0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d662290 0x55ee2d676c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 --2- 172.21.15.66:0/3349015704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d6618e0 0x55ee2d66a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- 172.21.15.66:0/3349015704 >> 172.21.15.66:0/3349015704 conn(0x55ee2d65bfb0 msgr2=0x55ee2d65b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- 172.21.15.66:0/3349015704 shutdown_connections 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- 172.21.15.66:0/3349015704 wait complete. 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- start start 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 Processor -- start 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d6618e0 0x55ee2d6deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d6df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d66ae10 0x55ee2d6dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1ad820 con 0x55ee2d662290 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1aec90 con 0x55ee2d66ae10 2024-04-21T09:10:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1adef0 con 0x55ee2d6618e0 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f378706a740 10 client.0 _ll_register_callbacks cb 0x7f377000abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f3782216640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d6df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f3782216640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d6df0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59044/0 (socket says 172.21.15.66:59044) 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f3782216640 1 -- 172.21.15.66:0/1697220895 learned_addr learned my addr 172.21.15.66:0/1697220895 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.218+0000 7f3781a15640 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d6618e0 0x55ee2d6deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3781a15640 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d66ae10 0x55ee2d6dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3782216640 1 -- 172.21.15.66:0/1697220895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d6618e0 msgr2=0x55ee2d6deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3782216640 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d6618e0 0x55ee2d6deb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3782216640 1 -- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d66ae10 msgr2=0x55ee2d6dd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3782216640 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d66ae10 0x55ee2d6dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3782216640 1 -- 172.21.15.66:0/1697220895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2d6ddf60 con 0x55ee2d662290 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3782216640 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d6df0d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f376c009eb0 tx=0x7f376c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950b13163ffa6c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3761ffb640 1 -- 172.21.15.66:0/1697220895 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c02e060 con 0x55ee2d662290 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3761ffb640 1 -- 172.21.15.66:0/1697220895 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f376c028370 con 0x55ee2d662290 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377000ae00 con 0x55ee2d662290 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3761ffb640 1 -- 172.21.15.66:0/1697220895 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f376c035050 con 0x55ee2d662290 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f378706a740 10 client.16491 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee2d6e6510 con 0x55ee2d662290 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3761ffb640 1 -- 172.21.15.66:0/1697220895 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377000ae00 con 0x55ee2d662290 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f3761ffb640 1 -- 172.21.15.66:0/1697220895 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f376c00f040 con 0x55ee2d662290 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f378706a740 10 client.16491 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f378706a740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.219+0000 7f378706a740 20 client.16491 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.220+0000 7f378706a740 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d6e69b0 0x55ee2d6e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.220+0000 7f378706a740 4 client.16491 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.220+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee2d5cd790 con 0x55ee2d6e69b0 2024-04-21T09:10:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.220+0000 7f3781a15640 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d6e69b0 0x55ee2d6e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.220+0000 7f3761ffb640 10 client.16491 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.220+0000 7f3781a15640 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d6e69b0 0x55ee2d6e8da0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120263a36dadcbe3 server_cookie=de4710249dd10e30 in_seq=0 out_seq=0 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f3761ffb640 1 -- 172.21.15.66:0/1697220895 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee2d5cd790 con 0x55ee2d6e69b0 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f3761ffb640 10 client.16491 handle_command_reply: tid=0 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 client.16491 shutdown 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 msgr2=0x55ee2d6df0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d6df0d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f376c009eb0 tx=0x7f376c009be0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 shutdown_connections 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ee2d6e69b0 0x55ee2d6e8da0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2d66ae10 0x55ee2d6dd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee2d662290 0x55ee2d6df0d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f3761ffb640 0 client.16491 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 --2- 172.21.15.66:0/1697220895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee2d6618e0 0x55ee2d6deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.221+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 >> 172.21.15.66:0/1697220895 conn(0x55ee2d65bfb0 msgr2=0x55ee2d667d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 shutdown_connections 2024-04-21T09:10:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378706a740 1 -- 172.21.15.66:0/1697220895 wait complete. 2024-04-21T09:10:27.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378706a740 20 client.16491 trim_cache size 0 max 16384 2024-04-21T09:10:27.228 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:10:27.228 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:10:27.228 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "5a97f62b-a2a2-4905-b8ae-c66c5bb256ff", 2024-04-21T09:10:27.228 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:10:27.228 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37640a3da0 msgr2=0x7f37640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37640a3da0 0x7f37640a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37740be6b0 tx=0x7f37740a9040 comp rx=0 tx=0).stop 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c19cf70 msgr2=0x7f377c19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c19cf70 0x7f377c19d420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3774009230 tx=0x7f37740045e0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 shutdown_connections 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f37640a3da0 0x7f37640a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f377c19cf70 0x7f377c19d420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377c107270 0x7f377c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 --2- 172.21.15.66:0/2124224211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f377c06a840 0x7f377c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 >> 172.21.15.66:0/2124224211 conn(0x7f377c077eb0 msgr2=0x7f377c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 shutdown_connections 2024-04-21T09:10:27.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.222+0000 7f378447a640 1 -- 172.21.15.66:0/2124224211 wait complete. 2024-04-21T09:10:27.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:10:27.234 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 -- 172.21.15.66:0/2235742505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 msgr2=0x7f8970108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 --2- 172.21.15.66:0/2235742505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970108440 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8960009930 tx=0x7f896002bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 -- 172.21.15.66:0/2235742505 shutdown_connections 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 --2- 172.21.15.66:0/2235742505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f897006af70 0x7f897010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 --2- 172.21.15.66:0/2235742505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f897006a5b0 0x7f897006aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 --2- 172.21.15.66:0/2235742505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970108440 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 -- 172.21.15.66:0/2235742505 >> 172.21.15.66:0/2235742505 conn(0x7f8970077c40 msgr2=0x7f8970078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 -- 172.21.15.66:0/2235742505 shutdown_connections 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.339+0000 7f8975881640 1 -- 172.21.15.66:0/2235742505 wait complete. 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 Processor -- start 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 -- start start 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f897006a5b0 0x7f8970101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f897006af70 0x7f89701016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8970112180 con 0x7f8970108060 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8970112000 con 0x7f897006af70 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8975881640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8970112300 con 0x7f897006a5b0 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f896f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f896f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59052/0 (socket says 172.21.15.66:59052) 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f896f7fe640 1 -- 172.21.15.66:0/2747925369 learned_addr learned my addr 172.21.15.66:0/2747925369 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.340+0000 7f8967fff640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f897006af70 0x7f89701016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896effd640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f897006a5b0 0x7f8970101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896f7fe640 1 -- 172.21.15.66:0/2747925369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f897006a5b0 msgr2=0x7f8970101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896f7fe640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f897006a5b0 0x7f8970101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896f7fe640 1 -- 172.21.15.66:0/2747925369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f897006af70 msgr2=0x7f89701016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896f7fe640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f897006af70 0x7f89701016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896f7fe640 1 -- 172.21.15.66:0/2747925369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89701a6b60 con 0x7f8970108060 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896effd640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f897006a5b0 0x7f8970101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f896f7fe640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970101bf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f896800eb10 tx=0x7f896800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f897487f640 1 -- 172.21.15.66:0/2747925369 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968004b20 con 0x7f8970108060 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f897487f640 1 -- 172.21.15.66:0/2747925369 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8968004cc0 con 0x7f8970108060 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89701a6e50 con 0x7f8970108060 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f897487f640 1 -- 172.21.15.66:0/2747925369 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968010670 con 0x7f8970108060 2024-04-21T09:10:27.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.341+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89701a7320 con 0x7f8970108060 2024-04-21T09:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.343+0000 7f897487f640 1 -- 172.21.15.66:0/2747925369 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8968005020 con 0x7f8970108060 2024-04-21T09:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 Processor -- start 2024-04-21T09:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 -- start start 2024-04-21T09:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44ac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d4b44111e0 con 0x55d4b44b5a80 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d4b43ae390 con 0x55d4b44ac580 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8978470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d4b440d1a0 con 0x55d4b44acea0 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f897487f640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f895c0a3d20 0x7f895c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59056/0 (socket says 172.21.15.66:59056) 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8967fff640 1 -- 172.21.15.66:0/3135698322 learned_addr learned my addr 172.21.15.66:0/3135698322 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f896f7fe640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f895c0a3d20 0x7f895c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.344+0000 7f8967fff640 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f896effd640 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f897487f640 1 -- 172.21.15.66:0/2747925369 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f89680d11b0 con 0x7f8970108060 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8967fff640 1 -- 172.21.15.66:0/3135698322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 msgr2=0x55d4b44b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8967fff640 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8967fff640 1 -- 172.21.15.66:0/3135698322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 msgr2=0x55d4b44ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8967fff640 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8967fff640 1 -- 172.21.15.66:0/3135698322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b41ed7d0 con 0x55d4b44b5a80 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8967fff640 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b7e70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8958013630 tx=0x7f895800c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9611976f61963a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f896f7fe640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f895c0a3d20 0x7f895c0a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8968062740 tx=0x7f896806b790 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8965ffb640 1 -- 172.21.15.66:0/3135698322 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958019030 con 0x55d4b44b5a80 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8965ffb640 1 -- 172.21.15.66:0/3135698322 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89580054d0 con 0x55d4b44b5a80 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.345+0000 7f8965ffb640 1 -- 172.21.15.66:0/3135698322 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89580115a0 con 0x55d4b44b5a80 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 -- 172.21.15.66:0/3135698322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 msgr2=0x55d4b44b7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b7e70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8958013630 tx=0x7f895800c690 comp rx=0 tx=0).stop 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 -- 172.21.15.66:0/3135698322 shutdown_connections 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b7e70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 --2- 172.21.15.66:0/3135698322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 -- 172.21.15.66:0/3135698322 >> 172.21.15.66:0/3135698322 conn(0x55d4b44a6cb0 msgr2=0x55d4b44a7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 -- 172.21.15.66:0/3135698322 shutdown_connections 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 -- 172.21.15.66:0/3135698322 wait complete. 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 -- start start 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.346+0000 7f8978470740 1 Processor -- start 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d4b44b8870 con 0x55d4b44b5a80 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d4b43ca2f0 con 0x55d4b44ac580 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d4b44b89f0 con 0x55d4b44acea0 2024-04-21T09:10:27.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 10 client.0 _ll_register_callbacks cb 0x55d4b43d7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59066/0 (socket says 172.21.15.66:59066) 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 -- 172.21.15.66:0/2888755987 learned_addr learned my addr 172.21.15.66:0/2888755987 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f896effd640 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 -- 172.21.15.66:0/2888755987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 msgr2=0x55d4b44b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 -- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 msgr2=0x55d4b44b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 -- 172.21.15.66:0/2888755987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b41233f0 con 0x55d4b44b5a80 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8967fff640 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3520 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8958013fd0 tx=0x7f895800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82aea93327454f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f894effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f894effd640 1 -- 172.21.15.66:0/2888755987 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958019040 con 0x55d4b44b5a80 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f894effd640 1 -- 172.21.15.66:0/2888755987 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8958004fe0 con 0x55d4b44b5a80 2024-04-21T09:10:27.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.347+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89580054d0 con 0x55d4b44b5a80 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f894effd640 1 -- 172.21.15.66:0/2888755987 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89580051e0 con 0x55d4b44b5a80 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 10 client.16503 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4b40e30f0 con 0x55d4b44b5a80 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f894effd640 1 -- 172.21.15.66:0/2888755987 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89580054d0 con 0x55d4b44b5a80 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f894effd640 1 -- 172.21.15.66:0/2888755987 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89580039d0 con 0x55d4b44b5a80 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 10 client.16503 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 20 client.16503 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b44c5970 0x55d4b44c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 4 client.16503 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4b441ec00 con 0x55d4b44c5970 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.348+0000 7f896effd640 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b44c5970 0x55d4b44c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.349+0000 7f894effd640 10 client.16503 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.349+0000 7f896effd640 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b44c5970 0x55d4b44c7d60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34de2aabf80f133f server_cookie=9f132d774bd1b9c2 in_seq=0 out_seq=0 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.349+0000 7f894effd640 1 -- 172.21.15.66:0/2888755987 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4b441ec00 con 0x55d4b44c5970 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.349+0000 7f894effd640 10 client.16503 handle_command_reply: tid=0 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 client.16503 shutdown 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 msgr2=0x55d4b44b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3520 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8958013fd0 tx=0x7f895800cf30 comp rx=0 tx=0).stop 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 shutdown_connections 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b44c5970 0x55d4b44c7d60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3520 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 --2- 172.21.15.66:0/2888755987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f894effd640 0 client.16503 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 >> 172.21.15.66:0/2888755987 conn(0x55d4b44a6cb0 msgr2=0x55d4b44a8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 shutdown_connections 2024-04-21T09:10:27.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 1 -- 172.21.15.66:0/2888755987 wait complete. 2024-04-21T09:10:27.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.350+0000 7f8978470740 20 client.16503 trim_cache size 0 max 16384 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.367+0000 7f8978470740 1 Processor -- start 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.367+0000 7f8978470740 1 -- start start 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f8978470740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f8978470740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44acea0 0x55d4b44c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f8978470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f8978470740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89701b3e00 con 0x55d4b44acea0 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f8978470740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89701b16d0 con 0x55d4b44ac580 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f8978470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f89701af500 con 0x55d4b44b5a80 2024-04-21T09:10:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44acea0 0x55d4b44c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44acea0 0x55d4b44c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59074/0 (socket says 172.21.15.66:59074) 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 -- 172.21.15.66:0/2074225228 learned_addr learned my addr 172.21.15.66:0/2074225228 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 -- 172.21.15.66:0/2074225228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44b5a80 msgr2=0x55d4b44b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 --2- 172.21.15.66:0/2074225228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 -- 172.21.15.66:0/2074225228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 msgr2=0x55d4b44b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 --2- 172.21.15.66:0/2074225228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 -- 172.21.15.66:0/2074225228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b44accd0 con 0x55d4b44acea0 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f896effd640 1 --2- 172.21.15.66:0/2074225228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44acea0 0x55d4b44c1760 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8960005c40 tx=0x7f8960000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58a173fa3d2fe42 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f89657fa640 1 -- 172.21.15.66:0/2074225228 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896004d020 con 0x55d4b44acea0 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f89657fa640 1 -- 172.21.15.66:0/2074225228 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896002c7e0 con 0x55d4b44acea0 2024-04-21T09:10:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.368+0000 7f89657fa640 1 -- 172.21.15.66:0/2074225228 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89600046c0 con 0x55d4b44acea0 2024-04-21T09:10:27.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 -- 172.21.15.66:0/2074225228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44acea0 msgr2=0x55d4b44c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 --2- 172.21.15.66:0/2074225228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44acea0 0x55d4b44c1760 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8960005c40 tx=0x7f8960000bc0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:27 smithi066 ceph-mon[27953]: pgmap v70: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6.3 MiB/s wr, 239 op/s 2024-04-21T09:10:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 -- 172.21.15.66:0/2074225228 shutdown_connections 2024-04-21T09:10:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 --2- 172.21.15.66:0/2074225228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44b5a80 0x55d4b44b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 --2- 172.21.15.66:0/2074225228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44acea0 0x55d4b44c1760 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 --2- 172.21.15.66:0/2074225228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 -- 172.21.15.66:0/2074225228 >> 172.21.15.66:0/2074225228 conn(0x55d4b44a6cb0 msgr2=0x55d4b44b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 -- 172.21.15.66:0/2074225228 shutdown_connections 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 -- 172.21.15.66:0/2074225228 wait complete. 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.369+0000 7f8978470740 1 -- start start 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 1 Processor -- start 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44be020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89701afb90 con 0x55d4b44b5a80 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89701b1000 con 0x55d4b44ac580 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f89701b0260 con 0x55d4b44acea0 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8978470740 10 client.0 _ll_register_callbacks cb 0x7f8958013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59086/0 (socket says 172.21.15.66:59086) 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 -- 172.21.15.66:0/2425650963 learned_addr learned my addr 172.21.15.66:0/2425650963 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f896effd640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44be020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 -- 172.21.15.66:0/2425650963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 msgr2=0x55d4b44b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 -- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 msgr2=0x55d4b44be020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44be020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 -- 172.21.15.66:0/2425650963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b44ba080 con 0x55d4b44b5a80 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f896effd640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.370+0000 7f8967fff640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b9880 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f89580059f0 tx=0x7f8958015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3d27b5a101623e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f894effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f894effd640 1 -- 172.21.15.66:0/2425650963 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958019070 con 0x55d4b44b5a80 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f894effd640 1 -- 172.21.15.66:0/2425650963 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8958005380 con 0x55d4b44b5a80 2024-04-21T09:10:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896002c7e0 con 0x55d4b44b5a80 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f894effd640 1 -- 172.21.15.66:0/2425650963 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8958042050 con 0x55d4b44b5a80 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 10 client.16509 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4b452bd20 con 0x55d4b44b5a80 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f894effd640 1 -- 172.21.15.66:0/2425650963 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896002c7e0 con 0x55d4b44b5a80 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f894effd640 1 -- 172.21.15.66:0/2425650963 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89580049d0 con 0x55d4b44b5a80 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 10 client.16509 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 20 client.16509 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b452c410 0x55d4b452e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 4 client.16509 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.371+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4b4418a70 con 0x55d4b452c410 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f896effd640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b452c410 0x55d4b452e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f894effd640 10 client.16509 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f896effd640 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b452c410 0x55d4b452e830 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41ba4369509175f server_cookie=94efd795c8a877ad in_seq=0 out_seq=0 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f894effd640 1 -- 172.21.15.66:0/2425650963 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d4b4418a70 con 0x55d4b452c410 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f894effd640 10 client.16509 handle_command_reply: tid=0 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f8978470740 1 client.16509 shutdown 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 msgr2=0x55d4b44b9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.372+0000 7f8978470740 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b9880 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f89580059f0 tx=0x7f8958015040 comp rx=0 tx=0).stop 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 shutdown_connections 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55d4b452c410 0x55d4b452e830 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4b44b5a80 0x55d4b44b9880 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f894effd640 0 client.16509 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4b44acea0 0x55d4b44b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 --2- 172.21.15.66:0/2425650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4b44ac580 0x55d4b44be020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 >> 172.21.15.66:0/2425650963 conn(0x55d4b44a6cb0 msgr2=0x55d4b44b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 shutdown_connections 2024-04-21T09:10:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 1 -- 172.21.15.66:0/2425650963 wait complete. 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8978470740 20 client.16509 trim_cache size 0 max 16384 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: "5a97f62b-a2a2-4905-b8ae-c66c5bb256ff": { 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "5a97f62b-a2a2-4905-b8ae-c66c5bb256ff", 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:10:27.380 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:10:27.381 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:10:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f895c0a3d20 msgr2=0x7f895c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f895c0a3d20 0x7f895c0a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8968062740 tx=0x7f896806b790 comp rx=0 tx=0).stop 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 msgr2=0x7f8970101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970101bf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f896800eb10 tx=0x7f896800c2d0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 shutdown_connections 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f895c0a3d20 0x7f895c0a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8970108060 0x7f8970101bf0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f897006af70 0x7f89701016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 --2- 172.21.15.66:0/2747925369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f897006a5b0 0x7f8970101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 >> 172.21.15.66:0/2747925369 conn(0x7f8970077c40 msgr2=0x7f89700ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 shutdown_connections 2024-04-21T09:10:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:27.373+0000 7f8975881640 1 -- 172.21.15.66:0/2747925369 wait complete. 2024-04-21T09:10:27.385 INFO:tasks.cephfs.filesystem:scrub status for tag:5a97f62b-a2a2-4905-b8ae-c66c5bb256ff - {'path': '/', 'tag': '5a97f62b-a2a2-4905-b8ae-c66c5bb256ff', 'options': 'recursive,force'} 2024-04-21T09:10:27.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:27 smithi186 ceph-mon[31505]: pgmap v70: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6.3 MiB/s wr, 239 op/s 2024-04-21T09:10:27.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:27 smithi083 ceph-mon[31413]: pgmap v70: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6.3 MiB/s wr, 239 op/s 2024-04-21T09:10:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:28 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:10:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:28 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:10:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:28 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:10:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:28 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:10:28.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:28 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:10:28.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:28 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:10:28.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:28 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:10:28.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:28 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:10:28.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:28 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:10:28.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:28 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:10:28.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:28 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:10:28.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:28 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:10:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:29 smithi186 ceph-mon[31505]: pgmap v71: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 243 op/s 2024-04-21T09:10:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:29 smithi083 ceph-mon[31413]: pgmap v71: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 243 op/s 2024-04-21T09:10:29.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:29 smithi066 ceph-mon[27953]: pgmap v71: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 243 op/s 2024-04-21T09:10:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:31 smithi186 ceph-mon[31505]: pgmap v72: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-04-21T09:10:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:31 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:10:31.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:31 smithi083 ceph-mon[31413]: pgmap v72: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-04-21T09:10:31.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:31 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:10:31.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:31 smithi066 ceph-mon[27953]: pgmap v72: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-04-21T09:10:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:31 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:10:33.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:33 smithi186 ceph-mon[31505]: pgmap v73: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-04-21T09:10:33.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:33 smithi083 ceph-mon[31413]: pgmap v73: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-04-21T09:10:33.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:33 smithi066 ceph-mon[27953]: pgmap v73: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-04-21T09:10:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:35 smithi186 ceph-mon[31505]: pgmap v74: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-21T09:10:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:35 smithi083 ceph-mon[31413]: pgmap v74: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-21T09:10:35.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:35 smithi066 ceph-mon[27953]: pgmap v74: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-21T09:10:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:37 smithi186 ceph-mon[31505]: pgmap v75: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-21T09:10:37.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:37 smithi083 ceph-mon[31413]: pgmap v75: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-21T09:10:37.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:37 smithi066 ceph-mon[27953]: pgmap v75: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-21T09:10:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:39 smithi186 ceph-mon[31505]: pgmap v76: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-21T09:10:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:39 smithi083 ceph-mon[31413]: pgmap v76: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-21T09:10:39.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:39 smithi066 ceph-mon[27953]: pgmap v76: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-21T09:10:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:41 smithi186 ceph-mon[31505]: pgmap v77: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:41.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:41 smithi083 ceph-mon[31413]: pgmap v77: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:41.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:41 smithi066 ceph-mon[27953]: pgmap v77: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:43.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:43 smithi186 ceph-mon[31505]: pgmap v78: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:43.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:43 smithi083 ceph-mon[31413]: pgmap v78: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:43.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:43 smithi066 ceph-mon[27953]: pgmap v78: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:45 smithi186 ceph-mon[31505]: pgmap v79: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:45 smithi083 ceph-mon[31413]: pgmap v79: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:45 smithi066 ceph-mon[27953]: pgmap v79: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:47 smithi186 ceph-mon[31505]: pgmap v80: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:47 smithi083 ceph-mon[31413]: pgmap v80: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:47 smithi066 ceph-mon[27953]: pgmap v80: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:49 smithi186 ceph-mon[31505]: pgmap v81: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:49.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:49 smithi083 ceph-mon[31413]: pgmap v81: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:49 smithi066 ceph-mon[27953]: pgmap v81: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:51 smithi186 ceph-mon[31505]: pgmap v82: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:51.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:51 smithi083 ceph-mon[31413]: pgmap v82: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:51.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:51 smithi066 ceph-mon[27953]: pgmap v82: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:53 smithi186 ceph-mon[31505]: pgmap v83: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:53.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:53 smithi083 ceph-mon[31413]: pgmap v83: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:53 smithi066 ceph-mon[27953]: pgmap v83: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:55 smithi186 ceph-mon[31505]: pgmap v84: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:55 smithi083 ceph-mon[31413]: pgmap v84: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:55 smithi066 ceph-mon[27953]: pgmap v84: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:57.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:10:57.386 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:10:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 -- 172.21.15.66:0/1664376865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd0106f20 msgr2=0x7f2cd0107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/1664376865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd0106f20 0x7f2cd0107300 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009900 tx=0x7f2cc002bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 -- 172.21.15.66:0/1664376865 shutdown_connections 2024-04-21T09:10:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/1664376865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd006af70 0x7f2cd010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/1664376865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd006a5b0 0x7f2cd006aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/1664376865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd0106f20 0x7f2cd0107300 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 -- 172.21.15.66:0/1664376865 >> 172.21.15.66:0/1664376865 conn(0x7f2cd0077c40 msgr2=0x7f2cd0078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 -- 172.21.15.66:0/1664376865 shutdown_connections 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 -- 172.21.15.66:0/1664376865 wait complete. 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 Processor -- start 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.506+0000 7f2cd66b3640 1 -- start start 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2cd66b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd006a5b0 0x7f2cd0101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2cd66b3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd006af70 0x7f2cd01016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2cd66b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd0106f20 0x7f2cd0101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2cd66b3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01121f0 con 0x7f2cd006af70 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2cd66b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0112070 con 0x7f2cd0106f20 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2cd66b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0112370 con 0x7f2cd006a5b0 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd006af70 0x7f2cd01016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd006af70 0x7f2cd01016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34206/0 (socket says 172.21.15.66:34206) 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 -- 172.21.15.66:0/2239926531 learned_addr learned my addr 172.21.15.66:0/2239926531 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd0106f20 0x7f2cd0101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccffff640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd006a5b0 0x7f2cd0101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 -- 172.21.15.66:0/2239926531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd006a5b0 msgr2=0x7f2cd0101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd006a5b0 0x7f2cd0101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 -- 172.21.15.66:0/2239926531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd0106f20 msgr2=0x7f2cd0101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd0106f20 0x7f2cd0101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.507+0000 7f2ccf7fe640 1 -- 172.21.15.66:0/2239926531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd01a6c10 con 0x7f2cd006af70 2024-04-21T09:10:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.508+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd006af70 0x7f2cd01016b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800c890 tx=0x7f2cb800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.508+0000 7f2ccd7fa640 1 -- 172.21.15.66:0/2239926531 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004220 con 0x7f2cd006af70 2024-04-21T09:10:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.508+0000 7f2ccd7fa640 1 -- 172.21.15.66:0/2239926531 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cb80043c0 con 0x7f2cd006af70 2024-04-21T09:10:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.508+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd01a6f00 con 0x7f2cd006af70 2024-04-21T09:10:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.508+0000 7f2ccd7fa640 1 -- 172.21.15.66:0/2239926531 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004a00 con 0x7f2cd006af70 2024-04-21T09:10:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.508+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd01a73c0 con 0x7f2cd006af70 2024-04-21T09:10:57.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.510+0000 7f2ccd7fa640 1 -- 172.21.15.66:0/2239926531 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2cb80079b0 con 0x7f2cd006af70 2024-04-21T09:10:57.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.510+0000 7f2cd9380740 1 Processor -- start 2024-04-21T09:10:57.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.510+0000 7f2cd9380740 1 -- start start 2024-04-21T09:10:57.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bbfd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc06540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc08e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd9380740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560d4bb621e0 con 0x560d4bbfd580 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd9380740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d4baff390 con 0x560d4bc06a80 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd9380740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560d4bb5e1a0 con 0x560d4bbfdea0 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2ccd7fa640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bbfd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bbfd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34222/0 (socket says 172.21.15.66:34222) 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 -- 172.21.15.66:0/2607261369 learned_addr learned my addr 172.21.15.66:0/2607261369 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2ccd7fa640 1 -- 172.21.15.66:0/2239926531 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2cb80d0810 con 0x7f2cd006af70 2024-04-21T09:10:57.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc08e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2ccffff640 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc06540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 -- 172.21.15.66:0/2607261369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 msgr2=0x560d4bc06540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc06540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 -- 172.21.15.66:0/2607261369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 msgr2=0x560d4bc08e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc08e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 -- 172.21.15.66:0/2607261369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4b93e7d0 con 0x560d4bbfd580 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc08e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bbfd960 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8008340 tx=0x7f2cc8008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89aa54ec74b3ea9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.511+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009900 tx=0x7f2cc801b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2607261369 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8027070 con 0x560d4bbfd580 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2607261369 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc8019d30 con 0x560d4bbfd580 2024-04-21T09:10:57.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2607261369 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8014040 con 0x560d4bbfd580 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 -- 172.21.15.66:0/2607261369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 msgr2=0x560d4bbfd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bbfd960 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8008340 tx=0x7f2cc8008800 comp rx=0 tx=0).stop 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 -- 172.21.15.66:0/2607261369 shutdown_connections 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc08e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc06540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 --2- 172.21.15.66:0/2607261369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bbfd960 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 -- 172.21.15.66:0/2607261369 >> 172.21.15.66:0/2607261369 conn(0x560d4bbf7cb0 msgr2=0x560d4bbf8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 -- 172.21.15.66:0/2607261369 shutdown_connections 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.512+0000 7f2cd9380740 1 -- 172.21.15.66:0/2607261369 wait complete. 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 -- start start 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 Processor -- start 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc03ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc03ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc04530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560d4bc09810 con 0x560d4bbfd580 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d4bb1b2f0 con 0x560d4bbfdea0 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560d4bc09990 con 0x560d4bc06a80 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2cd9380740 10 client.0 _ll_register_callbacks cb 0x560d4bb28580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2ccffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc03ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2ccffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc03ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34224/0 (socket says 172.21.15.66:34224) 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2ccffff640 1 -- 172.21.15.66:0/507399891 learned_addr learned my addr 172.21.15.66:0/507399891 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.513+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc03ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc04530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2ccffff640 1 -- 172.21.15.66:0/507399891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 msgr2=0x560d4bc04530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2ccffff640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc04530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2ccffff640 1 -- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 msgr2=0x560d4bc03ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2ccffff640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc03ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2ccffff640 1 -- 172.21.15.66:0/507399891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4b8743f0 con 0x560d4bbfd580 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc04530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2ccffff640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc03ab0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0103410 tx=0x7f2cc00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e252e13bd55fb8b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2caf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2caf7fe640 1 -- 172.21.15.66:0/507399891 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0046070 con 0x560d4bbfd580 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2caf7fe640 1 -- 172.21.15.66:0/507399891 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2cc0005250 con 0x560d4bbfd580 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc8019d30 con 0x560d4bbfd580 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2caf7fe640 1 -- 172.21.15.66:0/507399891 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cc0005450 con 0x560d4bbfd580 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2cd9380740 10 client.16521 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:57.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d4b8340f0 con 0x560d4bbfd580 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.514+0000 7f2caf7fe640 1 -- 172.21.15.66:0/507399891 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc8019d30 con 0x560d4bbfd580 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2caf7fe640 1 -- 172.21.15.66:0/507399891 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc00035f0 con 0x560d4bbfd580 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2cd9380740 10 client.16521 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2cd9380740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2cd9380740 20 client.16521 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2cd9380740 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc169b0 0x560d4bc18e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2cd9380740 4 client.16521 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d4bb6fc00 con 0x560d4bc169b0 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc169b0 0x560d4bc18e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2caf7fe640 10 client.16521 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:57.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.515+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc169b0 0x560d4bc18e00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8b5adda5c9f735 server_cookie=c36cfcd042e2dd12 in_seq=0 out_seq=0 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2caf7fe640 1 -- 172.21.15.66:0/507399891 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d4bb6fc00 con 0x560d4bc169b0 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2caf7fe640 10 client.16521 handle_command_reply: tid=0 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 client.16521 shutdown 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 msgr2=0x560d4bc03ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc03ab0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0103410 tx=0x7f2cc00042e0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 shutdown_connections 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc169b0 0x560d4bc18e00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc04530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc03ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2caf7fe640 0 client.16521 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 --2- 172.21.15.66:0/507399891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc03ab0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 >> 172.21.15.66:0/507399891 conn(0x560d4bbf7cb0 msgr2=0x560d4bbf9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 shutdown_connections 2024-04-21T09:10:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 1 -- 172.21.15.66:0/507399891 wait complete. 2024-04-21T09:10:57.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.516+0000 7f2cd9380740 20 client.16521 trim_cache size 0 max 16384 2024-04-21T09:10:57.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 Processor -- start 2024-04-21T09:10:57.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 -- start start 2024-04-21T09:10:57.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc02c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01b3e10 con 0x560d4bbfd580 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01b16e0 con 0x560d4bbfdea0 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2cd9380740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01af510 con 0x560d4bc06a80 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2ccffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2ccffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34230/0 (socket says 172.21.15.66:34230) 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.534+0000 7f2ccffff640 1 -- 172.21.15.66:0/1260890324 learned_addr learned my addr 172.21.15.66:0/1260890324 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccffff640 1 -- 172.21.15.66:0/1260890324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 msgr2=0x560d4bc02c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccffff640 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc02c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccffff640 1 -- 172.21.15.66:0/1260890324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 msgr2=0x560d4bc126d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccffff640 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc126d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccffff640 1 -- 172.21.15.66:0/1260890324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4bbf36d0 con 0x560d4bbfd580 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2ccffff640 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc061e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005d00 tx=0x7f2cc0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f799f7c15c593e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2cc5ffb640 1 -- 172.21.15.66:0/1260890324 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0046070 con 0x560d4bbfd580 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2cc5ffb640 1 -- 172.21.15.66:0/1260890324 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc002cbc0 con 0x560d4bbfd580 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.535+0000 7f2cc5ffb640 1 -- 172.21.15.66:0/1260890324 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0000e60 con 0x560d4bbfd580 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 -- 172.21.15.66:0/1260890324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 msgr2=0x560d4bc061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc061e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005d00 tx=0x7f2cc0005a20 comp rx=0 tx=0).stop 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 -- 172.21.15.66:0/1260890324 shutdown_connections 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bc06a80 0x560d4bc02c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bbfdea0 0x560d4bc126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 --2- 172.21.15.66:0/1260890324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc061e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 -- 172.21.15.66:0/1260890324 >> 172.21.15.66:0/1260890324 conn(0x560d4bbf7cb0 msgr2=0x560d4bbf64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 -- 172.21.15.66:0/1260890324 shutdown_connections 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 -- 172.21.15.66:0/1260890324 wait complete. 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 -- start start 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.536+0000 7f2cd9380740 1 Processor -- start 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc0a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc0aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc0afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01afba0 con 0x560d4bbfd580 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01b1010 con 0x560d4bc06a80 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01b0270 con 0x560d4bbfdea0 2024-04-21T09:10:57.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 10 client.0 _ll_register_callbacks cb 0x7f2cc0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc0a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc0a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34240/0 (socket says 172.21.15.66:34240) 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 -- 172.21.15.66:0/4214579728 learned_addr learned my addr 172.21.15.66:0/4214579728 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc0afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc0aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 -- 172.21.15.66:0/4214579728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 msgr2=0x560d4bc0aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc0aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 -- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 msgr2=0x560d4bc0afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc0afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 -- 172.21.15.66:0/4214579728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4bc7bf40 con 0x560d4bbfd580 2024-04-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd4c50640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc0afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2ccffff640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc0a520 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0000c00 tx=0x7f2cc0005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edac7a19f65a5f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2caf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2caf7fe640 1 -- 172.21.15.66:0/4214579728 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0046040 con 0x560d4bbfd580 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2caf7fe640 1 -- 172.21.15.66:0/4214579728 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2cc00048d0 con 0x560d4bbfd580 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.537+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc002cbc0 con 0x560d4bbfd580 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2caf7fe640 1 -- 172.21.15.66:0/4214579728 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cc0000e40 con 0x560d4bbfd580 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 10 client.16527 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d4bc7d160 con 0x560d4bbfd580 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2caf7fe640 1 -- 172.21.15.66:0/4214579728 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc002cbc0 con 0x560d4bbfd580 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2caf7fe640 1 -- 172.21.15.66:0/4214579728 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc0040aa0 con 0x560d4bbfd580 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 10 client.16527 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 20 client.16527 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc7d680 0x560d4bc7fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 4 client.16527 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d4bb69a70 con 0x560d4bc7d680 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.538+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc7d680 0x560d4bc7fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2ccf7fe640 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc7d680 0x560d4bc7fa70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a76f68d268e1a9d server_cookie=d12e97de4f3228e0 in_seq=0 out_seq=0 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2caf7fe640 10 client.16527 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2caf7fe640 1 -- 172.21.15.66:0/4214579728 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d4bb69a70 con 0x560d4bc7d680 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2caf7fe640 10 client.16527 handle_command_reply: tid=0 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 client.16527 shutdown 2024-04-21T09:10:57.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 msgr2=0x560d4bc0a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc0a520 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0000c00 tx=0x7f2cc0005820 comp rx=0 tx=0).stop 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 shutdown_connections 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560d4bc7d680 0x560d4bc7fa70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d4bc06a80 0x560d4bc0afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560d4bbfdea0 0x560d4bc0aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2caf7fe640 0 client.16527 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 --2- 172.21.15.66:0/4214579728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560d4bbfd580 0x560d4bc0a520 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 >> 172.21.15.66:0/4214579728 conn(0x560d4bbf7cb0 msgr2=0x560d4bbf64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 shutdown_connections 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 1 -- 172.21.15.66:0/4214579728 wait complete. 2024-04-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.539+0000 7f2cd9380740 20 client.16527 trim_cache size 0 max 16384 2024-04-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 msgr2=0x7f2cbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009900 tx=0x7f2cc801b000 comp rx=0 tx=0).stop 2024-04-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd006af70 msgr2=0x7f2cd01016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd006af70 0x7f2cd01016b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800c890 tx=0x7f2cb800cd50 comp rx=0 tx=0).stop 2024-04-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 shutdown_connections 2024-04-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd0106f20 0x7f2cd0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd006af70 0x7f2cd01016b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 --2- 172.21.15.66:0/2239926531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd006a5b0 0x7f2cd0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 >> 172.21.15.66:0/2239926531 conn(0x7f2cd0077c40 msgr2=0x7f2cd00feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 shutdown_connections 2024-04-21T09:10:57.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.540+0000 7f2cd66b3640 1 -- 172.21.15.66:0/2239926531 wait complete. 2024-04-21T09:10:57.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:10:57.551 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:10:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:57 smithi186 ceph-mon[31505]: pgmap v85: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:57.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:57 smithi083 ceph-mon[31413]: pgmap v85: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:57 smithi066 ceph-mon[27953]: pgmap v85: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:10:57.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.654+0000 7f46d68fb640 1 -- 172.21.15.66:0/3765162767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0101ad0 msgr2=0x7f46d010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.654+0000 7f46d68fb640 1 --2- 172.21.15.66:0/3765162767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0101ad0 0x7f46d010dcb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009ca0 tx=0x7f46c802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.654+0000 7f46d68fb640 1 -- 172.21.15.66:0/3765162767 shutdown_connections 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.654+0000 7f46d68fb640 1 --2- 172.21.15.66:0/3765162767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0101ad0 0x7f46d010dcb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.654+0000 7f46d68fb640 1 --2- 172.21.15.66:0/3765162767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d0101130 0x7f46d0101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.654+0000 7f46d68fb640 1 --2- 172.21.15.66:0/3765162767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0106f10 0x7f46d01072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.654+0000 7f46d68fb640 1 -- 172.21.15.66:0/3765162767 >> 172.21.15.66:0/3765162767 conn(0x7f46d0077e30 msgr2=0x7f46d0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.655+0000 7f46d68fb640 1 -- 172.21.15.66:0/3765162767 shutdown_connections 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.655+0000 7f46d68fb640 1 -- 172.21.15.66:0/3765162767 wait complete. 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.655+0000 7f46d68fb640 1 Processor -- start 2024-04-21T09:10:57.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.655+0000 7f46d68fb640 1 -- start start 2024-04-21T09:10:57.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.655+0000 7f46d68fb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d0101130 0x7f46d0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.655+0000 7f46d68fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0101ad0 0x7f46d0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d68fb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0106f10 0x7f46d019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d68fb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01110d0 con 0x7f46d0106f10 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d68fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0110f50 con 0x7f46d0101ad0 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d68fb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0111250 con 0x7f46d0101130 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0106f10 0x7f46d019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0106f10 0x7f46d019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34260/0 (socket says 172.21.15.66:34260) 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 -- 172.21.15.66:0/1301104067 learned_addr learned my addr 172.21.15.66:0/1301104067 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46cf7fe640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0101ad0 0x7f46d0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46cffff640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d0101130 0x7f46d0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 -- 172.21.15.66:0/1301104067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d0101130 msgr2=0x7f46d0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d0101130 0x7f46d0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 -- 172.21.15.66:0/1301104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0101ad0 msgr2=0x7f46d0198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0101ad0 0x7f46d0198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 -- 172.21.15.66:0/1301104067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d019d730 con 0x7f46d0106f10 2024-04-21T09:10:57.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d4e98640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0106f10 0x7f46d019d000 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f46c8005e30 tx=0x7f46c8032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46cd7fa640 1 -- 172.21.15.66:0/1301104067 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8041020 con 0x7f46d0106f10 2024-04-21T09:10:57.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d019d9c0 con 0x7f46d0106f10 2024-04-21T09:10:57.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46cd7fa640 1 -- 172.21.15.66:0/1301104067 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c8032970 con 0x7f46d0106f10 2024-04-21T09:10:57.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46cd7fa640 1 -- 172.21.15.66:0/1301104067 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8039040 con 0x7f46d0106f10 2024-04-21T09:10:57.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.656+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01a52b0 con 0x7f46d0106f10 2024-04-21T09:10:57.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.658+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d0069230 con 0x7f46d0106f10 2024-04-21T09:10:57.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.659+0000 7f46cd7fa640 1 -- 172.21.15.66:0/1301104067 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f46c8044020 con 0x7f46d0106f10 2024-04-21T09:10:57.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.659+0000 7f46cd7fa640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f46b80a3d20 0x7f46b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:57.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.659+0000 7f46cffff640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f46b80a3d20 0x7f46b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:57.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.659+0000 7f46cd7fa640 1 -- 172.21.15.66:0/1301104067 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f46c80f2740 con 0x7f46d0106f10 2024-04-21T09:10:57.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.660+0000 7f46cffff640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f46b80a3d20 0x7f46b80a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009b50 tx=0x7f46c4005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:57.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.664+0000 7f46cd7fa640 1 -- 172.21.15.66:0/1301104067 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46c804b020 con 0x7f46d0106f10 2024-04-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.888+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d01a5480 con 0x7f46d0106f10 2024-04-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.888+0000 7f46cd7fa640 1 -- 172.21.15.66:0/1301104067 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f46c80beb80 con 0x7f46d0106f10 2024-04-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.890+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f46b80a3d20 msgr2=0x7f46b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.890+0000 7f46d68fb640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f46b80a3d20 0x7f46b80a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009b50 tx=0x7f46c4005f90 comp rx=0 tx=0).stop 2024-04-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.890+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0106f10 msgr2=0x7f46d019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.890+0000 7f46d68fb640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0106f10 0x7f46d019d000 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f46c8005e30 tx=0x7f46c8032420 comp rx=0 tx=0).stop 2024-04-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 shutdown_connections 2024-04-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f46b80a3d20 0x7f46b80a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d0106f10 0x7f46d019d000 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0101ad0 0x7f46d0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 --2- 172.21.15.66:0/1301104067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d0101130 0x7f46d0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 >> 172.21.15.66:0/1301104067 conn(0x7f46d0077e30 msgr2=0x7f46d006a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 shutdown_connections 2024-04-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:57.891+0000 7f46d68fb640 1 -- 172.21.15.66:0/1301104067 wait complete. 2024-04-21T09:10:57.901 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/1623186803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd006b1f0 msgr2=0x7f4bd0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/1623186803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd006b1f0 0x7f4bd0106b90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009900 tx=0x7f4bb802bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/1623186803 shutdown_connections 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/1623186803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0107270 0x7f4bd010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/1623186803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd006b1f0 0x7f4bd0106b90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/1623186803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd006a840 0x7f4bd006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/1623186803 >> 172.21.15.66:0/1623186803 conn(0x7f4bd0077eb0 msgr2=0x7f4bd00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/1623186803 shutdown_connections 2024-04-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.007+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/1623186803 wait complete. 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 Processor -- start 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 -- start start 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd006a840 0x7f4bd0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd0107270 0x7f4bd01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0101900 0x7f4bd01a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0112190 con 0x7f4bd0107270 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0112010 con 0x7f4bd0101900 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bd4b4b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0112310 con 0x7f4bd006a840 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd0107270 0x7f4bd01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd0107270 0x7f4bd01013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34276/0 (socket says 172.21.15.66:34276) 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.008+0000 7f4bc7fff640 1 -- 172.21.15.66:0/2044151865 learned_addr learned my addr 172.21.15.66:0/2044151865 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bced9d640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0101900 0x7f4bd01a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bce59c640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd006a840 0x7f4bd0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc7fff640 1 -- 172.21.15.66:0/2044151865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd006a840 msgr2=0x7f4bd0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd006a840 0x7f4bd0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc7fff640 1 -- 172.21.15.66:0/2044151865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0101900 msgr2=0x7f4bd01a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0101900 0x7f4bd01a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc7fff640 1 -- 172.21.15.66:0/2044151865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd01a6fa0 con 0x7f4bd0107270 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bce59c640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd006a840 0x7f4bd0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bced9d640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0101900 0x7f4bd01a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd0107270 0x7f4bd01013c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8036040 tx=0x7f4bb80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc77fe640 1 -- 172.21.15.66:0/2044151865 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8043020 con 0x7f4bd0107270 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc77fe640 1 -- 172.21.15.66:0/2044151865 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb802cb70 con 0x7f4bd0107270 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd01a7230 con 0x7f4bd0107270 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bc77fe640 1 -- 172.21.15.66:0/2044151865 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8003bf0 con 0x7f4bd0107270 2024-04-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.009+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd01a7780 con 0x7f4bd0107270 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.011+0000 7f4bc77fe640 1 -- 172.21.15.66:0/2044151865 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4bb8046020 con 0x7f4bd0107270 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 Processor -- start 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 -- start start 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba7391f0 con 0x55e9ba7dda00 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba6d64b0 con 0x55e9ba7d44d0 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bd7813740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba7351b0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bc77fe640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4bbc0a3d50 0x7f4bbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bced9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bced9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:48800/0 (socket says 172.21.15.66:48800) 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34288/0 (socket says 172.21.15.66:34288) 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bced9d640 1 -- 172.21.15.66:0/3058019676 learned_addr learned my addr 172.21.15.66:0/3058019676 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bced9d640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4bbc0a3d50 0x7f4bbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bce59c640 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bc77fe640 1 -- 172.21.15.66:0/2044151865 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4bb80f2ca0 con 0x7f4bd0107270 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.012+0000 7f4bced9d640 1 -- 172.21.15.66:0/3058019676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7d4e80 msgr2=0x55e9ba7dd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bced9d640 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7dd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bced9d640 1 -- 172.21.15.66:0/3058019676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7dda00 msgr2=0x55e9ba7dfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bced9d640 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bced9d640 1 -- 172.21.15.66:0/3058019676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ba6c81b0 con 0x55e9ba7d44d0 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bced9d640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4bbc0a3d50 0x7f4bbc0a6210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8010ca0 tx=0x7f4bc801d000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bced9d640 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7d48b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8012d90 tx=0x7f4bc80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ffc20fbe3e58e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bc4ff9640 1 -- 172.21.15.66:0/3058019676 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8012580 con 0x55e9ba7d44d0 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bc4ff9640 1 -- 172.21.15.66:0/3058019676 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bc8012870 con 0x55e9ba7d44d0 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.013+0000 7f4bc4ff9640 1 -- 172.21.15.66:0/3058019676 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8005410 con 0x55e9ba7d44d0 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 -- 172.21.15.66:0/3058019676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 msgr2=0x55e9ba7d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7d48b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8012d90 tx=0x7f4bc80107a0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 -- 172.21.15.66:0/3058019676 shutdown_connections 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7dd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3058019676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7d48b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 -- 172.21.15.66:0/3058019676 >> 172.21.15.66:0/3058019676 conn(0x55e9ba7ceba0 msgr2=0x55e9ba7cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 -- 172.21.15.66:0/3058019676 shutdown_connections 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 -- 172.21.15.66:0/3058019676 wait complete. 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 -- start start 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.014+0000 7f4bd7813740 1 Processor -- start 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba7e08b0 con 0x55e9ba7d44d0 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba6f2500 con 0x55e9ba7d4e80 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba7e0a30 con 0x55e9ba7dda00 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 10 client.0 _ll_register_callbacks cb 0x55e9ba6c0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7dd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34292/0 (socket says 172.21.15.66:34292) 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 -- 172.21.15.66:0/3743210545 learned_addr learned my addr 172.21.15.66:0/3743210545 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bced9d640 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 -- 172.21.15.66:0/3743210545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 msgr2=0x55e9ba7dad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 -- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d4e80 msgr2=0x55e9ba7da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 -- 172.21.15.66:0/3743210545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ba44b3f0 con 0x55e9ba7d44d0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bce59c640 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7dd360 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000f6f0 tx=0x7f4bc000faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf31fc007893c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/3743210545 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0016070 con 0x55e9ba7d44d0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/3743210545 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4bc000ba60 con 0x55e9ba7d44d0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.015+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bc8012870 con 0x55e9ba7d44d0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/3743210545 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bc0025050 con 0x55e9ba7d44d0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 10 client.16545 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9ba5157d0 con 0x55e9ba7d44d0 2024-04-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/3743210545 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bc8012870 con 0x55e9ba7d44d0 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/3743210545 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4bc001a400 con 0x55e9ba7d44d0 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 10 client.16545 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 10 client.16545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 20 client.16545 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba7edc20 0x55e9ba7f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 4 client.16545 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9ba6d8930 con 0x55e9ba7edc20 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.016+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba7edc20 0x55e9ba7f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bb5ffb640 10 client.16545 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba7edc20 0x55e9ba7f00c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4afb1ce52c60dcb4 server_cookie=1e31f9e04d507d05 in_seq=0 out_seq=0 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/3743210545 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9ba6d8930 con 0x55e9ba7edc20 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bb5ffb640 10 client.16545 handle_command_reply: tid=0 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 client.16545 shutdown 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d44d0 msgr2=0x55e9ba7dd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7dd360 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000f6f0 tx=0x7f4bc000faf0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 shutdown_connections 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba7edc20 0x55e9ba7f00c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7dad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 --2- 172.21.15.66:0/3743210545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7dd360 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bb5ffb640 0 client.16545 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.017+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 >> 172.21.15.66:0/3743210545 conn(0x55e9ba7ceba0 msgr2=0x55e9ba7d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.018+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 shutdown_connections 2024-04-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.018+0000 7f4bd7813740 1 -- 172.21.15.66:0/3743210545 wait complete. 2024-04-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.018+0000 7f4bd7813740 20 client.16545 trim_cache size 0 max 16384 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 Processor -- start 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 -- start start 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01b3cd0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01b15a0 con 0x55e9ba7d44d0 2024-04-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bd7813740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01af3d0 con 0x55e9ba7dda00 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34300/0 (socket says 172.21.15.66:34300) 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bc7fff640 1 -- 172.21.15.66:0/735551175 learned_addr learned my addr 172.21.15.66:0/735551175 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bce59c640 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bc7fff640 1 -- 172.21.15.66:0/735551175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 msgr2=0x55e9ba7cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.031+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bc7fff640 1 -- 172.21.15.66:0/735551175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 msgr2=0x55e9ba7e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bc7fff640 1 -- 172.21.15.66:0/735551175 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ba7db070 con 0x55e9ba7d4e80 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d9850 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80bb3b0 tx=0x7f4bb80df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca03cb3656424a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bb7fff640 1 -- 172.21.15.66:0/735551175 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80bf0e0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bb7fff640 1 -- 172.21.15.66:0/735551175 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb80be9e0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bb7fff640 1 -- 172.21.15.66:0/735551175 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80bb0c0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 -- 172.21.15.66:0/735551175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 msgr2=0x55e9ba7d9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d9850 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80bb3b0 tx=0x7f4bb80df040 comp rx=0 tx=0).stop 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 -- 172.21.15.66:0/735551175 shutdown_connections 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d9850 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 --2- 172.21.15.66:0/735551175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 -- 172.21.15.66:0/735551175 >> 172.21.15.66:0/735551175 conn(0x55e9ba7ceba0 msgr2=0x55e9ba7cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 -- 172.21.15.66:0/735551175 shutdown_connections 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.032+0000 7f4bd7813740 1 -- 172.21.15.66:0/735551175 wait complete. 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 -- start start 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 Processor -- start 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01afa60 con 0x55e9ba7d4e80 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01b0ed0 con 0x55e9ba7d44d0 2024-04-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01b0130 con 0x55e9ba7dda00 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bd7813740 10 client.0 _ll_register_callbacks cb 0x7f4bc000de00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34312/0 (socket says 172.21.15.66:34312) 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 -- 172.21.15.66:0/4291393378 learned_addr learned my addr 172.21.15.66:0/4291393378 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bced9d640 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bce59c640 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 -- 172.21.15.66:0/4291393378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 msgr2=0x55e9ba7d74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7d74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 -- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 msgr2=0x55e9ba7e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.033+0000 7f4bc7fff640 1 -- 172.21.15.66:0/4291393378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ba7d7a00 con 0x55e9ba7d4e80 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bc7fff640 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d6f80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80092a0 tx=0x7f4bb8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fcfe2e47371260 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/4291393378 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80bf0e0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/4291393378 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4bb80bb0c0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb80be9e0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/4291393378 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bb80f0300 con 0x55e9ba7d4e80 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 10 client.16551 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9ba853f90 con 0x55e9ba7d4e80 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/4291393378 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb80be9e0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/4291393378 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4bb80ddba0 con 0x55e9ba7d4e80 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 10 client.16551 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 10 client.16551 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 20 client.16551 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba8546b0 0x55e9ba856ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 4 client.16551 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.034+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9ba746c10 con 0x55e9ba8546b0 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bce59c640 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba8546b0 0x55e9ba856ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bb5ffb640 10 client.16551 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bce59c640 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba8546b0 0x55e9ba856ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ef5066c151e24be server_cookie=9b964b2fe1edb703 in_seq=0 out_seq=0 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bb5ffb640 1 -- 172.21.15.66:0/4291393378 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9ba746c10 con 0x55e9ba8546b0 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bb5ffb640 10 client.16551 handle_command_reply: tid=0 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 client.16551 shutdown 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 msgr2=0x55e9ba7d6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d6f80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80092a0 tx=0x7f4bb8005fd0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 shutdown_connections 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55e9ba8546b0 0x55e9ba856ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e9ba7dda00 0x55e9ba7d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9ba7d4e80 0x55e9ba7d6f80 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 --2- 172.21.15.66:0/4291393378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9ba7d44d0 0x55e9ba7e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bb5ffb640 0 client.16551 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 >> 172.21.15.66:0/4291393378 conn(0x55e9ba7ceba0 msgr2=0x55e9ba7d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 shutdown_connections 2024-04-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 1 -- 172.21.15.66:0/4291393378 wait complete. 2024-04-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.035+0000 7f4bd7813740 20 client.16551 trim_cache size 0 max 16384 2024-04-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4bbc0a3d50 msgr2=0x7f4bbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4bbc0a3d50 0x7f4bbc0a6210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8010ca0 tx=0x7f4bc801d000 comp rx=0 tx=0).stop 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd0107270 msgr2=0x7f4bd01013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd0107270 0x7f4bd01013c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8036040 tx=0x7f4bb80045e0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 shutdown_connections 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4bbc0a3d50 0x7f4bbc0a6210 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0101900 0x7f4bd01a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bd0107270 0x7f4bd01013c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 --2- 172.21.15.66:0/2044151865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd006a840 0x7f4bd0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 >> 172.21.15.66:0/2044151865 conn(0x7f4bd0077eb0 msgr2=0x7f4bd0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 shutdown_connections 2024-04-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.036+0000 7f4bd4b4b640 1 -- 172.21.15.66:0/2044151865 wait complete. 2024-04-21T09:10:58.047 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 -- 172.21.15.66:0/538813604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 msgr2=0x7feaa41035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/538813604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa41035b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009900 tx=0x7fea9c02bd60 comp rx=0 tx=0).stop 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 -- 172.21.15.66:0/538813604 shutdown_connections 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/538813604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa410c510 0x7feaa410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/538813604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa4103af0 0x7feaa410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/538813604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa41035b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 -- 172.21.15.66:0/538813604 >> 172.21.15.66:0/538813604 conn(0x7feaa4077f50 msgr2=0x7feaa4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 -- 172.21.15.66:0/538813604 shutdown_connections 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 -- 172.21.15.66:0/538813604 wait complete. 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 Processor -- start 2024-04-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.153+0000 7feaa8ee0640 1 -- start start 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa8ee0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa4100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa8ee0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa4103af0 0x7feaa4101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa8ee0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa410c510 0x7feaa41018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa8ee0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4112180 con 0x7feaa41031d0 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa8ee0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4112000 con 0x7feaa410c510 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa8ee0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4112300 con 0x7feaa4103af0 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa4100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa4100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34328/0 (socket says 172.21.15.66:34328) 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 -- 172.21.15.66:0/1211717204 learned_addr learned my addr 172.21.15.66:0/1211717204 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa4103af0 0x7feaa4101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 -- 172.21.15.66:0/1211717204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa4103af0 msgr2=0x7feaa4101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa4103af0 0x7feaa4101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa410c510 0x7feaa41018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 -- 172.21.15.66:0/1211717204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa410c510 msgr2=0x7feaa41018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa410c510 0x7feaa41018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 -- 172.21.15.66:0/1211717204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41a6c80 con 0x7feaa41031d0 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa259c640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa4100e20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005940 tx=0x7fea9c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7fea937fe640 1 -- 172.21.15.66:0/1211717204 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c043020 con 0x7feaa41031d0 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa41a6f10 con 0x7feaa41031d0 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7fea937fe640 1 -- 172.21.15.66:0/1211717204 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c02dc30 con 0x7feaa41031d0 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.154+0000 7fea937fe640 1 -- 172.21.15.66:0/1211717204 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c003a10 con 0x7feaa41031d0 2024-04-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.155+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41a7420 con 0x7feaa41031d0 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.157+0000 7feaabbaf740 1 Processor -- start 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.157+0000 7fea937fe640 1 -- 172.21.15.66:0/1211717204 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fea9c046020 con 0x7feaa41031d0 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaabbaf740 1 -- start start 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 0x5593b8664430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8666e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaabbaf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5593b85c01f0 con 0x5593b865b4d0 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaabbaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5593b855d4b0 con 0x5593b8664a00 2024-04-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaabbaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5593b85bc1b0 con 0x5593b865be80 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34336/0 (socket says 172.21.15.66:34336) 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 -- 172.21.15.66:0/2371644288 learned_addr learned my addr 172.21.15.66:0/2371644288 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa259c640 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 0x5593b8664430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8666e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7fea937fe640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea8c0a3d20 0x7fea8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 -- 172.21.15.66:0/2371644288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 msgr2=0x5593b8664430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 0x5593b8664430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 -- 172.21.15.66:0/2371644288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 msgr2=0x5593b8666e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8666e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.158+0000 7feaa2d9d640 1 -- 172.21.15.66:0/2371644288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b854f1b0 con 0x5593b865b4d0 2024-04-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.159+0000 7fea937fe640 1 -- 172.21.15.66:0/1211717204 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fea9c0f2c20 con 0x7feaa41031d0 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.159+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea8c0a3d20 0x7fea8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.159+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865b8b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fea94009df0 tx=0x7fea9400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92aae443d85e65e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.159+0000 7fea90ff9640 1 -- 172.21.15.66:0/2371644288 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9401e950 con 0x5593b865b4d0 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.159+0000 7fea90ff9640 1 -- 172.21.15.66:0/2371644288 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea94016070 con 0x5593b865b4d0 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.159+0000 7fea90ff9640 1 -- 172.21.15.66:0/2371644288 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9401db80 con 0x5593b865b4d0 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- 172.21.15.66:0/2371644288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 msgr2=0x5593b865b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea8c0a3d20 0x7fea8c0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea9400f010 tx=0x7fea94020000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865b8b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fea94009df0 tx=0x7fea9400e6c0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- 172.21.15.66:0/2371644288 shutdown_connections 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8666e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 0x5593b8664430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 --2- 172.21.15.66:0/2371644288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865b8b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- 172.21.15.66:0/2371644288 >> 172.21.15.66:0/2371644288 conn(0x5593b8655ba0 msgr2=0x5593b8655f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- 172.21.15.66:0/2371644288 shutdown_connections 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- 172.21.15.66:0/2371644288 wait complete. 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- start start 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 Processor -- start 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865b4d0 0x5593b8664280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865be80 0x5593b8661610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8661b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5593b8667970 con 0x5593b865be80 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5593b8579500 con 0x5593b8664a00 2024-04-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5593b8667af0 con 0x5593b865b4d0 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.160+0000 7feaabbaf740 10 client.0 _ll_register_callbacks cb 0x5593b8547b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865be80 0x5593b8661610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865be80 0x5593b8661610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34350/0 (socket says 172.21.15.66:34350) 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 -- 172.21.15.66:0/1788034066 learned_addr learned my addr 172.21.15.66:0/1788034066 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa259c640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865b4d0 0x5593b8664280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8661b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 -- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865b4d0 msgr2=0x5593b8664280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865b4d0 0x5593b8664280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 -- 172.21.15.66:0/1788034066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 msgr2=0x5593b8661b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8661b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 -- 172.21.15.66:0/1788034066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b82d23f0 con 0x5593b865be80 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa259c640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865b4d0 0x5593b8664280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865be80 0x5593b8661610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fea980126a0 tx=0x7fea98012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a338dc9c47564db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7fea89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7fea89ffb640 1 -- 172.21.15.66:0/1788034066 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9801f060 con 0x5593b865be80 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7fea89ffb640 1 -- 172.21.15.66:0/1788034066 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fea98010d60 con 0x5593b865be80 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea94016070 con 0x5593b865be80 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7fea89ffb640 1 -- 172.21.15.66:0/1788034066 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea98026050 con 0x5593b865be80 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaabbaf740 10 client.16563 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593b839c7d0 con 0x5593b865be80 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.161+0000 7fea89ffb640 1 -- 172.21.15.66:0/1788034066 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea94016070 con 0x5593b865be80 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7fea89ffb640 1 -- 172.21.15.66:0/1788034066 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea98019030 con 0x5593b865be80 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaabbaf740 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaabbaf740 10 client.16563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaabbaf740 20 client.16563 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b8674c30 0x5593b86770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaabbaf740 4 client.16563 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593b855f930 con 0x5593b8674c30 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaa259c640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b8674c30 0x5593b86770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7feaa259c640 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b8674c30 0x5593b86770b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95bd805ac1781072 server_cookie=930f98aa7da3c32d in_seq=0 out_seq=0 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.162+0000 7fea89ffb640 10 client.16563 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7fea89ffb640 1 -- 172.21.15.66:0/1788034066 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593b855f930 con 0x5593b8674c30 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7fea89ffb640 10 client.16563 handle_command_reply: tid=0 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 client.16563 shutdown 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865be80 msgr2=0x5593b8661610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865be80 0x5593b8661610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fea980126a0 tx=0x7fea98012b60 comp rx=0 tx=0).stop 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 shutdown_connections 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8661b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b8674c30 0x5593b86770b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865be80 0x5593b8661610 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1788034066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865b4d0 0x5593b8664280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7fea89ffb640 0 client.16563 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 >> 172.21.15.66:0/1788034066 conn(0x5593b8655ba0 msgr2=0x5593b86578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 shutdown_connections 2024-04-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 1 -- 172.21.15.66:0/1788034066 wait complete. 2024-04-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.163+0000 7feaabbaf740 20 client.16563 trim_cache size 0 max 16384 2024-04-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 Processor -- start 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 -- start start 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b8670830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 0x5593b8657680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8663d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41b3e40 con 0x5593b865b4d0 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41b1710 con 0x5593b8664a00 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaabbaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41af540 con 0x5593b865be80 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b8670830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b8670830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34358/0 (socket says 172.21.15.66:34358) 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 -- 172.21.15.66:0/529114319 learned_addr learned my addr 172.21.15.66:0/529114319 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8663d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 -- 172.21.15.66:0/529114319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 msgr2=0x5593b8657680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 0x5593b8657680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 -- 172.21.15.66:0/529114319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 msgr2=0x5593b8663d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8663d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 -- 172.21.15.66:0/529114319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b864ffe0 con 0x5593b865b4d0 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8663d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7feaa259c640 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b8670830 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0be8f0 tx=0x7fea9c0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8ed1b9029b09d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7fea8bfff640 1 -- 172.21.15.66:0/529114319 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0be960 con 0x5593b865b4d0 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7fea8bfff640 1 -- 172.21.15.66:0/529114319 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c0c2d10 con 0x5593b865b4d0 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.177+0000 7fea8bfff640 1 -- 172.21.15.66:0/529114319 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0bafd0 con 0x5593b865b4d0 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 -- 172.21.15.66:0/529114319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 msgr2=0x5593b8670830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b8670830 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0be8f0 tx=0x7fea9c0df040 comp rx=0 tx=0).stop 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 -- 172.21.15.66:0/529114319 shutdown_connections 2024-04-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b8664a00 0x5593b8663d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b865be80 0x5593b8657680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 --2- 172.21.15.66:0/529114319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b8670830 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 -- 172.21.15.66:0/529114319 >> 172.21.15.66:0/529114319 conn(0x5593b8655ba0 msgr2=0x5593b8656f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 -- 172.21.15.66:0/529114319 shutdown_connections 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 -- 172.21.15.66:0/529114319 wait complete. 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 -- start start 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.178+0000 7feaabbaf740 1 Processor -- start 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b865be80 0x5593b865eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b8664a00 0x5593b86da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41afbd0 con 0x5593b865b4d0 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41b1040 con 0x5593b865be80 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41b02a0 con 0x5593b8664a00 2024-04-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 10 client.0 _ll_register_callbacks cb 0x7fea98009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34374/0 (socket says 172.21.15.66:34374) 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 -- 172.21.15.66:0/1472213196 learned_addr learned my addr 172.21.15.66:0/1472213196 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b865be80 0x5593b865eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa2d9d640 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b8664a00 0x5593b86da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 -- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b8664a00 msgr2=0x5593b86da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b8664a00 0x5593b86da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 -- 172.21.15.66:0/1472213196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b865be80 msgr2=0x5593b865eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b865be80 0x5593b865eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 -- 172.21.15.66:0/1472213196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b86da730 con 0x5593b865b4d0 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaa259c640 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865e650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0be820 tx=0x7fea9c0f2f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e365d878d95088d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7fea89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7fea89ffb640 1 -- 172.21.15.66:0/1472213196 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0be960 con 0x5593b865b4d0 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7fea89ffb640 1 -- 172.21.15.66:0/1472213196 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fea9c0bafd0 con 0x5593b865b4d0 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c0c2d10 con 0x5593b865b4d0 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7fea89ffb640 1 -- 172.21.15.66:0/1472213196 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea9c0c75d0 con 0x5593b865b4d0 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 10 client.16569 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.179+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593b86db7e0 con 0x5593b865b4d0 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7fea89ffb640 1 -- 172.21.15.66:0/1472213196 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c0c2d10 con 0x5593b865b4d0 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7fea89ffb640 1 -- 172.21.15.66:0/1472213196 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea9c003bb0 con 0x5593b865b4d0 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaabbaf740 10 client.16569 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaabbaf740 10 client.16569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaabbaf740 20 client.16569 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b86dbea0 0x5593b86de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaabbaf740 4 client.16569 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593b85cdc10 con 0x5593b86dbea0 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b86dbea0 0x5593b86de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7feaa1d9b640 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b86dbea0 0x5593b86de2c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=412360cb6ec32517 server_cookie=764e0ce0a3285bfd in_seq=0 out_seq=0 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.180+0000 7fea89ffb640 10 client.16569 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7fea89ffb640 1 -- 172.21.15.66:0/1472213196 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593b85cdc10 con 0x5593b86dbea0 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7fea89ffb640 10 client.16569 handle_command_reply: tid=0 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 client.16569 shutdown 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 msgr2=0x5593b865e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865e650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0be820 tx=0x7fea9c0f2f70 comp rx=0 tx=0).stop 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 shutdown_connections 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593b8664a00 0x5593b86da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593b865be80 0x5593b865eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5593b86dbea0 0x5593b86de2c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 --2- 172.21.15.66:0/1472213196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5593b865b4d0 0x5593b865e650 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7fea89ffb640 0 client.16569 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 >> 172.21.15.66:0/1472213196 conn(0x5593b8655ba0 msgr2=0x5593b8657800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 shutdown_connections 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 1 -- 172.21.15.66:0/1472213196 wait complete. 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaabbaf740 20 client.16569 trim_cache size 0 max 16384 2024-04-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea8c0a3d20 msgr2=0x7fea8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea8c0a3d20 0x7fea8c0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea9400f010 tx=0x7fea94020000 comp rx=0 tx=0).stop 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 msgr2=0x7feaa4100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa4100e20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005940 tx=0x7fea9c0045e0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 shutdown_connections 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fea8c0a3d20 0x7fea8c0a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa410c510 0x7feaa41018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaa4103af0 0x7feaa4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 --2- 172.21.15.66:0/1211717204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaa41031d0 0x7feaa4100e20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 >> 172.21.15.66:0/1211717204 conn(0x7feaa4077f50 msgr2=0x7feaa40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 shutdown_connections 2024-04-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.181+0000 7feaa8ee0640 1 -- 172.21.15.66:0/1211717204 wait complete. 2024-04-21T09:10:58.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 -- 172.21.15.66:0/810142934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c102190 msgr2=0x7f316c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 --2- 172.21.15.66:0/810142934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c102190 0x7f316c102570 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3164009900 tx=0x7f316402bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 -- 172.21.15.66:0/810142934 shutdown_connections 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 --2- 172.21.15.66:0/810142934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c107260 0x7f316c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 --2- 172.21.15.66:0/810142934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c102b40 0x7f316c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 --2- 172.21.15.66:0/810142934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c102190 0x7f316c102570 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 -- 172.21.15.66:0/810142934 >> 172.21.15.66:0/810142934 conn(0x7f316c0fd990 msgr2=0x7f316c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 -- 172.21.15.66:0/810142934 shutdown_connections 2024-04-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.296+0000 7f3173c81640 1 -- 172.21.15.66:0/810142934 wait complete. 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 Processor -- start 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 -- start start 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c102190 0x7f316c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107260 0x7f316c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c198610 0x7f316c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f316c112210 con 0x7f316c107260 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f316c112090 con 0x7f316c102190 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3173c81640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f316c112390 con 0x7f316c198610 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107260 0x7f316c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107260 0x7f316c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34384/0 (socket says 172.21.15.66:34384) 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 -- 172.21.15.66:0/4061048200 learned_addr learned my addr 172.21.15.66:0/4061048200 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f3171a1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c102190 0x7f316c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317221e640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c198610 0x7f316c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 -- 172.21.15.66:0/4061048200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c198610 msgr2=0x7f316c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c198610 0x7f316c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 -- 172.21.15.66:0/4061048200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c102190 msgr2=0x7f316c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c102190 0x7f316c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.297+0000 7f317121c640 1 -- 172.21.15.66:0/4061048200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c199130 con 0x7f316c107260 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.298+0000 7f317121c640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107260 0x7f316c19e9a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f316000d740 tx=0x7f316000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.298+0000 7f316affd640 1 -- 172.21.15.66:0/4061048200 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160017070 con 0x7f316c107260 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.298+0000 7f316affd640 1 -- 172.21.15.66:0/4061048200 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316000de50 con 0x7f316c107260 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.298+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c199420 con 0x7f316c107260 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.298+0000 7f316affd640 1 -- 172.21.15.66:0/4061048200 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160004ce0 con 0x7f316c107260 2024-04-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.298+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a5620 con 0x7f316c107260 2024-04-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f316affd640 1 -- 172.21.15.66:0/4061048200 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3160004e80 con 0x7f316c107260 2024-04-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 Processor -- start 2024-04-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 -- start start 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb78790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56253fadd1f0 con 0x56253fb78d60 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56253fa7a4b0 con 0x56253fb81e60 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.301+0000 7f317687a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56253fad91b0 con 0x56253fb783b0 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb81890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34396/0 (socket says 172.21.15.66:34396) 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 -- 172.21.15.66:0/3175154860 learned_addr learned my addr 172.21.15.66:0/3175154860 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f316affd640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f31540a3d20 0x7f31540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f317121c640 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f317221e640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f31540a3d20 0x7f31540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f317221e640 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb78790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 -- 172.21.15.66:0/3175154860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 msgr2=0x56253fb78790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f316affd640 1 -- 172.21.15.66:0/4061048200 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f31600d0480 con 0x7f316c107260 2024-04-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb78790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 -- 172.21.15.66:0/3175154860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 msgr2=0x56253fb822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 -- 172.21.15.66:0/3175154860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253fa6c1b0 con 0x56253fb78d60 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f317221e640 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb78790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f317121c640 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3171a1d640 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb81890 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3164000c00 tx=0x7f316402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be56447f29ffdbc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f317221e640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f31540a3d20 0x7f31540a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f315c0099a0 tx=0x7f315c0091e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3153fff640 1 -- 172.21.15.66:0/3175154860 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164045020 con 0x56253fb78d60 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3153fff640 1 -- 172.21.15.66:0/3175154860 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316402c7e0 con 0x56253fb78d60 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.302+0000 7f3153fff640 1 -- 172.21.15.66:0/3175154860 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31640046c0 con 0x56253fb78d60 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 -- 172.21.15.66:0/3175154860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 msgr2=0x56253fb81890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb81890 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3164000c00 tx=0x7f316402c5f0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 -- 172.21.15.66:0/3175154860 shutdown_connections 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb81890 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 --2- 172.21.15.66:0/3175154860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb78790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 -- 172.21.15.66:0/3175154860 >> 172.21.15.66:0/3175154860 conn(0x56253fb72ba0 msgr2=0x56253fb72f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 -- 172.21.15.66:0/3175154860 shutdown_connections 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.303+0000 7f317687a740 1 -- 172.21.15.66:0/3175154860 wait complete. 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 -- start start 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 Processor -- start 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb88200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb85ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56253fb88d40 con 0x56253fb78d60 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56253fa96500 con 0x56253fb81e60 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56253fb88ec0 con 0x56253fb783b0 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317687a740 10 client.0 _ll_register_callbacks cb 0x56253fa64b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb85590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34402/0 (socket says 172.21.15.66:34402) 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 -- 172.21.15.66:0/4267059313 learned_addr learned my addr 172.21.15.66:0/4267059313 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317221e640 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb85ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f3171a1d640 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb88200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 -- 172.21.15.66:0/4267059313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 msgr2=0x56253fb88200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb88200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 -- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 msgr2=0x56253fb85ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb85ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.304+0000 7f317121c640 1 -- 172.21.15.66:0/4267059313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253f7ef3f0 con 0x56253fb78d60 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317121c640 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb85590 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3160005d00 tx=0x7f3160066ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883e8291dee489da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f31517fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f31517fa640 1 -- 172.21.15.66:0/4267059313 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160020050 con 0x56253fb78d60 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f31517fa640 1 -- 172.21.15.66:0/4267059313 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f31600987c0 con 0x56253fb78d60 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316402c7e0 con 0x56253fb78d60 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f31517fa640 1 -- 172.21.15.66:0/4267059313 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f316009c150 con 0x56253fb78d60 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 10 client.16581 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56253f8b97d0 con 0x56253fb78d60 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f31517fa640 1 -- 172.21.15.66:0/4267059313 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316402c7e0 con 0x56253fb78d60 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f31517fa640 1 -- 172.21.15.66:0/4267059313 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f31600a4930 con 0x56253fb78d60 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 10 client.16581 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 20 client.16581 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fb95f60 0x56253fb98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 4 client.16581 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.305+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56253fa7c930 con 0x56253fb95f60 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.306+0000 7f3171a1d640 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fb95f60 0x56253fb98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.306+0000 7f3171a1d640 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fb95f60 0x56253fb98350 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66fac134955b688d server_cookie=203b36d89e2f4871 in_seq=0 out_seq=0 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.306+0000 7f31517fa640 10 client.16581 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f31517fa640 1 -- 172.21.15.66:0/4267059313 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56253fa7c930 con 0x56253fb95f60 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f31517fa640 10 client.16581 handle_command_reply: tid=0 2024-04-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 client.16581 shutdown 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 msgr2=0x56253fb85590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb85590 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3160005d00 tx=0x7f3160066ca0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 shutdown_connections 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fb95f60 0x56253fb98350 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb85ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb85590 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 --2- 172.21.15.66:0/4267059313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb88200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 >> 172.21.15.66:0/4267059313 conn(0x56253fb72ba0 msgr2=0x56253fb748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f31517fa640 0 client.16581 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 shutdown_connections 2024-04-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 1 -- 172.21.15.66:0/4267059313 wait complete. 2024-04-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.307+0000 7f317687a740 20 client.16581 trim_cache size 0 max 16384 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 Processor -- start 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 -- start start 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb91c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb80080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb81440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1afa60 con 0x56253fb78d60 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1ad330 con 0x56253fb81e60 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317687a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1ab160 con 0x56253fb783b0 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb80080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb80080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34416/0 (socket says 172.21.15.66:34416) 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 -- 172.21.15.66:0/1579355858 learned_addr learned my addr 172.21.15.66:0/1579355858 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f3171a1d640 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb91c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317221e640 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb81440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 -- 172.21.15.66:0/1579355858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 msgr2=0x56253fb91c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb91c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 -- 172.21.15.66:0/1579355858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 msgr2=0x56253fb81440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb81440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 -- 172.21.15.66:0/1579355858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253fb77fe0 con 0x56253fb78d60 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f3171a1d640 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb91c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317221e640 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb81440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f317121c640 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb80080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f31600985b0 tx=0x7f3160097ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6397041714654e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f31537fe640 1 -- 172.21.15.66:0/1579355858 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31600b5070 con 0x56253fb78d60 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.321+0000 7f31537fe640 1 -- 172.21.15.66:0/1579355858 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31600a4dc0 con 0x56253fb78d60 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f31537fe640 1 -- 172.21.15.66:0/1579355858 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160020050 con 0x56253fb78d60 2024-04-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 -- 172.21.15.66:0/1579355858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 msgr2=0x56253fb80080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb80080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f31600985b0 tx=0x7f3160097ea0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 -- 172.21.15.66:0/1579355858 shutdown_connections 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb81440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb80080 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 --2- 172.21.15.66:0/1579355858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb91c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 -- 172.21.15.66:0/1579355858 >> 172.21.15.66:0/1579355858 conn(0x56253fb72ba0 msgr2=0x56253fb80d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 -- 172.21.15.66:0/1579355858 shutdown_connections 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 -- 172.21.15.66:0/1579355858 wait complete. 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.322+0000 7f317687a740 1 -- start start 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 1 Processor -- start 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb87f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1ab7f0 con 0x56253fb78d60 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1acc60 con 0x56253fb81e60 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1abec0 con 0x56253fb783b0 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317687a740 10 client.0 _ll_register_callbacks cb 0x7f316009bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb879c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34418/0 (socket says 172.21.15.66:34418) 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 -- 172.21.15.66:0/1348653146 learned_addr learned my addr 172.21.15.66:0/1348653146 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317221e640 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb87f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f3171a1d640 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 -- 172.21.15.66:0/1348653146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 msgr2=0x56253fb87480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb87480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 -- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 msgr2=0x56253fb87f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb87f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.323+0000 7f317121c640 1 -- 172.21.15.66:0/1348653146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253fbfb5f0 con 0x56253fb78d60 2024-04-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317121c640 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb879c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3160098b20 tx=0x7f31600d0380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770701347da5f903 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f31517fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f31517fa640 1 -- 172.21.15.66:0/1348653146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160020070 con 0x56253fb78d60 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f31517fa640 1 -- 172.21.15.66:0/1348653146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f31600a4b30 con 0x56253fb78d60 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31600a4dc0 con 0x56253fb78d60 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f31517fa640 1 -- 172.21.15.66:0/1348653146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31600b5040 con 0x56253fb78d60 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 10 client.16587 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56253fbfc860 con 0x56253fb78d60 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f31517fa640 1 -- 172.21.15.66:0/1348653146 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31600a4dc0 con 0x56253fb78d60 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f31517fa640 1 -- 172.21.15.66:0/1348653146 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f316009b210 con 0x56253fb78d60 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 10 client.16587 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 20 client.16587 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fbfcd10 0x56253fbff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 4 client.16587 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56253faeac10 con 0x56253fbfcd10 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.324+0000 7f3171a1d640 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fbfcd10 0x56253fbff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f3171a1d640 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fbfcd10 0x56253fbff130 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107f92180ca2d583 server_cookie=c914cef0c5ec41c4 in_seq=0 out_seq=0 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f31517fa640 10 client.16587 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f31517fa640 1 -- 172.21.15.66:0/1348653146 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56253faeac10 con 0x56253fbfcd10 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f31517fa640 10 client.16587 handle_command_reply: tid=0 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 client.16587 shutdown 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 msgr2=0x56253fb879c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb879c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3160098b20 tx=0x7f31600d0380 comp rx=0 tx=0).stop 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 shutdown_connections 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x56253fbfcd10 0x56253fbff130 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56253fb81e60 0x56253fb87f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56253fb78d60 0x56253fb879c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 --2- 172.21.15.66:0/1348653146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56253fb783b0 0x56253fb87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f31517fa640 0 client.16587 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 >> 172.21.15.66:0/1348653146 conn(0x56253fb72ba0 msgr2=0x56253fb815c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 shutdown_connections 2024-04-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 1 -- 172.21.15.66:0/1348653146 wait complete. 2024-04-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.325+0000 7f317687a740 20 client.16587 trim_cache size 0 max 16384 2024-04-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f31540a3d20 msgr2=0x7f31540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f31540a3d20 0x7f31540a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f315c0099a0 tx=0x7f315c0091e0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107260 msgr2=0x7f316c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107260 0x7f316c19e9a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f316000d740 tx=0x7f316000dc00 comp rx=0 tx=0).stop 2024-04-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 shutdown_connections 2024-04-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f31540a3d20 0x7f31540a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f316c198610 0x7f316c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f316c107260 0x7f316c19e9a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 --2- 172.21.15.66:0/4061048200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f316c102190 0x7f316c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 >> 172.21.15.66:0/4061048200 conn(0x7f316c0fd990 msgr2=0x7f316c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 shutdown_connections 2024-04-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:58.326+0000 7f3173c81640 1 -- 172.21.15.66:0/4061048200 wait complete. 2024-04-21T09:10:58.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:58 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1301104067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:58.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:58 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1301104067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:58.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:58 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1301104067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:10:59.338 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:10:59.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:10:59.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:10:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 -- 172.21.15.66:0/1695802589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0101a50 msgr2=0x7f04c0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 --2- 172.21.15.66:0/1695802589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0101a50 0x7f04c0105a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009900 tx=0x7f04b002bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 -- 172.21.15.66:0/1695802589 shutdown_connections 2024-04-21T09:10:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 --2- 172.21.15.66:0/1695802589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c0106170 0x7f04c010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 --2- 172.21.15.66:0/1695802589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0101a50 0x7f04c0105a90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 --2- 172.21.15.66:0/1695802589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c01010a0 0x7f04c0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 -- 172.21.15.66:0/1695802589 >> 172.21.15.66:0/1695802589 conn(0x7f04c0077eb0 msgr2=0x7f04c00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 -- 172.21.15.66:0/1695802589 shutdown_connections 2024-04-21T09:10:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 -- 172.21.15.66:0/1695802589 wait complete. 2024-04-21T09:10:59.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 Processor -- start 2024-04-21T09:10:59.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.459+0000 7f04c777e640 1 -- start start 2024-04-21T09:10:59.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c777e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c01010a0 0x7f04c019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c777e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0106170 0x7f04c019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c777e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c0198590 0x7f04c0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c777e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01121f0 con 0x7f04c0106170 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c777e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0112070 con 0x7f04c01010a0 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c777e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0112370 con 0x7f04c0198590 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0106170 0x7f04c019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0106170 0x7f04c019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34436/0 (socket says 172.21.15.66:34436) 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 -- 172.21.15.66:0/4165329113 learned_addr learned my addr 172.21.15.66:0/4165329113 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c0198590 0x7f04c0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c551a640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c01010a0 0x7f04c019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 -- 172.21.15.66:0/4165329113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c0198590 msgr2=0x7f04c0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c0198590 0x7f04c0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 -- 172.21.15.66:0/4165329113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c01010a0 msgr2=0x7f04c019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c01010a0 0x7f04c019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 -- 172.21.15.66:0/4165329113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c01990b0 con 0x7f04c0106170 2024-04-21T09:10:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c551a640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c01010a0 0x7f04c019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.460+0000 7f04c4d19640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0106170 0x7f04c019e920 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f04b0036040 tx=0x7f04b002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.461+0000 7f04b67fc640 1 -- 172.21.15.66:0/4165329113 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0043020 con 0x7f04c0106170 2024-04-21T09:10:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.461+0000 7f04b67fc640 1 -- 172.21.15.66:0/4165329113 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b002cbe0 con 0x7f04c0106170 2024-04-21T09:10:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.461+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c0199340 con 0x7f04c0106170 2024-04-21T09:10:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.461+0000 7f04b67fc640 1 -- 172.21.15.66:0/4165329113 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0003bf0 con 0x7f04c0106170 2024-04-21T09:10:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.461+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c01a5590 con 0x7f04c0106170 2024-04-21T09:10:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.463+0000 7f04b67fc640 1 -- 172.21.15.66:0/4165329113 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f04b0046020 con 0x7f04c0106170 2024-04-21T09:10:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 Processor -- start 2024-04-21T09:10:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 -- start start 2024-04-21T09:10:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f4f8e0 0x559590f4fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f58840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f58e10 0x559590f5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559590eb3f00 con 0x559590f50290 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559590e51590 con 0x559590f4f8e0 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04ca36f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559590eafec0 con 0x559590f58e10 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04b67fc640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04c551a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f58840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04c551a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f58840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34446/0 (socket says 172.21.15.66:34446) 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04c551a640 1 -- 172.21.15.66:0/741018366 learned_addr learned my addr 172.21.15.66:0/741018366 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f4f8e0 0x559590f4fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04c4d19640 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f58e10 0x559590f5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.464+0000 7f04b67fc640 1 -- 172.21.15.66:0/4165329113 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f04b00f2d00 con 0x7f04c0106170 2024-04-21T09:10:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04c551a640 1 -- 172.21.15.66:0/741018366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f58e10 msgr2=0x559590f5b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04c551a640 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f58e10 0x559590f5b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04c551a640 1 -- 172.21.15.66:0/741018366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f4f8e0 msgr2=0x559590f4fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04c551a640 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f4f8e0 0x559590f4fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04c551a640 1 -- 172.21.15.66:0/741018366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559590c8e7e0 con 0x559590f50290 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009990 tx=0x7f04b8009aa0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04c551a640 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f58840 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00bd40 tx=0x7f04bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec93e0120ea886b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04ab7fe640 1 -- 172.21.15.66:0/741018366 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc015070 con 0x559590f50290 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04ab7fe640 1 -- 172.21.15.66:0/741018366 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04bc00be30 con 0x559590f50290 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.465+0000 7f04ab7fe640 1 -- 172.21.15.66:0/741018366 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0106c0 con 0x559590f50290 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 -- 172.21.15.66:0/741018366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 msgr2=0x559590f58840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f58840 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00bd40 tx=0x7f04bc00f730 comp rx=0 tx=0).stop 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 -- 172.21.15.66:0/741018366 shutdown_connections 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f58e10 0x559590f5b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f58840 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 --2- 172.21.15.66:0/741018366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f4f8e0 0x559590f4fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 -- 172.21.15.66:0/741018366 >> 172.21.15.66:0/741018366 conn(0x559590f49fb0 msgr2=0x559590f4a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 -- 172.21.15.66:0/741018366 shutdown_connections 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 -- 172.21.15.66:0/741018366 wait complete. 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 -- start start 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.466+0000 7f04ca36f740 1 Processor -- start 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f55db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559590f5bd80 con 0x559590f4f8e0 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559590e6d580 con 0x559590f58e10 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559590f5bf00 con 0x559590f50290 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 10 client.0 _ll_register_callbacks cb 0x559590e73650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f55db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f55db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34462/0 (socket says 172.21.15.66:34462) 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 -- 172.21.15.66:0/1417264540 learned_addr learned my addr 172.21.15.66:0/1417264540 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c4d19640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 -- 172.21.15.66:0/1417264540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 msgr2=0x559590f562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 -- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 msgr2=0x559590f56830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 -- 172.21.15.66:0/1417264540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559590ceef70 con 0x559590f4f8e0 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04c551a640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f55db0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00ba90 tx=0x7f04bc00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfe15f49caf1630 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04a8ff9640 1 -- 172.21.15.66:0/1417264540 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc029060 con 0x559590f4f8e0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04a8ff9640 1 -- 172.21.15.66:0/1417264540 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f04bc011650 con 0x559590f4f8e0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.467+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04bc00be30 con 0x559590f4f8e0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04a8ff9640 1 -- 172.21.15.66:0/1417264540 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04bc030050 con 0x559590f4f8e0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 10 client.16599 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559590b86170 con 0x559590f4f8e0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04a8ff9640 1 -- 172.21.15.66:0/1417264540 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04bc00be30 con 0x559590f4f8e0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04a8ff9640 1 -- 172.21.15.66:0/1417264540 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04bc01a460 con 0x559590f4f8e0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 10 client.16599 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 20 client.16599 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590f69250 0x559590f6b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 4 client.16599 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559590ec1920 con 0x559590f69250 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.468+0000 7f04c4d19640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590f69250 0x559590f6b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.469+0000 7f04c4d19640 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590f69250 0x559590f6b640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9070a1e3636895f7 server_cookie=3aa13f5bf6092408 in_seq=0 out_seq=0 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.469+0000 7f04a8ff9640 10 client.16599 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.469+0000 7f04a8ff9640 1 -- 172.21.15.66:0/1417264540 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559590ec1920 con 0x559590f69250 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.469+0000 7f04a8ff9640 10 client.16599 handle_command_reply: tid=0 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 client.16599 shutdown 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 msgr2=0x559590f55db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f55db0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00ba90 tx=0x7f04bc00bac0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 shutdown_connections 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590f69250 0x559590f6b640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04a8ff9640 0 client.16599 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1417264540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f55db0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 >> 172.21.15.66:0/1417264540 conn(0x559590f49fb0 msgr2=0x559590f4bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 shutdown_connections 2024-04-21T09:10:59.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 1 -- 172.21.15.66:0/1417264540 wait complete. 2024-04-21T09:10:59.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.470+0000 7f04ca36f740 20 client.16599 trim_cache size 0 max 16384 2024-04-21T09:10:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 Processor -- start 2024-04-21T09:10:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 -- start start 2024-04-21T09:10:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f64c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f556b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b1b20 con 0x559590f4f8e0 2024-04-21T09:10:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01af3f0 con 0x559590f58e10 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04ca36f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01ad220 con 0x559590f50290 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04c551a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f64c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04c551a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f64c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34464/0 (socket says 172.21.15.66:34464) 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.488+0000 7f04c551a640 1 -- 172.21.15.66:0/1870640619 learned_addr learned my addr 172.21.15.66:0/1870640619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c551a640 1 -- 172.21.15.66:0/1870640619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 msgr2=0x559590f556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c4d19640 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c551a640 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c551a640 1 -- 172.21.15.66:0/1870640619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 msgr2=0x559590f56480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c551a640 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c551a640 1 -- 172.21.15.66:0/1870640619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559590f50bf0 con 0x559590f4f8e0 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c4d19640 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04c551a640 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f64c40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00f770 tx=0x7f04bc00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61164b5879017b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04aaffd640 1 -- 172.21.15.66:0/1870640619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc029040 con 0x559590f4f8e0 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04aaffd640 1 -- 172.21.15.66:0/1870640619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04bc00b3a0 con 0x559590f4f8e0 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04aaffd640 1 -- 172.21.15.66:0/1870640619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc015050 con 0x559590f4f8e0 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04ca36f740 1 -- 172.21.15.66:0/1870640619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 msgr2=0x559590f64c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f64c40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00f770 tx=0x7f04bc00bfc0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04ca36f740 1 -- 172.21.15.66:0/1870640619 shutdown_connections 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f56480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f50290 0x559590f556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04ca36f740 1 --2- 172.21.15.66:0/1870640619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f4f8e0 0x559590f64c40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.489+0000 7f04ca36f740 1 -- 172.21.15.66:0/1870640619 >> 172.21.15.66:0/1870640619 conn(0x559590f49fb0 msgr2=0x559590f55d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 -- 172.21.15.66:0/1870640619 shutdown_connections 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 -- 172.21.15.66:0/1870640619 wait complete. 2024-04-21T09:10:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 -- start start 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 Processor -- start 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f4f8e0 0x559590f61500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f52790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f52cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01ad8b0 con 0x559590f50290 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01aed20 con 0x559590f58e10 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01adf80 con 0x559590f4f8e0 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04ca36f740 10 client.0 _ll_register_callbacks cb 0x7f04b8009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.490+0000 7f04c4d19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f52790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f52790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34474/0 (socket says 172.21.15.66:34474) 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 -- 172.21.15.66:0/2259446764 learned_addr learned my addr 172.21.15.66:0/2259446764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c551a640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f4f8e0 0x559590f61500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f52cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 -- 172.21.15.66:0/2259446764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f4f8e0 msgr2=0x559590f61500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f4f8e0 0x559590f61500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 -- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 msgr2=0x559590f52cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f52cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 -- 172.21.15.66:0/2259446764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559590f53500 con 0x559590f50290 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c5d1b640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f52cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04c4d19640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f52790 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f04b00be740 tx=0x7f04b00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e7920aa2ac4bff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04a8ff9640 1 -- 172.21.15.66:0/2259446764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b00d7070 con 0x559590f50290 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04a8ff9640 1 -- 172.21.15.66:0/2259446764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f04b00bcd40 con 0x559590f50290 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04bc00b3a0 con 0x559590f50290 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04a8ff9640 1 -- 172.21.15.66:0/2259446764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04b00bea40 con 0x559590f50290 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04ca36f740 10 client.16605 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559590fcf4f0 con 0x559590f50290 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.491+0000 7f04a8ff9640 1 -- 172.21.15.66:0/2259446764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04bc00b3a0 con 0x559590f50290 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04a8ff9640 1 -- 172.21.15.66:0/2259446764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04b00bb0b0 con 0x559590f50290 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04ca36f740 10 client.16605 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04ca36f740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04ca36f740 20 client.16605 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04ca36f740 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590fcfc20 0x559590fd2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04ca36f740 4 client.16605 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559590ebb790 con 0x559590fcfc20 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04c551a640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590fcfc20 0x559590fd2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04a8ff9640 10 client.16605 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.492+0000 7f04c551a640 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590fcfc20 0x559590fd2040 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d943730651832b3 server_cookie=2109430ec9fe72e0 in_seq=0 out_seq=0 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.493+0000 7f04a8ff9640 1 -- 172.21.15.66:0/2259446764 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559590ebb790 con 0x559590fcfc20 2024-04-21T09:10:59.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.493+0000 7f04a8ff9640 10 client.16605 handle_command_reply: tid=0 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.493+0000 7f04ca36f740 1 client.16605 shutdown 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 msgr2=0x559590f52790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f52790 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f04b00be740 tx=0x7f04b00036f0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 shutdown_connections 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559590fcfc20 0x559590fd2040 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559590f58e10 0x559590f52cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04a8ff9640 0 client.16605 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559590f50290 0x559590f52790 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 --2- 172.21.15.66:0/2259446764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559590f4f8e0 0x559590f61500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 >> 172.21.15.66:0/2259446764 conn(0x559590f49fb0 msgr2=0x559590f565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 shutdown_connections 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 1 -- 172.21.15.66:0/2259446764 wait complete. 2024-04-21T09:10:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04ca36f740 20 client.16605 trim_cache size 0 max 16384 2024-04-21T09:10:59.504 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:10:59.504 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:10:59.504 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "6e0763aa-ce0a-448d-bbb9-88bf780801db", 2024-04-21T09:10:59.504 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:10:59.504 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:10:59.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04ac0a3d20 msgr2=0x7f04ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009990 tx=0x7f04b8009aa0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0106170 msgr2=0x7f04c019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0106170 0x7f04c019e920 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f04b0036040 tx=0x7f04b002c7e0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 shutdown_connections 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c0198590 0x7f04c0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04c0106170 0x7f04c019e920 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 --2- 172.21.15.66:0/4165329113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c01010a0 0x7f04c019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.494+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 >> 172.21.15.66:0/4165329113 conn(0x7f04c0077eb0 msgr2=0x7f04c00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.495+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 shutdown_connections 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.495+0000 7f04c777e640 1 -- 172.21.15.66:0/4165329113 wait complete. 2024-04-21T09:10:59.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:10:59.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:10:59.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:10:59 smithi083 ceph-mon[31413]: pgmap v86: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:59.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 -- 172.21.15.66:0/2664946893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a806a5b0 msgr2=0x7fb9a806aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 --2- 172.21.15.66:0/2664946893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a806a5b0 0x7fb9a806aa30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb994009930 tx=0x7fb99402bc30 comp rx=0 tx=0).stop 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 -- 172.21.15.66:0/2664946893 shutdown_connections 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 --2- 172.21.15.66:0/2664946893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9a806af70 0x7fb9a810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 --2- 172.21.15.66:0/2664946893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a806a5b0 0x7fb9a806aa30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 --2- 172.21.15.66:0/2664946893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a8108060 0x7fb9a8108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 -- 172.21.15.66:0/2664946893 >> 172.21.15.66:0/2664946893 conn(0x7fb9a8077c40 msgr2=0x7fb9a8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 -- 172.21.15.66:0/2664946893 shutdown_connections 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.612+0000 7fb9aef84640 1 -- 172.21.15.66:0/2664946893 wait complete. 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 Processor -- start 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 -- start start 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a806a5b0 0x7fb9a81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9a806af70 0x7fb9a81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a8108060 0x7fb9a819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8112200 con 0x7fb9a8108060 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8112080 con 0x7fb9a806a5b0 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9aef84640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8112380 con 0x7fb9a806af70 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a8108060 0x7fb9a819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a8108060 0x7fb9a819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34482/0 (socket says 172.21.15.66:34482) 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 -- 172.21.15.66:0/1397268558 learned_addr learned my addr 172.21.15.66:0/1397268558 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9acd20640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a806a5b0 0x7fb9a81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9a806af70 0x7fb9a81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 -- 172.21.15.66:0/1397268558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9a806af70 msgr2=0x7fb9a81a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9a806af70 0x7fb9a81a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 -- 172.21.15.66:0/1397268558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a806a5b0 msgr2=0x7fb9a81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a806a5b0 0x7fb9a81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.613+0000 7fb9ad521640 1 -- 172.21.15.66:0/1397268558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a819afb0 con 0x7fb9a8108060 2024-04-21T09:10:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.614+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a8108060 0x7fb9a819a6f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b970 tx=0x7fb9a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.614+0000 7fb9a5ffb640 1 -- 172.21.15.66:0/1397268558 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0019070 con 0x7fb9a8108060 2024-04-21T09:10:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.614+0000 7fb9a5ffb640 1 -- 172.21.15.66:0/1397268558 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a0014440 con 0x7fb9a8108060 2024-04-21T09:10:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.614+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a819b2a0 con 0x7fb9a8108060 2024-04-21T09:10:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.614+0000 7fb9a5ffb640 1 -- 172.21.15.66:0/1397268558 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00075d0 con 0x7fb9a8108060 2024-04-21T09:10:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.614+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a81a75b0 con 0x7fb9a8108060 2024-04-21T09:10:59.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.616+0000 7fb9a5ffb640 1 -- 172.21.15.66:0/1397268558 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb9a001c070 con 0x7fb9a8108060 2024-04-21T09:10:59.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.616+0000 7fb9b1b79740 1 Processor -- start 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.616+0000 7fb9b1b79740 1 -- start start 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407637960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9a5ffb640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb9980a3d20 0x7fb9980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e407640540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e407642e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e40759c1e0 con 0x55e407640a80 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e407539390 con 0x55e407637ea0 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e4075981a0 con 0x55e407637580 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e407642e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb9980a3d20 0x7fb9980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e407642e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34492/0 (socket says 172.21.15.66:34492) 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 -- 172.21.15.66:0/3278960189 learned_addr learned my addr 172.21.15.66:0/3278960189 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9a5ffb640 1 -- 172.21.15.66:0/1397268558 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb9a00d0010 con 0x7fb9a8108060 2024-04-21T09:10:59.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e407640540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407637960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 -- 172.21.15.66:0/3278960189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 msgr2=0x55e407637960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407637960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 -- 172.21.15.66:0/3278960189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 msgr2=0x55e407640540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e407640540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 -- 172.21.15.66:0/3278960189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4073787d0 con 0x55e407640a80 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e407640540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb9980a3d20 0x7fb9980a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb99c005d00 tx=0x7fb99c00b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb9ad521640 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e407642e70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0004a60 tx=0x7fb9a0050ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1a7144d39cd270 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb992ffd640 1 -- 172.21.15.66:0/3278960189 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a009be00 con 0x55e407640a80 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.617+0000 7fb992ffd640 1 -- 172.21.15.66:0/3278960189 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a0052070 con 0x55e407640a80 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb992ffd640 1 -- 172.21.15.66:0/3278960189 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a000cc80 con 0x55e407640a80 2024-04-21T09:10:59.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 -- 172.21.15.66:0/3278960189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 msgr2=0x55e407642e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e407642e70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0004a60 tx=0x7fb9a0050ff0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 -- 172.21.15.66:0/3278960189 shutdown_connections 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e407642e70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e407640540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/3278960189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407637960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 -- 172.21.15.66:0/3278960189 >> 172.21.15.66:0/3278960189 conn(0x55e407631cb0 msgr2=0x55e407632090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 -- 172.21.15.66:0/3278960189 shutdown_connections 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 -- 172.21.15.66:0/3278960189 wait complete. 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.618+0000 7fb9b1b79740 1 -- start start 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 1 Processor -- start 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e40763dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e4076437b0 con 0x55e407640a80 2024-04-21T09:10:59.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e4075552f0 con 0x55e407637ea0 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e407643930 con 0x55e407637580 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9b1b79740 10 client.0 _ll_register_callbacks cb 0x55e407562580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34506/0 (socket says 172.21.15.66:34506) 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 -- 172.21.15.66:0/1607699883 learned_addr learned my addr 172.21.15.66:0/1607699883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e40763dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 -- 172.21.15.66:0/1607699883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 msgr2=0x55e40763dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e40763dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 -- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 msgr2=0x55e40763e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.619+0000 7fb9ad521640 1 -- 172.21.15.66:0/1607699883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4072ae3f0 con 0x55e407640a80 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763e560 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00987d0 tx=0x7fb9a009ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d88b5913e6fb992 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb987fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:59.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb987fff640 1 -- 172.21.15.66:0/1607699883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00b4020 con 0x55e407640a80 2024-04-21T09:10:59.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb987fff640 1 -- 172.21.15.66:0/1607699883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb9a00cd670 con 0x55e407640a80 2024-04-21T09:10:59.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a0052070 con 0x55e407640a80 2024-04-21T09:10:59.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb987fff640 1 -- 172.21.15.66:0/1607699883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9a00cd870 con 0x55e407640a80 2024-04-21T09:10:59.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 10 client.16617 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:59.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e40726e0f0 con 0x55e407640a80 2024-04-21T09:10:59.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:10:59 smithi066 ceph-mon[27953]: pgmap v86: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb987fff640 1 -- 172.21.15.66:0/1607699883 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a0052070 con 0x55e407640a80 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb987fff640 1 -- 172.21.15.66:0/1607699883 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9a00a4530 con 0x55e407640a80 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 10 client.16617 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 20 client.16617 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e407650870 0x55e407652c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 4 client.16617 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:10:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.620+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4075a9c00 con 0x55e407650870 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.621+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e407650870 0x55e407652c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.621+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e407650870 0x55e407652c60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87913ccb821a0bed server_cookie=6b3562e4cf0b39c3 in_seq=0 out_seq=0 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.621+0000 7fb987fff640 10 client.16617 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb987fff640 1 -- 172.21.15.66:0/1607699883 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4075a9c00 con 0x55e407650870 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb987fff640 10 client.16617 handle_command_reply: tid=0 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 client.16617 shutdown 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 msgr2=0x55e40763e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763e560 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00987d0 tx=0x7fb9a009ba40 comp rx=0 tx=0).stop 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 shutdown_connections 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e407650870 0x55e407652c60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763e560 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1607699883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e40763dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb987fff640 0 client.16617 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 >> 172.21.15.66:0/1607699883 conn(0x55e407631cb0 msgr2=0x55e407633a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 shutdown_connections 2024-04-21T09:10:59.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.622+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1607699883 wait complete. 2024-04-21T09:10:59.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.623+0000 7fb9b1b79740 20 client.16617 trim_cache size 0 max 16384 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 Processor -- start 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 -- start start 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407637580 0x55e40764c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407640a80 0x55e407633810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a81b3bf0 con 0x55e407637580 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a81b14c0 con 0x55e407637ea0 2024-04-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a81af2f0 con 0x55e407640a80 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9acd20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407637580 0x55e40764c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9acd20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407637580 0x55e40764c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34520/0 (socket says 172.21.15.66:34520) 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9acd20640 1 -- 172.21.15.66:0/1236782181 learned_addr learned my addr 172.21.15.66:0/1236782181 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.640+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407640a80 0x55e407633810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9acd20640 1 -- 172.21.15.66:0/1236782181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407640a80 msgr2=0x55e407633810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407640a80 0x55e407633810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9acd20640 1 -- 172.21.15.66:0/1236782181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 msgr2=0x55e40763e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9acd20640 1 -- 172.21.15.66:0/1236782181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e407652fd0 con 0x55e407637580 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407640a80 0x55e407633810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407637580 0x55e40764c670 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb99c011600 tx=0x7fb99c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c8cf88ee4066c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9927fc640 1 -- 172.21.15.66:0/1236782181 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0275f0 con 0x55e407637580 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9927fc640 1 -- 172.21.15.66:0/1236782181 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb99c027790 con 0x55e407637580 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.641+0000 7fb9927fc640 1 -- 172.21.15.66:0/1236782181 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01ea60 con 0x55e407637580 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1236782181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407637580 msgr2=0x55e40764c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407637580 0x55e40764c670 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb99c011600 tx=0x7fb99c013000 comp rx=0 tx=0).stop 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1236782181 shutdown_connections 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407640a80 0x55e407633810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1236782181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407637580 0x55e40764c670 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1236782181 >> 172.21.15.66:0/1236782181 conn(0x55e407631cb0 msgr2=0x55e407633100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1236782181 shutdown_connections 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1236782181 wait complete. 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 -- start start 2024-04-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 Processor -- start 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.642+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407648fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a81af980 con 0x55e407640a80 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a81b0df0 con 0x55e407637ea0 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a81b0050 con 0x55e407637580 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 10 client.0 _ll_register_callbacks cb 0x7fb9a00146a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34534/0 (socket says 172.21.15.66:34534) 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 -- 172.21.15.66:0/1125915103 learned_addr learned my addr 172.21.15.66:0/1125915103 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407648fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 -- 172.21.15.66:0/1125915103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 msgr2=0x55e407648fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407648fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 -- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 msgr2=0x55e40763a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 -- 172.21.15.66:0/1125915103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40763b1a0 con 0x55e407640a80 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9a7fff640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9ad521640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763a9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb9a009bb00 tx=0x7fb9a009a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a62844f6a304061 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb987fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb987fff640 1 -- 172.21.15.66:0/1125915103 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00a0100 con 0x55e407640a80 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb987fff640 1 -- 172.21.15.66:0/1125915103 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb9a00a4530 con 0x55e407640a80 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb99c027790 con 0x55e407640a80 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb987fff640 1 -- 172.21.15.66:0/1125915103 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9a00a49c0 con 0x55e407640a80 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 10 client.16623 fetch_fsmap learned FSMap version 25 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4076b6a00 con 0x55e407640a80 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.643+0000 7fb987fff640 1 -- 172.21.15.66:0/1125915103 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb99c027790 con 0x55e407640a80 2024-04-21T09:10:59.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb987fff640 1 -- 172.21.15.66:0/1125915103 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9a00cd700 con 0x55e407640a80 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9b1b79740 10 client.16623 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9b1b79740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9b1b79740 20 client.16623 populate_metadata read hostname 'smithi066' 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e4076b6f20 0x55e4076b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9b1b79740 4 client.16623 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4075a3a70 con 0x55e4076b6f20 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e4076b6f20 0x55e4076b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb987fff640 10 client.16623 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.644+0000 7fb9acd20640 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e4076b6f20 0x55e4076b9310 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c10fca60e647ac server_cookie=e2d6eb0e8c6c1882 in_seq=0 out_seq=0 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb987fff640 1 -- 172.21.15.66:0/1125915103 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e4075a3a70 con 0x55e4076b6f20 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb987fff640 10 client.16623 handle_command_reply: tid=0 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 client.16623 shutdown 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 msgr2=0x55e40763a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763a9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb9a009bb00 tx=0x7fb9a009a100 comp rx=0 tx=0).stop 2024-04-21T09:10:59.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 shutdown_connections 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e4076b6f20 0x55e4076b9310 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e407640a80 0x55e40763a9c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e407637ea0 0x55e40763a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb987fff640 0 client.16623 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 --2- 172.21.15.66:0/1125915103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e407637580 0x55e407648fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 >> 172.21.15.66:0/1125915103 conn(0x55e407631cb0 msgr2=0x55e407633900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 shutdown_connections 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 1 -- 172.21.15.66:0/1125915103 wait complete. 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9b1b79740 20 client.16623 trim_cache size 0 max 16384 2024-04-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: "6e0763aa-ce0a-448d-bbb9-88bf780801db": { 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "6e0763aa-ce0a-448d-bbb9-88bf780801db", 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:10:59.652 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb9980a3d20 msgr2=0x7fb9980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9aef84640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb9980a3d20 0x7fb9980a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb99c005d00 tx=0x7fb99c00b410 comp rx=0 tx=0).stop 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a8108060 msgr2=0x7fb9a819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.645+0000 7fb9aef84640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a8108060 0x7fb9a819a6f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b970 tx=0x7fb9a000be30 comp rx=0 tx=0).stop 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 shutdown_connections 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb9980a3d20 0x7fb9980a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9a8108060 0x7fb9a819a6f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9a806af70 0x7fb9a81a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 --2- 172.21.15.66:0/1397268558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a806a5b0 0x7fb9a81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:10:59.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 >> 172.21.15.66:0/1397268558 conn(0x7fb9a8077c40 msgr2=0x7fb9a810d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:10:59.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 shutdown_connections 2024-04-21T09:10:59.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:10:59.646+0000 7fb9aef84640 1 -- 172.21.15.66:0/1397268558 wait complete. 2024-04-21T09:10:59.656 INFO:tasks.cephfs.filesystem:scrub status for tag:6e0763aa-ce0a-448d-bbb9-88bf780801db - {'path': '/', 'tag': '6e0763aa-ce0a-448d-bbb9-88bf780801db', 'options': 'recursive,force'} 2024-04-21T09:10:59.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:10:59 smithi186 ceph-mon[31505]: pgmap v86: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:00.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:00 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:11:00.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:00 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:00.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:00 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:11:00.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:00 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:00.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:00 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:11:00.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:00 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:00.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:00 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:11:00.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:00 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:00.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:00 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:11:00.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:00 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:00.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:00 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:11:00.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:00 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:01.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:01 smithi083 ceph-mon[31413]: pgmap v87: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:01.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:01 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:11:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:01 smithi066 ceph-mon[27953]: pgmap v87: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:01.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:01 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:11:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:01 smithi186 ceph-mon[31505]: pgmap v87: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:01 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:11:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:03 smithi083 ceph-mon[31413]: pgmap v88: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:03.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:03 smithi066 ceph-mon[27953]: pgmap v88: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:03.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:03 smithi186 ceph-mon[31505]: pgmap v88: 129 pgs: 129 active+clean; 259 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:05.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:05 smithi083 ceph-mon[31413]: pgmap v89: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:05.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:05 smithi066 ceph-mon[27953]: pgmap v89: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:05.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:05 smithi186 ceph-mon[31505]: pgmap v89: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:07.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:07 smithi083 ceph-mon[31413]: pgmap v90: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:07 smithi066 ceph-mon[27953]: pgmap v90: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:07 smithi186 ceph-mon[31505]: pgmap v90: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:09.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:09 smithi066 ceph-mon[27953]: pgmap v91: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:09.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:09 smithi186 ceph-mon[31505]: pgmap v91: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:09 smithi083 ceph-mon[31413]: pgmap v91: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:11.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:11 smithi066 ceph-mon[27953]: pgmap v92: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:11 smithi186 ceph-mon[31505]: pgmap v92: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:11.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:11 smithi083 ceph-mon[31413]: pgmap v92: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:13.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:13 smithi066 ceph-mon[27953]: pgmap v93: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:13 smithi186 ceph-mon[31505]: pgmap v93: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:13.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:13 smithi083 ceph-mon[31413]: pgmap v93: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:11:15.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:15 smithi066 ceph-mon[27953]: pgmap v94: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:15 smithi186 ceph-mon[31505]: pgmap v94: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:15 smithi083 ceph-mon[31413]: pgmap v94: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:11:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:17 smithi186 ceph-mon[31505]: pgmap v95: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:11:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:17 smithi083 ceph-mon[31413]: pgmap v95: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:11:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:17 smithi066 ceph-mon[27953]: pgmap v95: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:11:19.640 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:19 smithi186 ceph-mon[31505]: pgmap v96: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T09:11:19.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:19 smithi083 ceph-mon[31413]: pgmap v96: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T09:11:19.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:19 smithi066 ceph-mon[27953]: pgmap v96: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T09:11:20.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:20 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:11:20.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:20 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:11:20.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:20 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:11:20.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:20 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:11:20.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:20 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:11:20.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:20 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:11:20.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:20 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:11:20.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:20 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:11:20.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:20 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:11:20.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:20 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:11:20.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:20 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:11:20.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:20 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:11:21.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:21 smithi083 ceph-mon[31413]: pgmap v97: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:21 smithi186 ceph-mon[31505]: pgmap v97: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:21.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:21 smithi066 ceph-mon[27953]: pgmap v97: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:23 smithi186 ceph-mon[31505]: pgmap v98: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:23 smithi083 ceph-mon[31413]: pgmap v98: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:23.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:23 smithi066 ceph-mon[27953]: pgmap v98: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:25.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:25 smithi186 ceph-mon[31505]: pgmap v99: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:25.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:25 smithi083 ceph-mon[31413]: pgmap v99: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:25.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:25 smithi066 ceph-mon[27953]: pgmap v99: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:27.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:27 smithi186 ceph-mon[31505]: pgmap v100: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:27.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:27 smithi083 ceph-mon[31413]: pgmap v100: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:27.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:27 smithi066 ceph-mon[27953]: pgmap v100: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:29.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:11:29.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:11:29.671 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:29 smithi066 ceph-mon[27953]: pgmap v101: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:29.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/856170761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb0101a60 msgr2=0x7fdbb0105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/856170761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb0101a60 0x7fdbb0105aa0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009900 tx=0x7fdb9c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:11:29.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/856170761 shutdown_connections 2024-04-21T09:11:29.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/856170761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0106180 0x7fdbb010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/856170761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb0101a60 0x7fdbb0105aa0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/856170761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb01010b0 0x7fdbb0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/856170761 >> 172.21.15.66:0/856170761 conn(0x7fdbb00771d0 msgr2=0x7fdbb00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/856170761 shutdown_connections 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.775+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/856170761 wait complete. 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 Processor -- start 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 -- start start 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb01010b0 0x7fdbb01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0106180 0x7fdbb0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb019cf90 0x7fdbb019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01110a0 con 0x7fdbb019cf90 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0110f20 con 0x7fdbb01010b0 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb6bfd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0111220 con 0x7fdbb0106180 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb519a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb019cf90 0x7fdbb019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb519a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb019cf90 0x7fdbb019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54224/0 (socket says 172.21.15.66:54224) 2024-04-21T09:11:29.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.776+0000 7fdbb519a640 1 -- 172.21.15.66:0/4217156381 learned_addr learned my addr 172.21.15.66:0/4217156381 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbaffff640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0106180 0x7fdbb0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb4999640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb01010b0 0x7fdbb01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb519a640 1 -- 172.21.15.66:0/4217156381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0106180 msgr2=0x7fdbb0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb519a640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0106180 0x7fdbb0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb519a640 1 -- 172.21.15.66:0/4217156381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb01010b0 msgr2=0x7fdbb01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb519a640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb01010b0 0x7fdbb01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb519a640 1 -- 172.21.15.66:0/4217156381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb019da80 con 0x7fdbb019cf90 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb519a640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb019cf90 0x7fdbb019d440 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fdba800cce0 tx=0x7fdba800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbadffb640 1 -- 172.21.15.66:0/4217156381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba800f450 con 0x7fdbb019cf90 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb019dd70 con 0x7fdbb019cf90 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbadffb640 1 -- 172.21.15.66:0/4217156381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba800faa0 con 0x7fdbb019cf90 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbadffb640 1 -- 172.21.15.66:0/4217156381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8018b70 con 0x7fdbb019cf90 2024-04-21T09:11:29.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.777+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a54e0 con 0x7fdbb019cf90 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbadffb640 1 -- 172.21.15.66:0/4217156381 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdba8018d10 con 0x7fdbb019cf90 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 Processor -- start 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 -- start start 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3523151e0 con 0x55f3523b0ea0 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3522b2390 con 0x55f3523b9a80 2024-04-21T09:11:29.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.780+0000 7fdbb97df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f3523111a0 con 0x55f3523b0580 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbadffb640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdba00a3d20 0x7fdba00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54228/0 (socket says 172.21.15.66:54228) 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 -- 172.21.15.66:0/474187642 learned_addr learned my addr 172.21.15.66:0/474187642 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb519a640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdba00a3d20 0x7fdba00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbaffff640 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbaffff640 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbadffb640 1 -- 172.21.15.66:0/4217156381 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdba80d0ed0 con 0x7fdbb019cf90 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 -- 172.21.15.66:0/474187642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 msgr2=0x55f3523b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 -- 172.21.15.66:0/474187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 msgr2=0x55f3523bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 -- 172.21.15.66:0/474187642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3520f17d0 con 0x55f3523b0ea0 2024-04-21T09:11:29.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb4999640 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b9540 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdba400ac20 tx=0x7fdba400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be3d41bc56612c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdbb519a640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdba00a3d20 0x7fdba00a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdba8098f60 tx=0x7fdba8098e90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdb9affd640 1 -- 172.21.15.66:0/474187642 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba401d070 con 0x55f3523b0ea0 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdb9affd640 1 -- 172.21.15.66:0/474187642 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba4018af0 con 0x55f3523b0ea0 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.781+0000 7fdb9affd640 1 -- 172.21.15.66:0/474187642 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4010a40 con 0x55f3523b0ea0 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 -- 172.21.15.66:0/474187642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 msgr2=0x55f3523b9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b9540 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdba400ac20 tx=0x7fdba400f790 comp rx=0 tx=0).stop 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 -- 172.21.15.66:0/474187642 shutdown_connections 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b9540 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 --2- 172.21.15.66:0/474187642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 -- 172.21.15.66:0/474187642 >> 172.21.15.66:0/474187642 conn(0x55f3523aacb0 msgr2=0x55f3523ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 -- 172.21.15.66:0/474187642 shutdown_connections 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 -- 172.21.15.66:0/474187642 wait complete. 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.782+0000 7fdbb97df740 1 -- start start 2024-04-21T09:11:29.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 1 Processor -- start 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0580 0x55f3523b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b9a80 0x55f3523b7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3523bc870 con 0x55f3523b0ea0 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3522ce2f0 con 0x55f3523b0580 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f3523bc9f0 con 0x55f3523b9a80 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb97df740 10 client.0 _ll_register_callbacks cb 0x55f3522db580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54230/0 (socket says 172.21.15.66:54230) 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 -- 172.21.15.66:0/555570673 learned_addr learned my addr 172.21.15.66:0/555570673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbaffff640 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b9a80 0x55f3523b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbaffff640 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0580 0x55f3523b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 -- 172.21.15.66:0/555570673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b9a80 msgr2=0x55f3523b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b9a80 0x55f3523b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 -- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0580 msgr2=0x55f3523b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0580 0x55f3523b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.783+0000 7fdbb4999640 1 -- 172.21.15.66:0/555570673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3520273f0 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb4999640 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b7030 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fdba4010680 tx=0x7fdba40186d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece52b55bf6f0c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdb8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdb8ffff640 1 -- 172.21.15.66:0/555570673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba401d050 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdb8ffff640 1 -- 172.21.15.66:0/555570673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdba4005240 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba4018af0 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdb8ffff640 1 -- 172.21.15.66:0/555570673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdba4032050 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 10 client.16635 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f351fe70f0 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdb8ffff640 1 -- 172.21.15.66:0/555570673 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba4018af0 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdb8ffff640 1 -- 172.21.15.66:0/555570673 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdba40046c0 con 0x55f3523b0ea0 2024-04-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 10 client.16635 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 20 client.16635 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3523c9a60 0x55f3523cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 4 client.16635 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.784+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f352322c00 con 0x55f3523c9a60 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.785+0000 7fdbaffff640 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3523c9a60 0x55f3523cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.785+0000 7fdbaffff640 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3523c9a60 0x55f3523cbe50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a507fa42cbddd3 server_cookie=a4d165d4cba5e32d in_seq=0 out_seq=0 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.785+0000 7fdb8ffff640 10 client.16635 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdb8ffff640 1 -- 172.21.15.66:0/555570673 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f352322c00 con 0x55f3523c9a60 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdb8ffff640 10 client.16635 handle_command_reply: tid=0 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 client.16635 shutdown 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 msgr2=0x55f3523b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b7030 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fdba4010680 tx=0x7fdba40186d0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 shutdown_connections 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3523c9a60 0x55f3523cbe50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b9a80 0x55f3523b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0ea0 0x55f3523b7030 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 --2- 172.21.15.66:0/555570673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0580 0x55f3523b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdb8ffff640 0 client.16635 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 >> 172.21.15.66:0/555570673 conn(0x55f3523aacb0 msgr2=0x55f3523aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 shutdown_connections 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 1 -- 172.21.15.66:0/555570673 wait complete. 2024-04-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.786+0000 7fdbb97df740 20 client.16635 trim_cache size 0 max 16384 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 Processor -- start 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 -- start start 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0ea0 0x55f3523c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b9a80 0x55f3523ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01b1a60 con 0x55f3523b9a80 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01af330 con 0x55f3523b0ea0 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.804+0000 7fdbb97df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01ad160 con 0x55f3523b0580 2024-04-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b9a80 0x55f3523ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b9a80 0x55f3523ac7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54246/0 (socket says 172.21.15.66:54246) 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 -- 172.21.15.66:0/2233805225 learned_addr learned my addr 172.21.15.66:0/2233805225 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbb4999640 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0ea0 0x55f3523c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 -- 172.21.15.66:0/2233805225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 msgr2=0x55f3523b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 -- 172.21.15.66:0/2233805225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0ea0 msgr2=0x55f3523c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0ea0 0x55f3523c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 -- 172.21.15.66:0/2233805225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3523b5040 con 0x55f3523b9a80 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b9a80 0x55f3523ac7f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009230 tx=0x7fdb9c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad4ab7d953d76c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdb9a7fc640 1 -- 172.21.15.66:0/2233805225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c050070 con 0x55f3523b9a80 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdb9a7fc640 1 -- 172.21.15.66:0/2233805225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9c02cc90 con 0x55f3523b9a80 2024-04-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.805+0000 7fdb9a7fc640 1 -- 172.21.15.66:0/2233805225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c04c2a0 con 0x55f3523b9a80 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 -- 172.21.15.66:0/2233805225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b9a80 msgr2=0x55f3523ac7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b9a80 0x55f3523ac7f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009230 tx=0x7fdb9c005fd0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 -- 172.21.15.66:0/2233805225 shutdown_connections 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b9a80 0x55f3523ac7f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b0ea0 0x55f3523c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2233805225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0580 0x55f3523b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 -- 172.21.15.66:0/2233805225 >> 172.21.15.66:0/2233805225 conn(0x55f3523aacb0 msgr2=0x55f3523ac0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 -- 172.21.15.66:0/2233805225 shutdown_connections 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 -- 172.21.15.66:0/2233805225 wait complete. 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 -- start start 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.806+0000 7fdbb97df740 1 Processor -- start 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0580 0x55f3523c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0ea0 0x55f3523bd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01ad7f0 con 0x55f3523b0580 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01aec60 con 0x55f3523b9a80 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01adec0 con 0x55f3523b0ea0 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 10 client.0 _ll_register_callbacks cb 0x7fdba401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0580 0x55f3523c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0580 0x55f3523c1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54256/0 (socket says 172.21.15.66:54256) 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 -- 172.21.15.66:0/2627321181 learned_addr learned my addr 172.21.15.66:0/2627321181 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb4999640 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0ea0 0x55f3523bd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 -- 172.21.15.66:0/2627321181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0ea0 msgr2=0x55f3523bd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0ea0 0x55f3523bd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 -- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 msgr2=0x55f3523bd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 -- 172.21.15.66:0/2627321181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3523bde60 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbaffff640 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0580 0x55f3523c1f10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009430 tx=0x7fdb9c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3b91650ea0b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdb8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdb8ffff640 1 -- 172.21.15.66:0/2627321181 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c058070 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdb8ffff640 1 -- 172.21.15.66:0/2627321181 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fdb9c04c4a0 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.807+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c02cc90 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdb8ffff640 1 -- 172.21.15.66:0/2627321181 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb9c063050 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 10 client.16641 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f35242fbb0 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdb8ffff640 1 -- 172.21.15.66:0/2627321181 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9c02cc90 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdb8ffff640 1 -- 172.21.15.66:0/2627321181 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdb9c042440 con 0x55f3523b0580 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 10 client.16641 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 20 client.16641 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3524301f0 0x55f352432610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 4 client.16641 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f35231ca70 con 0x55f3524301f0 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.808+0000 7fdbb4999640 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3524301f0 0x55f352432610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb4999640 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3524301f0 0x55f352432610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fed39777f10e13 server_cookie=a039d8d36aaf5788 in_seq=0 out_seq=0 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdb8ffff640 10 client.16641 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdb8ffff640 1 -- 172.21.15.66:0/2627321181 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f35231ca70 con 0x55f3524301f0 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdb8ffff640 10 client.16641 handle_command_reply: tid=0 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 client.16641 shutdown 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0580 msgr2=0x55f3523c1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0580 0x55f3523c1f10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009430 tx=0x7fdb9c009680 comp rx=0 tx=0).stop 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 shutdown_connections 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f3524301f0 0x55f352432610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3523b9a80 0x55f3523bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3523b0ea0 0x55f3523bd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdb8ffff640 0 client.16641 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 --2- 172.21.15.66:0/2627321181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3523b0580 0x55f3523c1f10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 >> 172.21.15.66:0/2627321181 conn(0x55f3523aacb0 msgr2=0x55f3523b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 shutdown_connections 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 1 -- 172.21.15.66:0/2627321181 wait complete. 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.809+0000 7fdbb97df740 20 client.16641 trim_cache size 0 max 16384 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdba00a3d20 msgr2=0x7fdba00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdba00a3d20 0x7fdba00a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdba8098f60 tx=0x7fdba8098e90 comp rx=0 tx=0).stop 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb019cf90 msgr2=0x7fdbb019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb019cf90 0x7fdbb019d440 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fdba800cce0 tx=0x7fdba800a620 comp rx=0 tx=0).stop 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 shutdown_connections 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fdba00a3d20 0x7fdba00a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb019cf90 0x7fdbb019d440 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0106180 0x7fdbb0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 --2- 172.21.15.66:0/4217156381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb01010b0 0x7fdbb01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 >> 172.21.15.66:0/4217156381 conn(0x7fdbb00771d0 msgr2=0x7fdbb006a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 shutdown_connections 2024-04-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.810+0000 7fdbb6bfd640 1 -- 172.21.15.66:0/4217156381 wait complete. 2024-04-21T09:11:29.820 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:11:29.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:11:29.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:29 smithi186 ceph-mon[31505]: pgmap v101: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:29.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:29 smithi083 ceph-mon[31413]: pgmap v101: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 -- 172.21.15.66:0/375786051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f270410c5a0 msgr2=0x7f270410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 --2- 172.21.15.66:0/375786051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f270410c5a0 0x7f270410e9d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009ca0 tx=0x7f26fc02c180 comp rx=0 tx=0).stop 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 -- 172.21.15.66:0/375786051 shutdown_connections 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 --2- 172.21.15.66:0/375786051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f270410c5a0 0x7f270410e9d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 --2- 172.21.15.66:0/375786051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2704103af0 0x7f270410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 --2- 172.21.15.66:0/375786051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27041031d0 0x7f27041035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 -- 172.21.15.66:0/375786051 >> 172.21.15.66:0/375786051 conn(0x7f27040771d0 msgr2=0x7f27040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:29.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 -- 172.21.15.66:0/375786051 shutdown_connections 2024-04-21T09:11:29.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.964+0000 7f270a104640 1 -- 172.21.15.66:0/375786051 wait complete. 2024-04-21T09:11:29.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 Processor -- start 2024-04-21T09:11:29.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 -- start start 2024-04-21T09:11:29.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27041031d0 0x7f2704198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2704103af0 0x7f2704198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270410c5a0 0x7f270419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27041121a0 con 0x7f27041031d0 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2704112020 con 0x7f270410c5a0 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f270a104640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2704112320 con 0x7f2704103af0 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27041031d0 0x7f2704198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27041031d0 0x7f2704198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54284/0 (socket says 172.21.15.66:54284) 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 -- 172.21.15.66:0/782172780 learned_addr learned my addr 172.21.15.66:0/782172780 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f2703fff640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270410c5a0 0x7f270419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f2702ffd640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2704103af0 0x7f2704198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 -- 172.21.15.66:0/782172780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2704103af0 msgr2=0x7f2704198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2704103af0 0x7f2704198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 -- 172.21.15.66:0/782172780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270410c5a0 msgr2=0x7f270419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270410c5a0 0x7f270419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.965+0000 7f27037fe640 1 -- 172.21.15.66:0/782172780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270419d600 con 0x7f27041031d0 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.966+0000 7f27037fe640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27041031d0 0x7f2704198660 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c920 tx=0x7f26f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.966+0000 7f2700ff9640 1 -- 172.21.15.66:0/782172780 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8007c70 con 0x7f27041031d0 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.966+0000 7f2700ff9640 1 -- 172.21.15.66:0/782172780 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f8007e10 con 0x7f27041031d0 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.966+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270419d8f0 con 0x7f27041031d0 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.966+0000 7f2700ff9640 1 -- 172.21.15.66:0/782172780 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8018820 con 0x7f27041031d0 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.966+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041a5180 con 0x7f27041031d0 2024-04-21T09:11:29.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.967+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2704069230 con 0x7f27041031d0 2024-04-21T09:11:29.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.968+0000 7f2700ff9640 1 -- 172.21.15.66:0/782172780 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f26f800f450 con 0x7f27041031d0 2024-04-21T09:11:29.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.969+0000 7f2700ff9640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26ec0a3d20 0x7f26ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:29.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.969+0000 7f2702ffd640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26ec0a3d20 0x7f26ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:29.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.969+0000 7f2700ff9640 1 -- 172.21.15.66:0/782172780 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f26f80d0ca0 con 0x7f27041031d0 2024-04-21T09:11:29.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.969+0000 7f2702ffd640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26ec0a3d20 0x7f26ec0a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005d80 tx=0x7f26f4005c90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:29.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:29.974+0000 7f2700ff9640 1 -- 172.21.15.66:0/782172780 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26f8014030 con 0x7f27041031d0 2024-04-21T09:11:30.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.202+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2704199960 con 0x7f27041031d0 2024-04-21T09:11:30.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.203+0000 7f2700ff9640 1 -- 172.21.15.66:0/782172780 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f26f809cfd0 con 0x7f27041031d0 2024-04-21T09:11:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:11:30.203 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:11:30.203 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:11:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26ec0a3d20 msgr2=0x7f26ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26ec0a3d20 0x7f26ec0a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005d80 tx=0x7f26f4005c90 comp rx=0 tx=0).stop 2024-04-21T09:11:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27041031d0 msgr2=0x7f2704198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27041031d0 0x7f2704198660 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c920 tx=0x7f26f800cde0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 shutdown_connections 2024-04-21T09:11:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26ec0a3d20 0x7f26ec0a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270410c5a0 0x7f270419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2704103af0 0x7f2704198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 --2- 172.21.15.66:0/782172780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27041031d0 0x7f2704198660 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.205+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 >> 172.21.15.66:0/782172780 conn(0x7f27040771d0 msgr2=0x7f27040fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.206+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 shutdown_connections 2024-04-21T09:11:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.206+0000 7f270a104640 1 -- 172.21.15.66:0/782172780 wait complete. 2024-04-21T09:11:30.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 -- 172.21.15.66:0/225482096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 msgr2=0x7f833410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 --2- 172.21.15.66:0/225482096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f833410ee40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f832c009ca0 tx=0x7f832c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 -- 172.21.15.66:0/225482096 shutdown_connections 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 --2- 172.21.15.66:0/225482096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f833410ee40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 --2- 172.21.15.66:0/225482096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833406b1f0 0x7f8334106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 --2- 172.21.15.66:0/225482096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833406a840 0x7f833406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 -- 172.21.15.66:0/225482096 >> 172.21.15.66:0/225482096 conn(0x7f8334077eb0 msgr2=0x7f83340782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 -- 172.21.15.66:0/225482096 shutdown_connections 2024-04-21T09:11:30.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.320+0000 7f833b10e640 1 -- 172.21.15.66:0/225482096 wait complete. 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 Processor -- start 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 -- start start 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833406a840 0x7f8334198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f8334198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833419cff0 0x7f833419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8334112230 con 0x7f8334107270 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83341120b0 con 0x7f833419cff0 2024-04-21T09:11:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f833b10e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83341123b0 con 0x7f833406a840 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f8334198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f8334198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54300/0 (socket says 172.21.15.66:54300) 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 -- 172.21.15.66:0/2188854509 learned_addr learned my addr 172.21.15.66:0/2188854509 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f83396ab640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833419cff0 0x7f833419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8338eaa640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833406a840 0x7f8334198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 -- 172.21.15.66:0/2188854509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833406a840 msgr2=0x7f8334198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833406a840 0x7f8334198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 -- 172.21.15.66:0/2188854509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833419cff0 msgr2=0x7f833419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833419cff0 0x7f833419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.321+0000 7f8333fff640 1 -- 172.21.15.66:0/2188854509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833419dae0 con 0x7f8334107270 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.322+0000 7f8333fff640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f8334198c70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f832000b450 tx=0x7f832000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.322+0000 7f8331ffb640 1 -- 172.21.15.66:0/2188854509 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320015020 con 0x7f8334107270 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.322+0000 7f8331ffb640 1 -- 172.21.15.66:0/2188854509 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8320011d40 con 0x7f8334107270 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.322+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833419ddd0 con 0x7f8334107270 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.322+0000 7f8331ffb640 1 -- 172.21.15.66:0/2188854509 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832000f340 con 0x7f8334107270 2024-04-21T09:11:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.322+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a55c0 con 0x7f8334107270 2024-04-21T09:11:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.324+0000 7f8331ffb640 1 -- 172.21.15.66:0/2188854509 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8320018020 con 0x7f8334107270 2024-04-21T09:11:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.324+0000 7f833ddd9740 1 Processor -- start 2024-04-21T09:11:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.324+0000 7f833ddd9740 1 -- start start 2024-04-21T09:11:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.324+0000 7f8331ffb640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f83240a3d20 0x7f83240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549a2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a2d60 0x55c9549ab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f833ddd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c9549071f0 con 0x55c9549a2d60 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f833ddd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9548a44b0 con 0x55c9549a23b0 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f833ddd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c9549031b0 con 0x55c9549abe60 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f83396ab640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f83240a3d20 0x7f83240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8331ffb640 1 -- 172.21.15.66:0/2188854509 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f83200d0460 con 0x7f8334107270 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a2d60 0x55c9549ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a2d60 0x55c9549ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54308/0 (socket says 172.21.15.66:54308) 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 -- 172.21.15.66:0/1112403098 learned_addr learned my addr 172.21.15.66:0/1112403098 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f83396ab640 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 -- 172.21.15.66:0/1112403098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 msgr2=0x55c9549ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 -- 172.21.15.66:0/1112403098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 msgr2=0x55c9549a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 -- 172.21.15.66:0/1112403098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9548961b0 con 0x55c9549a2d60 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f83396ab640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f83240a3d20 0x7f83240a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f832c004af0 tx=0x7f832c032ca0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f8333fff640 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a2d60 0x55c9549ab890 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f83200cd9b0 tx=0x7f832009bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c855e21ddd3542 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f831effd640 1 -- 172.21.15.66:0/1112403098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320086e60 con 0x55c9549a2d60 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f831effd640 1 -- 172.21.15.66:0/1112403098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83200a0550 con 0x55c9549a2d60 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.325+0000 7f831effd640 1 -- 172.21.15.66:0/1112403098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320010500 con 0x55c9549a2d60 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 -- 172.21.15.66:0/1112403098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a2d60 msgr2=0x55c9549ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a2d60 0x55c9549ab890 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f83200cd9b0 tx=0x7f832009bc30 comp rx=0 tx=0).stop 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 -- 172.21.15.66:0/1112403098 shutdown_connections 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a2d60 0x55c9549ab890 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1112403098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 -- 172.21.15.66:0/1112403098 >> 172.21.15.66:0/1112403098 conn(0x55c95499cba0 msgr2=0x55c95499cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 -- 172.21.15.66:0/1112403098 shutdown_connections 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 -- 172.21.15.66:0/1112403098 wait complete. 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.326+0000 7f833ddd9740 1 -- start start 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 1 Processor -- start 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549abe60 0x55c9549afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c9549b2ce0 con 0x55c9549abe60 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9548c0500 con 0x55c9549a23b0 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c9549b2e60 con 0x55c9549a2d60 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f833ddd9740 10 client.0 _ll_register_callbacks cb 0x55c95488eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549abe60 0x55c9549afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549abe60 0x55c9549afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54324/0 (socket says 172.21.15.66:54324) 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 -- 172.21.15.66:0/1495869709 learned_addr learned my addr 172.21.15.66:0/1495869709 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f8333fff640 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 -- 172.21.15.66:0/1495869709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 msgr2=0x55c9549af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 -- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 msgr2=0x55c9549b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 -- 172.21.15.66:0/1495869709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9546193f0 con 0x55c9549abe60 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.327+0000 7f83396ab640 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549abe60 0x55c9549afba0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f832c009ca0 tx=0x7f832c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf243954c9ed119 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f8313fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f8313fff640 1 -- 172.21.15.66:0/1495869709 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c051060 con 0x55c9549abe60 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f8313fff640 1 -- 172.21.15.66:0/1495869709 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f832c004450 con 0x55c9549abe60 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83200a0550 con 0x55c9549abe60 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f8313fff640 1 -- 172.21.15.66:0/1495869709 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f832c058050 con 0x55c9549abe60 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 10 client.16659 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:30.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9546e37d0 con 0x55c9549abe60 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f8313fff640 1 -- 172.21.15.66:0/1495869709 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83200a0550 con 0x55c9549abe60 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f8313fff640 1 -- 172.21.15.66:0/1495869709 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f832c03f070 con 0x55c9549abe60 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 10 client.16659 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 10 client.16659 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 20 client.16659 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c9549bffd0 0x55c9549c2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 4 client.16659 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.328+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9548a6930 con 0x55c9549bffd0 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.329+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c9549bffd0 0x55c9549c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.329+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c9549bffd0 0x55c9549c2430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=299a89aa1b1e771d server_cookie=e44cd4014eb5879a in_seq=0 out_seq=0 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.329+0000 7f8313fff640 10 client.16659 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f8313fff640 1 -- 172.21.15.66:0/1495869709 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9548a6930 con 0x55c9549bffd0 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f8313fff640 10 client.16659 handle_command_reply: tid=0 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 client.16659 shutdown 2024-04-21T09:11:30.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549abe60 msgr2=0x55c9549afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549abe60 0x55c9549afba0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f832c009ca0 tx=0x7f832c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 shutdown_connections 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c9549bffd0 0x55c9549c2430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549abe60 0x55c9549afba0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f8313fff640 0 client.16659 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1495869709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a23b0 0x55c9549b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 >> 172.21.15.66:0/1495869709 conn(0x55c95499cba0 msgr2=0x55c95499e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 shutdown_connections 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 1 -- 172.21.15.66:0/1495869709 wait complete. 2024-04-21T09:11:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.330+0000 7f833ddd9740 20 client.16659 trim_cache size 0 max 16384 2024-04-21T09:11:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.343+0000 7f833ddd9740 1 Processor -- start 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.343+0000 7f833ddd9740 1 -- start start 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.343+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.343+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a2d60 0x55c9549ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f833ddd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83341af9d0 con 0x55c9549a23b0 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f833ddd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ad2a0 con 0x55c9549a2d60 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f833ddd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ab0d0 con 0x55c9549abe60 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549bbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54334/0 (socket says 172.21.15.66:54334) 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 -- 172.21.15.66:0/303865379 learned_addr learned my addr 172.21.15.66:0/303865379 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f83396ab640 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 -- 172.21.15.66:0/303865379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 msgr2=0x55c9549ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8333fff640 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a2d60 0x55c9549ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 -- 172.21.15.66:0/303865379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a2d60 msgr2=0x55c9549ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a2d60 0x55c9549ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 -- 172.21.15.66:0/303865379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9549a1db0 con 0x55c9549a23b0 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f83396ab640 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f8338eaa640 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549bbbe0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f832800b620 tx=0x7f832800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6303da1197b5ab5e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f831e7fc640 1 -- 172.21.15.66:0/303865379 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328003d20 con 0x55c9549a23b0 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f831e7fc640 1 -- 172.21.15.66:0/303865379 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832801d920 con 0x55c9549a23b0 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.344+0000 7f831e7fc640 1 -- 172.21.15.66:0/303865379 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328004a00 con 0x55c9549a23b0 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 -- 172.21.15.66:0/303865379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 msgr2=0x55c9549bbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549bbbe0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f832800b620 tx=0x7f832800bd90 comp rx=0 tx=0).stop 2024-04-21T09:11:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 -- 172.21.15.66:0/303865379 shutdown_connections 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549abe60 0x55c9549ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549a2d60 0x55c9549ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 --2- 172.21.15.66:0/303865379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549bbbe0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 -- 172.21.15.66:0/303865379 >> 172.21.15.66:0/303865379 conn(0x55c95499cba0 msgr2=0x55c95499b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 -- 172.21.15.66:0/303865379 shutdown_connections 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 -- 172.21.15.66:0/303865379 wait complete. 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 -- start start 2024-04-21T09:11:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.345+0000 7f833ddd9740 1 Processor -- start 2024-04-21T09:11:30.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549abe60 0x55c9549ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ab760 con 0x55c9549a23b0 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83341acbd0 con 0x55c9549abe60 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83341abe30 con 0x55c9549a2d60 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 10 client.0 _ll_register_callbacks cb 0x7f832c0327f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54336/0 (socket says 172.21.15.66:54336) 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 -- 172.21.15.66:0/1723920395 learned_addr learned my addr 172.21.15.66:0/1723920395 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8333fff640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f83396ab640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549abe60 0x55c9549ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 -- 172.21.15.66:0/1723920395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 msgr2=0x55c9549b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 -- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549abe60 msgr2=0x55c9549ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549abe60 0x55c9549ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 -- 172.21.15.66:0/1723920395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9549acbd0 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f83396ab640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549abe60 0x55c9549ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8338eaa640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549b80f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f83280056c0 tx=0x7f832801d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd8cbfc710f47f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8313fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8313fff640 1 -- 172.21.15.66:0/1723920395 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328011440 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f8313fff640 1 -- 172.21.15.66:0/1723920395 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f83280115e0 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.346+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c954914c10 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f8313fff640 1 -- 172.21.15.66:0/1723920395 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8328037050 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 10 client.16665 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c954a26020 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f8313fff640 1 -- 172.21.15.66:0/1723920395 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c954914c10 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f8313fff640 1 -- 172.21.15.66:0/1723920395 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f83280269c0 con 0x55c9549a23b0 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 10 client.16665 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 10 client.16665 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:11:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 20 client.16665 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c954a266b0 0x55c954a28ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 4 client.16665 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c95490ea80 con 0x55c954a266b0 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.347+0000 7f8333fff640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c954a266b0 0x55c954a28ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f8313fff640 10 client.16665 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f8333fff640 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c954a266b0 0x55c954a28ad0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59f4b189b79652c3 server_cookie=50bad0a76ec76053 in_seq=0 out_seq=0 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f8313fff640 1 -- 172.21.15.66:0/1723920395 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c95490ea80 con 0x55c954a266b0 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f8313fff640 10 client.16665 handle_command_reply: tid=0 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 client.16665 shutdown 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 msgr2=0x55c9549b80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549b80f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f83280056c0 tx=0x7f832801d790 comp rx=0 tx=0).stop 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 shutdown_connections 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55c954a266b0 0x55c954a28ad0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9549abe60 0x55c9549ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9549a2d60 0x55c9549b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 --2- 172.21.15.66:0/1723920395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9549a23b0 0x55c9549b80f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f8313fff640 0 client.16665 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 >> 172.21.15.66:0/1723920395 conn(0x55c95499cba0 msgr2=0x55c9549aa080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 shutdown_connections 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 1 -- 172.21.15.66:0/1723920395 wait complete. 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.348+0000 7f833ddd9740 20 client.16665 trim_cache size 0 max 16384 2024-04-21T09:11:30.354 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f83240a3d20 msgr2=0x7f83240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f83240a3d20 0x7f83240a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f832c004af0 tx=0x7f832c032ca0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 msgr2=0x7f8334198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f8334198c70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f832000b450 tx=0x7f832000b910 comp rx=0 tx=0).stop 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 shutdown_connections 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f83240a3d20 0x7f83240a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833419cff0 0x7f833419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8334107270 0x7f8334198c70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 --2- 172.21.15.66:0/2188854509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833406a840 0x7f8334198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 >> 172.21.15.66:0/2188854509 conn(0x7f8334077eb0 msgr2=0x7f8334100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 shutdown_connections 2024-04-21T09:11:30.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.349+0000 7f833b10e640 1 -- 172.21.15.66:0/2188854509 wait complete. 2024-04-21T09:11:30.359 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 -- 172.21.15.66:0/2352707133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 msgr2=0x7f232c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 --2- 172.21.15.66:0/2352707133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c102570 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f231c009900 tx=0x7f231c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 -- 172.21.15.66:0/2352707133 shutdown_connections 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 --2- 172.21.15.66:0/2352707133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c107260 0x7f232c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 --2- 172.21.15.66:0/2352707133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f232c102b40 0x7f232c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 --2- 172.21.15.66:0/2352707133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c102570 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 -- 172.21.15.66:0/2352707133 >> 172.21.15.66:0/2352707133 conn(0x7f232c0fd990 msgr2=0x7f232c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 -- 172.21.15.66:0/2352707133 shutdown_connections 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.465+0000 7f2330d11640 1 -- 172.21.15.66:0/2352707133 wait complete. 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 Processor -- start 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 -- start start 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f232c107260 0x7f232c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c19cfe0 0x7f232c19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1121c0 con 0x7f232c102190 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112040 con 0x7f232c19cfe0 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f2330d11640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112340 con 0x7f232c107260 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f232a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f232a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54358/0 (socket says 172.21.15.66:54358) 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.466+0000 7f232a59c640 1 -- 172.21.15.66:0/1610198807 learned_addr learned my addr 172.21.15.66:0/1610198807 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f2323fff640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f232c107260 0x7f232c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232ad9d640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c19cfe0 0x7f232c19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232a59c640 1 -- 172.21.15.66:0/1610198807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f232c107260 msgr2=0x7f232c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232a59c640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f232c107260 0x7f232c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232a59c640 1 -- 172.21.15.66:0/1610198807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c19cfe0 msgr2=0x7f232c19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232a59c640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c19cfe0 0x7f232c19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232a59c640 1 -- 172.21.15.66:0/1610198807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c19dad0 con 0x7f232c102190 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232ad9d640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c19cfe0 0x7f232c19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f232a59c640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c198720 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f231c005940 tx=0x7f231c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f23237fe640 1 -- 172.21.15.66:0/1610198807 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c043020 con 0x7f232c102190 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c19dd60 con 0x7f232c102190 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f23237fe640 1 -- 172.21.15.66:0/1610198807 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c02cbe0 con 0x7f232c102190 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f23237fe640 1 -- 172.21.15.66:0/1610198807 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c003bf0 con 0x7f232c102190 2024-04-21T09:11:30.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.467+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a55b0 con 0x7f232c102190 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.469+0000 7f23237fe640 1 -- 172.21.15.66:0/1610198807 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f231c046020 con 0x7f232c102190 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.469+0000 7f23339d6740 1 Processor -- start 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23339d6740 1 -- start start 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23339d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23339d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23339d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23339d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3fa8161f0 con 0x55f3fa8baa00 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23339d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3fa7b34b0 con 0x55f3fa8b1e80 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23339d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f3fa8121b0 con 0x55f3fa8b14d0 2024-04-21T09:11:30.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f2323fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f232a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f2323fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8bce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54364/0 (socket says 172.21.15.66:54364) 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f2323fff640 1 -- 172.21.15.66:0/3657215232 learned_addr learned my addr 172.21.15.66:0/3657215232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f23237fe640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f23180a3d20 0x7f23180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.470+0000 7f232ad9d640 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232ad9d640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f23180a3d20 0x7f23180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f23237fe640 1 -- 172.21.15.66:0/1610198807 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f231c0f2d80 con 0x7f232c102190 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232a59c640 1 -- 172.21.15.66:0/3657215232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 msgr2=0x55f3fa8b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232a59c640 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232a59c640 1 -- 172.21.15.66:0/3657215232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 msgr2=0x55f3fa8bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232a59c640 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232a59c640 1 -- 172.21.15.66:0/3657215232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3fa7a51b0 con 0x55f3fa8b1e80 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f2323fff640 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232a59c640 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8ba430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f231c0f0270 tx=0x7f231c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43ee7280a7bb3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f232ad9d640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f23180a3d20 0x7f23180a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f23240072a0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f2320ff9640 1 -- 172.21.15.66:0/3657215232 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0be420 con 0x55f3fa8b1e80 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f2320ff9640 1 -- 172.21.15.66:0/3657215232 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c0bf1c0 con 0x55f3fa8b1e80 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.471+0000 7f2320ff9640 1 -- 172.21.15.66:0/3657215232 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0bcdc0 con 0x55f3fa8b1e80 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- 172.21.15.66:0/3657215232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 msgr2=0x55f3fa8ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8ba430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f231c0f0270 tx=0x7f231c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- 172.21.15.66:0/3657215232 shutdown_connections 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8ba430 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 --2- 172.21.15.66:0/3657215232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- 172.21.15.66:0/3657215232 >> 172.21.15.66:0/3657215232 conn(0x55f3fa8abba0 msgr2=0x55f3fa8abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- 172.21.15.66:0/3657215232 shutdown_connections 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- 172.21.15.66:0/3657215232 wait complete. 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- start start 2024-04-21T09:11:30.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 Processor -- start 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3fa8bd730 con 0x55f3fa8baa00 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3fa7cf500 con 0x55f3fa8b1e80 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.472+0000 7f23339d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f3fa8bd8b0 con 0x55f3fa8b14d0 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f23339d6740 10 client.0 _ll_register_callbacks cb 0x55f3fa79db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f2323fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54368/0 (socket says 172.21.15.66:54368) 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 -- 172.21.15.66:0/3103387097 learned_addr learned my addr 172.21.15.66:0/3103387097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232a59c640 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 -- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 msgr2=0x55f3fa8ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 -- 172.21.15.66:0/3103387097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 msgr2=0x55f3fa8b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 -- 172.21.15.66:0/3103387097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3fa5283f0 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232a59c640 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f232ad9d640 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b7ce0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2324010440 tx=0x7f2324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55566979e32fc14 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f2311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f2311ffb640 1 -- 172.21.15.66:0/3103387097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324022070 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f2311ffb640 1 -- 172.21.15.66:0/3103387097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2324004d20 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c0bf1c0 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f2311ffb640 1 -- 172.21.15.66:0/3103387097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f232402d050 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.473+0000 7f23339d6740 10 client.16677 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3fa5f27d0 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f2311ffb640 1 -- 172.21.15.66:0/3103387097 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c0bf1c0 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f2311ffb640 1 -- 172.21.15.66:0/3103387097 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f232401d040 con 0x55f3fa8baa00 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f23339d6740 10 client.16677 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:30.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f23339d6740 10 client.16677 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f23339d6740 20 client.16677 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f23339d6740 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa8ca970 0x55f3fa8ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f23339d6740 4 client.16677 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3fa7b5930 con 0x55f3fa8ca970 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.474+0000 7f232a59c640 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa8ca970 0x55f3fa8ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.475+0000 7f232a59c640 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa8ca970 0x55f3fa8ccdf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23e36b07bf826ec7 server_cookie=687abd6a8269f3b3 in_seq=0 out_seq=0 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.475+0000 7f2311ffb640 10 client.16677 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.475+0000 7f2311ffb640 1 -- 172.21.15.66:0/3103387097 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3fa7b5930 con 0x55f3fa8ca970 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.475+0000 7f2311ffb640 10 client.16677 handle_command_reply: tid=0 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.475+0000 7f23339d6740 1 client.16677 shutdown 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 msgr2=0x55f3fa8b7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b7ce0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2324010440 tx=0x7f2324004a40 comp rx=0 tx=0).stop 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 shutdown_connections 2024-04-21T09:11:30.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa8ca970 0x55f3fa8ccdf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b7ce0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f2311ffb640 0 client.16677 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 --2- 172.21.15.66:0/3103387097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 >> 172.21.15.66:0/3103387097 conn(0x55f3fa8abba0 msgr2=0x55f3fa8ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 shutdown_connections 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 1 -- 172.21.15.66:0/3103387097 wait complete. 2024-04-21T09:11:30.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.476+0000 7f23339d6740 20 client.16677 trim_cache size 0 max 16384 2024-04-21T09:11:30.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 Processor -- start 2024-04-21T09:11:30.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 -- start start 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1b1b70 con 0x55f3fa8b14d0 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1af440 con 0x55f3fa8baa00 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f23339d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1ad270 con 0x55f3fa8b1e80 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54376/0 (socket says 172.21.15.66:54376) 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 -- 172.21.15.66:0/2064997213 learned_addr learned my addr 172.21.15.66:0/2064997213 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f2323fff640 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232ad9d640 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 -- 172.21.15.66:0/2064997213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b1e80 msgr2=0x55f3fa8b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 -- 172.21.15.66:0/2064997213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8baa00 msgr2=0x55f3fa8b9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232a59c640 1 -- 172.21.15.66:0/2064997213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3fa8b82e0 con 0x55f3fa8b14d0 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.489+0000 7f232ad9d640 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f232a59c640 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c6520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f231c000c00 tx=0x7f231c0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf870946b96010 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f2313fff640 1 -- 172.21.15.66:0/2064997213 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0bad60 con 0x55f3fa8b14d0 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f2313fff640 1 -- 172.21.15.66:0/2064997213 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c03c350 con 0x55f3fa8b14d0 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f2313fff640 1 -- 172.21.15.66:0/2064997213 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c003e40 con 0x55f3fa8b14d0 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 -- 172.21.15.66:0/2064997213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b14d0 msgr2=0x55f3fa8c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c6520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f231c000c00 tx=0x7f231c0df040 comp rx=0 tx=0).stop 2024-04-21T09:11:30.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 -- 172.21.15.66:0/2064997213 shutdown_connections 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8baa00 0x55f3fa8b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 --2- 172.21.15.66:0/2064997213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c6520 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 -- 172.21.15.66:0/2064997213 >> 172.21.15.66:0/2064997213 conn(0x55f3fa8abba0 msgr2=0x55f3fa8b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 -- 172.21.15.66:0/2064997213 shutdown_connections 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.490+0000 7f23339d6740 1 -- 172.21.15.66:0/2064997213 wait complete. 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 -- start start 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 Processor -- start 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8baa00 0x55f3fa8be530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1ad900 con 0x55f3fa8b1e80 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1aed70 con 0x55f3fa8b14d0 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1adfd0 con 0x55f3fa8baa00 2024-04-21T09:11:30.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f23339d6740 10 client.0 _ll_register_callbacks cb 0x7f232400ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8bdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54382/0 (socket says 172.21.15.66:54382) 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 -- 172.21.15.66:0/320086980 learned_addr learned my addr 172.21.15.66:0/320086980 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f232ad9d640 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8baa00 0x55f3fa8be530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f232a59c640 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 -- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8baa00 msgr2=0x55f3fa8be530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8baa00 0x55f3fa8be530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 -- 172.21.15.66:0/320086980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b14d0 msgr2=0x55f3fa8c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.491+0000 7f2323fff640 1 -- 172.21.15.66:0/320086980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3fa8beb20 con 0x55f3fa8b1e80 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f2323fff640 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8bdff0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2314005b90 tx=0x7f2314011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0d91e3c3974917 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f2311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:30.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f2311ffb640 1 -- 172.21.15.66:0/320086980 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314005220 con 0x55f3fa8b1e80 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f2311ffb640 1 -- 172.21.15.66:0/320086980 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f23140053c0 con 0x55f3fa8b1e80 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c03c350 con 0x55f3fa8b1e80 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f2311ffb640 1 -- 172.21.15.66:0/320086980 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2314027050 con 0x55f3fa8b1e80 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 10 client.16683 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3fa930d00 con 0x55f3fa8b1e80 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f2311ffb640 1 -- 172.21.15.66:0/320086980 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c03c350 con 0x55f3fa8b1e80 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f2311ffb640 1 -- 172.21.15.66:0/320086980 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f231401b070 con 0x55f3fa8b1e80 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 10 client.16683 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 10 client.16683 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 20 client.16683 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa9314d0 0x55f3fa9338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 4 client.16683 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.492+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3fa823c10 con 0x55f3fa9314d0 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f232a59c640 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa9314d0 0x55f3fa9338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f2311ffb640 10 client.16683 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f232a59c640 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa9314d0 0x55f3fa9338f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=862863dd72c98baf server_cookie=9f2cc8036d4e0a54 in_seq=0 out_seq=0 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f2311ffb640 1 -- 172.21.15.66:0/320086980 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3fa823c10 con 0x55f3fa9314d0 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f2311ffb640 10 client.16683 handle_command_reply: tid=0 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 client.16683 shutdown 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b1e80 msgr2=0x55f3fa8bdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8bdff0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2314005b90 tx=0x7f2314011b50 comp rx=0 tx=0).stop 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 shutdown_connections 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3fa8baa00 0x55f3fa8be530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55f3fa9314d0 0x55f3fa9338f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3fa8b1e80 0x55f3fa8bdff0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 --2- 172.21.15.66:0/320086980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fa8b14d0 0x55f3fa8c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f2311ffb640 0 client.16683 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:11:30.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 >> 172.21.15.66:0/320086980 conn(0x55f3fa8abba0 msgr2=0x55f3fa8aa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 shutdown_connections 2024-04-21T09:11:30.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 1 -- 172.21.15.66:0/320086980 wait complete. 2024-04-21T09:11:30.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.493+0000 7f23339d6740 20 client.16683 trim_cache size 0 max 16384 2024-04-21T09:11:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f23180a3d20 msgr2=0x7f23180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f23180a3d20 0x7f23180a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f23240072a0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 msgr2=0x7f232c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c198720 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f231c005940 tx=0x7f231c02c7e0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 shutdown_connections 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f23180a3d20 0x7f23180a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c19cfe0 0x7f232c19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f232c107260 0x7f232c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 --2- 172.21.15.66:0/1610198807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c102190 0x7f232c198720 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 >> 172.21.15.66:0/1610198807 conn(0x7f232c0fd990 msgr2=0x7f232c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 shutdown_connections 2024-04-21T09:11:30.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.494+0000 7f2330d11640 1 -- 172.21.15.66:0/1610198807 wait complete. 2024-04-21T09:11:30.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:30 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/782172780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:11:30.504 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 -- 172.21.15.66:0/2264726621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 msgr2=0x7ff8701035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 --2- 172.21.15.66:0/2264726621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff8701035c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff864009900 tx=0x7ff86402bd60 comp rx=0 tx=0).stop 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 -- 172.21.15.66:0/2264726621 shutdown_connections 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 --2- 172.21.15.66:0/2264726621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87010c520 0x7ff87010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 --2- 172.21.15.66:0/2264726621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff870103b00 0x7ff87010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 --2- 172.21.15.66:0/2264726621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff8701035c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 -- 172.21.15.66:0/2264726621 >> 172.21.15.66:0/2264726621 conn(0x7ff8700771d0 msgr2=0x7ff8700775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 -- 172.21.15.66:0/2264726621 shutdown_connections 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 -- 172.21.15.66:0/2264726621 wait complete. 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 Processor -- start 2024-04-21T09:11:30.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.609+0000 7ff876426640 1 -- start start 2024-04-21T09:11:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff876426640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff870100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff876426640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff870103b00 0x7ff870101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff876426640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87010c520 0x7ff8701018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff876426640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff870112200 con 0x7ff8701031e0 2024-04-21T09:11:30.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff876426640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff870112080 con 0x7ff87010c520 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff876426640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff870112380 con 0x7ff870103b00 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff870100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff870100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54410/0 (socket says 172.21.15.66:54410) 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 -- 172.21.15.66:0/1884541321 learned_addr learned my addr 172.21.15.66:0/1884541321 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff8749c3640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87010c520 0x7ff8701018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff870103b00 0x7ff870101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 -- 172.21.15.66:0/1884541321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff870103b00 msgr2=0x7ff870101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff870103b00 0x7ff870101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 -- 172.21.15.66:0/1884541321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87010c520 msgr2=0x7ff8701018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87010c520 0x7ff8701018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 -- 172.21.15.66:0/1884541321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8701a6c20 con 0x7ff8701031e0 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86ffff640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff870100e30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff864000c00 tx=0x7ff86402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86d7fa640 1 -- 172.21.15.66:0/1884541321 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864043020 con 0x7ff8701031e0 2024-04-21T09:11:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.610+0000 7ff86d7fa640 1 -- 172.21.15.66:0/1884541321 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff86402dde0 con 0x7ff8701031e0 2024-04-21T09:11:30.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.611+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8701a6eb0 con 0x7ff8701031e0 2024-04-21T09:11:30.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.611+0000 7ff86d7fa640 1 -- 172.21.15.66:0/1884541321 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864003b80 con 0x7ff8701031e0 2024-04-21T09:11:30.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.611+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8701a7370 con 0x7ff8701031e0 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff86d7fa640 1 -- 172.21.15.66:0/1884541321 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff864046020 con 0x7ff8701031e0 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 Processor -- start 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 -- start start 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5557805481f0 con 0x5557805e34d0 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5557804e54b0 con 0x5557805e3e80 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff879010740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5557805441b0 con 0x5557805eca00 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff8749c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff8749c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805e38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54418/0 (socket says 172.21.15.66:54418) 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff8749c3640 1 -- 172.21.15.66:0/3579373602 learned_addr learned my addr 172.21.15.66:0/3579373602 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff86ffff640 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.613+0000 7ff86d7fa640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8580a3d50 0x7ff8580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8749c3640 1 -- 172.21.15.66:0/3579373602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 msgr2=0x5557805eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8749c3640 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805eee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8749c3640 1 -- 172.21.15.66:0/3579373602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 msgr2=0x5557805ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8749c3640 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8749c3640 1 -- 172.21.15.66:0/3579373602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557804d71b0 con 0x5557805e34d0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff86d7fa640 1 -- 172.21.15.66:0/1884541321 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff8640f2af0 con 0x7ff8701031e0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8749c3640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8580a3d50 0x7ff8580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8749c3640 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805e38b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff868009f70 tx=0x7ff868010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a04c414b5b9327a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8567fc640 1 -- 172.21.15.66:0/3579373602 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868016070 con 0x5557805e34d0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8567fc640 1 -- 172.21.15.66:0/3579373602 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff86801c440 con 0x5557805e34d0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.614+0000 7ff8567fc640 1 -- 172.21.15.66:0/3579373602 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86801b620 con 0x5557805e34d0 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 -- 172.21.15.66:0/3579373602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 msgr2=0x5557805e38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff8749c3640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8580a3d50 0x7ff8580a6210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff868011010 tx=0x7ff86801bd20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805e38b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff868009f70 tx=0x7ff868010800 comp rx=0 tx=0).stop 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 -- 172.21.15.66:0/3579373602 shutdown_connections 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 --2- 172.21.15.66:0/3579373602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805e38b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 -- 172.21.15.66:0/3579373602 >> 172.21.15.66:0/3579373602 conn(0x5557805ddba0 msgr2=0x5557805ddf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 -- 172.21.15.66:0/3579373602 shutdown_connections 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 -- 172.21.15.66:0/3579373602 wait complete. 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 -- start start 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.615+0000 7ff879010740 1 Processor -- start 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff879010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff879010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805e9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff879010740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff879010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5557805ef970 con 0x5557805e34d0 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff879010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555780501500 con 0x5557805e3e80 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff879010740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5557805efaf0 con 0x5557805eca00 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff879010740 10 client.0 _ll_register_callbacks cb 0x5557804cfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805ec2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54420/0 (socket says 172.21.15.66:54420) 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 -- 172.21.15.66:0/3283052950 learned_addr learned my addr 172.21.15.66:0/3283052950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff8749c3640 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 -- 172.21.15.66:0/3283052950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 msgr2=0x5557805e9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805e9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 -- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 msgr2=0x5557805e9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805e9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 -- 172.21.15.66:0/3283052950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578025a3f0 con 0x5557805e34d0 2024-04-21T09:11:30.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff86ffff640 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805ec2a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff8640f0040 tx=0x7ff8640035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55751e91a2aaf8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff84f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.616+0000 7ff84f7fe640 1 -- 172.21.15.66:0/3283052950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8640be190 con 0x5557805e34d0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff84f7fe640 1 -- 172.21.15.66:0/3283052950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff8640f0150 con 0x5557805e34d0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff86801c440 con 0x5557805e34d0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff84f7fe640 1 -- 172.21.15.66:0/3283052950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8640f0350 con 0x5557805e34d0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 10 client.16695 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557803247d0 con 0x5557805e34d0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff84f7fe640 1 -- 172.21.15.66:0/3283052950 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff86801c440 con 0x5557805e34d0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff84f7fe640 1 -- 172.21.15.66:0/3283052950 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff864005490 con 0x5557805e34d0 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 10 client.16695 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 20 client.16695 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5557805fcbe0 0x5557805ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 4 client.16695 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557804e7930 con 0x5557805fcbe0 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.617+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5557805fcbe0 0x5557805ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.618+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5557805fcbe0 0x5557805ff080 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbcb0ee831561687 server_cookie=69e5c53830585f0e in_seq=0 out_seq=0 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.618+0000 7ff84f7fe640 10 client.16695 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.618+0000 7ff84f7fe640 1 -- 172.21.15.66:0/3283052950 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557804e7930 con 0x5557805fcbe0 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.618+0000 7ff84f7fe640 10 client.16695 handle_command_reply: tid=0 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 client.16695 shutdown 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 msgr2=0x5557805ec2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805ec2a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff8640f0040 tx=0x7ff8640035f0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 shutdown_connections 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5557805fcbe0 0x5557805ff080 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x5557805e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805e9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 --2- 172.21.15.66:0/3283052950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e34d0 0x5557805ec2a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff84f7fe640 0 client.16695 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 >> 172.21.15.66:0/3283052950 conn(0x5557805ddba0 msgr2=0x5557805df8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 shutdown_connections 2024-04-21T09:11:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 1 -- 172.21.15.66:0/3283052950 wait complete. 2024-04-21T09:11:30.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.619+0000 7ff879010740 20 client.16695 trim_cache size 0 max 16384 2024-04-21T09:11:30.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.632+0000 7ff879010740 1 Processor -- start 2024-04-21T09:11:30.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.632+0000 7ff879010740 1 -- start start 2024-04-21T09:11:30.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff879010740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805e34d0 0x5557805f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff879010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff879010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805eca00 0x5557805ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff879010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701b3e00 con 0x5557805eca00 2024-04-21T09:11:30.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff879010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701b16d0 con 0x5557805e3e80 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff879010740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701af500 con 0x5557805e34d0 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805eca00 0x5557805ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805eca00 0x5557805ec2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54422/0 (socket says 172.21.15.66:54422) 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 -- 172.21.15.66:0/1720753624 learned_addr learned my addr 172.21.15.66:0/1720753624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff86ffff640 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805e34d0 0x5557805f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 -- 172.21.15.66:0/1720753624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805e34d0 msgr2=0x5557805f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805e34d0 0x5557805f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 -- 172.21.15.66:0/1720753624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 msgr2=0x5557805ebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 -- 172.21.15.66:0/1720753624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557805ff700 con 0x5557805eca00 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff8749c3640 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805eca00 0x5557805ec2a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff868013fd0 tx=0x7ff868027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a971287b626fa132 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff855ffb640 1 -- 172.21.15.66:0/1720753624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86802d040 con 0x5557805eca00 2024-04-21T09:11:30.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff855ffb640 1 -- 172.21.15.66:0/1720753624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff868012470 con 0x5557805eca00 2024-04-21T09:11:30.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.633+0000 7ff855ffb640 1 -- 172.21.15.66:0/1720753624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868003970 con 0x5557805eca00 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 -- 172.21.15.66:0/1720753624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805eca00 msgr2=0x5557805ec2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805eca00 0x5557805ec2a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff868013fd0 tx=0x7ff868027000 comp rx=0 tx=0).stop 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 -- 172.21.15.66:0/1720753624 shutdown_connections 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805eca00 0x5557805ec2a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e3e80 0x5557805ebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 --2- 172.21.15.66:0/1720753624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805e34d0 0x5557805f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 -- 172.21.15.66:0/1720753624 >> 172.21.15.66:0/1720753624 conn(0x5557805ddba0 msgr2=0x5557805dcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 -- 172.21.15.66:0/1720753624 shutdown_connections 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 -- 172.21.15.66:0/1720753624 wait complete. 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.634+0000 7ff879010740 1 -- start start 2024-04-21T09:11:30.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 1 Processor -- start 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e34d0 0x5557805f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e3e80 0x5557805f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x555780662200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701afb90 con 0x5557805e3e80 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701b1000 con 0x5557805e34d0 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701b0260 con 0x5557805eca00 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff879010740 10 client.0 _ll_register_callbacks cb 0x7ff8640ba410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e3e80 0x5557805f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e3e80 0x5557805f1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54438/0 (socket says 172.21.15.66:54438) 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 -- 172.21.15.66:0/706211049 learned_addr learned my addr 172.21.15.66:0/706211049 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e34d0 0x5557805f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff8749c3640 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x555780662200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 -- 172.21.15.66:0/706211049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 msgr2=0x555780662200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x555780662200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 -- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e34d0 msgr2=0x5557805f0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e34d0 0x5557805f0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86f7fe640 1 -- 172.21.15.66:0/706211049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557805e30b0 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.635+0000 7ff86ffff640 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e34d0 0x5557805f0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff86f7fe640 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e3e80 0x5557805f1320 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff86000acb0 tx=0x7ff8600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42028785e015c765 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff84f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff84f7fe640 1 -- 172.21.15.66:0/706211049 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86001d880 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff84f7fe640 1 -- 172.21.15.66:0/706211049 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff860004020 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff868012470 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff84f7fe640 1 -- 172.21.15.66:0/706211049 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff86002c050 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 10 client.16701 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557806637d0 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff84f7fe640 1 -- 172.21.15.66:0/706211049 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff868012470 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff84f7fe640 1 -- 172.21.15.66:0/706211049 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff86000f420 con 0x5557805e3e80 2024-04-21T09:11:30.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 10 client.16701 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 20 client.16701 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555780663ef0 0x555780666310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 4 client.16701 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.636+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555780555c10 con 0x555780663ef0 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff86ffff640 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555780663ef0 0x555780666310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff84f7fe640 10 client.16701 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff86ffff640 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555780663ef0 0x555780666310 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0b25c9059131ae server_cookie=611d46000d49e7dc in_seq=0 out_seq=0 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff84f7fe640 1 -- 172.21.15.66:0/706211049 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555780555c10 con 0x555780663ef0 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff84f7fe640 10 client.16701 handle_command_reply: tid=0 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 client.16701 shutdown 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e3e80 msgr2=0x5557805f1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e3e80 0x5557805f1320 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff86000acb0 tx=0x7ff8600118d0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 shutdown_connections 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555780663ef0 0x555780666310 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557805eca00 0x555780662200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557805e3e80 0x5557805f1320 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff84f7fe640 0 client.16701 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 --2- 172.21.15.66:0/706211049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557805e34d0 0x5557805f0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 >> 172.21.15.66:0/706211049 conn(0x5557805ddba0 msgr2=0x5557805def20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 shutdown_connections 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.637+0000 7ff879010740 1 -- 172.21.15.66:0/706211049 wait complete. 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff879010740 20 client.16701 trim_cache size 0 max 16384 2024-04-21T09:11:30.643 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:11:30.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8580a3d50 msgr2=0x7ff8580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8580a3d50 0x7ff8580a6210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff868011010 tx=0x7ff86801bd20 comp rx=0 tx=0).stop 2024-04-21T09:11:30.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 msgr2=0x7ff870100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:30.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff870100e30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff864000c00 tx=0x7ff86402d920 comp rx=0 tx=0).stop 2024-04-21T09:11:30.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 shutdown_connections 2024-04-21T09:11:30.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8580a3d50 0x7ff8580a6210 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87010c520 0x7ff8701018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff870103b00 0x7ff870101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 --2- 172.21.15.66:0/1884541321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8701031e0 0x7ff870100e30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:30.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 >> 172.21.15.66:0/1884541321 conn(0x7ff8700771d0 msgr2=0x7ff8700fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:30.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 shutdown_connections 2024-04-21T09:11:30.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:30.638+0000 7ff876426640 1 -- 172.21.15.66:0/1884541321 wait complete. 2024-04-21T09:11:30.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:30 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/782172780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:11:30.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:30 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/782172780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:11:31.649 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:11:31.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:11:31.649 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:11:31.676 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:31 smithi066 ceph-mon[27953]: pgmap v102: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:31.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 -- 172.21.15.66:0/1069707588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c106f50 msgr2=0x7f4f4c107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1069707588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c106f50 0x7f4f4c107330 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009930 tx=0x7f4f3c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:11:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 -- 172.21.15.66:0/1069707588 shutdown_connections 2024-04-21T09:11:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1069707588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c100e80 0x7f4f4c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1069707588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f4c1004c0 0x7f4f4c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1069707588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c106f50 0x7f4f4c107330 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 -- 172.21.15.66:0/1069707588 >> 172.21.15.66:0/1069707588 conn(0x7f4f4c077450 msgr2=0x7f4f4c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 -- 172.21.15.66:0/1069707588 shutdown_connections 2024-04-21T09:11:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 -- 172.21.15.66:0/1069707588 wait complete. 2024-04-21T09:11:31.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.770+0000 7f4f50b32640 1 Processor -- start 2024-04-21T09:11:31.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f50b32640 1 -- start start 2024-04-21T09:11:31.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f50b32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c1004c0 0x7f4f4c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f50b32640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c100e80 0x7f4f4c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f50b32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f4c106f50 0x7f4f4c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f50b32640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c112260 con 0x7f4f4c1004c0 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f50b32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1120e0 con 0x7f4f4c106f50 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f50b32640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1123e0 con 0x7f4f4c100e80 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c1004c0 0x7f4f4c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c1004c0 0x7f4f4c19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54464/0 (socket says 172.21.15.66:54464) 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1900016140 learned_addr learned my addr 172.21.15.66:0/1900016140 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c100e80 0x7f4f4c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f4c106f50 0x7f4f4c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1900016140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c100e80 msgr2=0x7f4f4c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c100e80 0x7f4f4c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1900016140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f4c106f50 msgr2=0x7f4f4c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f4c106f50 0x7f4f4c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1900016140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c198de0 con 0x7f4f4c1004c0 2024-04-21T09:11:31.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.771+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c1004c0 0x7f4f4c19e460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005940 tx=0x7f4f3c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.772+0000 7f4f3b7fe640 1 -- 172.21.15.66:0/1900016140 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c043020 con 0x7f4f4c1004c0 2024-04-21T09:11:31.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.772+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c199070 con 0x7f4f4c1004c0 2024-04-21T09:11:31.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.772+0000 7f4f3b7fe640 1 -- 172.21.15.66:0/1900016140 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0044c0 con 0x7f4f4c1004c0 2024-04-21T09:11:31.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.772+0000 7f4f3b7fe640 1 -- 172.21.15.66:0/1900016140 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c003bf0 con 0x7f4f4c1004c0 2024-04-21T09:11:31.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.772+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c1a5320 con 0x7f4f4c1004c0 2024-04-21T09:11:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f3b7fe640 1 -- 172.21.15.66:0/1900016140 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4f3c046020 con 0x7f4f4c1004c0 2024-04-21T09:11:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 Processor -- start 2024-04-21T09:11:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 -- start start 2024-04-21T09:11:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a25840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a28260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557f96980f00 con 0x557f96a1d290 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f9691e590 con 0x557f96a1c8e0 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f53803740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557f9697cec0 con 0x557f96a25e10 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f4a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a25840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f4a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a25840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54478/0 (socket says 172.21.15.66:54478) 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.775+0000 7f4f4a59c640 1 -- 172.21.15.66:0/4274214461 learned_addr learned my addr 172.21.15.66:0/4274214461 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f3b7fe640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f340a3d20 0x7f4f340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a28260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f340a3d20 0x7f4f340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f3b7fe640 1 -- 172.21.15.66:0/1900016140 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4f3c0f2580 con 0x7f4f4c1004c0 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4a59c640 1 -- 172.21.15.66:0/4274214461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 msgr2=0x557f96a28260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a28260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4a59c640 1 -- 172.21.15.66:0/4274214461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 msgr2=0x557f96a1ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a1ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4a59c640 1 -- 172.21.15.66:0/4274214461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f9675b7e0 con 0x557f96a1d290 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a25840 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0ef9b0 tx=0x7f4f3c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8db23fa12eb5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f38ff9640 1 -- 172.21.15.66:0/4274214461 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0bde00 con 0x557f96a1d290 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f340a3d20 0x7f4f340a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f44005fd0 tx=0x7f4f44009600 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f38ff9640 1 -- 172.21.15.66:0/4274214461 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0be9c0 con 0x557f96a1d290 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.776+0000 7f4f38ff9640 1 -- 172.21.15.66:0/4274214461 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0bdc20 con 0x557f96a1d290 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 -- 172.21.15.66:0/4274214461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 msgr2=0x557f96a25840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a25840 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0ef9b0 tx=0x7f4f3c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 -- 172.21.15.66:0/4274214461 shutdown_connections 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a28260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a25840 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 --2- 172.21.15.66:0/4274214461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a1ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 -- 172.21.15.66:0/4274214461 >> 172.21.15.66:0/4274214461 conn(0x557f96a16fb0 msgr2=0x557f96a17390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 -- 172.21.15.66:0/4274214461 shutdown_connections 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 -- 172.21.15.66:0/4274214461 wait complete. 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 -- start start 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.777+0000 7f4f53803740 1 Processor -- start 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f53803740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1c8e0 0x557f96a22d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f53803740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f53803740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a23790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f53803740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557f96980f00 con 0x557f96a1c8e0 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f53803740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f9693a580 con 0x557f96a1d290 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f53803740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557f96a28b40 con 0x557f96a25e10 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1c8e0 0x557f96a22d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f53803740 10 client.0 _ll_register_callbacks cb 0x557f96940650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1c8e0 0x557f96a22d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54480/0 (socket says 172.21.15.66:54480) 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1289554159 learned_addr learned my addr 172.21.15.66:0/1289554159 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a23790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1289554159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 msgr2=0x557f96a23790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a23790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 msgr2=0x557f96a23250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a23250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 -- 172.21.15.66:0/1289554159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f967bbf70 con 0x557f96a1c8e0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1c8e0 0x557f96a22d10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0ba720 tx=0x7f4f3c0bdc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3054a0f9470af83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.778+0000 7f4f31ffb640 1 -- 172.21.15.66:0/1289554159 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0be2c0 con 0x557f96a1c8e0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f31ffb640 1 -- 172.21.15.66:0/1289554159 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4f3c0efbe0 con 0x557f96a1c8e0 2024-04-21T09:11:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3c0be9c0 con 0x557f96a1c8e0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f31ffb640 1 -- 172.21.15.66:0/1289554159 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f3c0efde0 con 0x557f96a1c8e0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 10 client.16713 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f96653170 con 0x557f96a1c8e0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f31ffb640 1 -- 172.21.15.66:0/1289554159 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0be9c0 con 0x557f96a1c8e0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f31ffb640 1 -- 172.21.15.66:0/1289554159 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f3c0c6b30 con 0x557f96a1c8e0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 10 client.16713 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 20 client.16713 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a35cb0 0x557f96a380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 4 client.16713 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f9698e920 con 0x557f96a35cb0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.779+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a35cb0 0x557f96a380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.780+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a35cb0 0x557f96a380a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ba5d3dc7991691 server_cookie=204d59e8eb61f0a in_seq=0 out_seq=0 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.780+0000 7f4f31ffb640 10 client.16713 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.780+0000 7f4f31ffb640 1 -- 172.21.15.66:0/1289554159 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f9698e920 con 0x557f96a35cb0 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.780+0000 7f4f31ffb640 10 client.16713 handle_command_reply: tid=0 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 client.16713 shutdown 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1c8e0 msgr2=0x557f96a22d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1c8e0 0x557f96a22d10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0ba720 tx=0x7f4f3c0bdc20 comp rx=0 tx=0).stop 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 shutdown_connections 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a35cb0 0x557f96a380a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a23790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a23250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f31ffb640 0 client.16713 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 --2- 172.21.15.66:0/1289554159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1c8e0 0x557f96a22d10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 >> 172.21.15.66:0/1289554159 conn(0x557f96a16fb0 msgr2=0x557f96a18d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 shutdown_connections 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 1 -- 172.21.15.66:0/1289554159 wait complete. 2024-04-21T09:11:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.781+0000 7f4f53803740 20 client.16713 trim_cache size 0 max 16384 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 Processor -- start 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 -- start start 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a1c8e0 0x557f96a25440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a25e10 0x557f96a1e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1af9a0 con 0x557f96a25e10 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1ad270 con 0x557f96a1d290 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f53803740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1ab0a0 con 0x557f96a1c8e0 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a25e10 0x557f96a1e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a25e10 0x557f96a1e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54488/0 (socket says 172.21.15.66:54488) 2024-04-21T09:11:31.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 -- 172.21.15.66:0/618323815 learned_addr learned my addr 172.21.15.66:0/618323815 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a1c8e0 0x557f96a25440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 -- 172.21.15.66:0/618323815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a1c8e0 msgr2=0x557f96a25440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a1c8e0 0x557f96a25440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 -- 172.21.15.66:0/618323815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 msgr2=0x557f96a317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 -- 172.21.15.66:0/618323815 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f96a21260 con 0x557f96a25e10 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a1c8e0 0x557f96a25440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.799+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a25e10 0x557f96a1e460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4f44010840 tx=0x7f4f44009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188b7ec37a3109d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f33fff640 1 -- 172.21.15.66:0/618323815 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44018070 con 0x557f96a25e10 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f33fff640 1 -- 172.21.15.66:0/618323815 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4401a040 con 0x557f96a25e10 2024-04-21T09:11:31.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f33fff640 1 -- 172.21.15.66:0/618323815 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4402adf0 con 0x557f96a25e10 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 -- 172.21.15.66:0/618323815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a25e10 msgr2=0x557f96a1e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a25e10 0x557f96a1e460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4f44010840 tx=0x7f4f44009a00 comp rx=0 tx=0).stop 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 -- 172.21.15.66:0/618323815 shutdown_connections 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a25e10 0x557f96a1e460 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1d290 0x557f96a317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 --2- 172.21.15.66:0/618323815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a1c8e0 0x557f96a25440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 -- 172.21.15.66:0/618323815 >> 172.21.15.66:0/618323815 conn(0x557f96a16fb0 msgr2=0x557f96a183e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 -- 172.21.15.66:0/618323815 shutdown_connections 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 -- 172.21.15.66:0/618323815 wait complete. 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.800+0000 7f4f53803740 1 -- start start 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 1 Processor -- start 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a29720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a29c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a2a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1ab730 con 0x557f96a1d290 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1acba0 con 0x557f96a1c8e0 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1abe00 con 0x557f96a25e10 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f53803740 10 client.0 _ll_register_callbacks cb 0x7f4f3c0f5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a29c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a29c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54496/0 (socket says 172.21.15.66:54496) 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 -- 172.21.15.66:0/4171973351 learned_addr learned my addr 172.21.15.66:0/4171973351 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f4ad9d640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a2a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a29720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 -- 172.21.15.66:0/4171973351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 msgr2=0x557f96a2a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a2a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 -- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 msgr2=0x557f96a29720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a29720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 -- 172.21.15.66:0/4171973351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f96a9aee0 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a29720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f49d9b640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a29c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4f40000c00 tx=0x7f4f40005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b9f7a79b7d9c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.801+0000 7f4f31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f31ffb640 1 -- 172.21.15.66:0/4171973351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40004c70 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f31ffb640 1 -- 172.21.15.66:0/4171973351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f4f4000b770 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4401a040 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f31ffb640 1 -- 172.21.15.66:0/4171973351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f40031050 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 10 client.16719 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f96a9c1c0 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f31ffb640 1 -- 172.21.15.66:0/4171973351 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4401a040 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f31ffb640 1 -- 172.21.15.66:0/4171973351 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f40021500 con 0x557f96a1d290 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 10 client.16719 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:31.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:31.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 20 client.16719 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:31.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a9c970 0x557f96a9ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 4 client.16719 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:11:31.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557f96988790 con 0x557f96a9c970 2024-04-21T09:11:31.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.802+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a9c970 0x557f96a9ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.803+0000 7f4f31ffb640 10 client.16719 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:31.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.803+0000 7f4f4a59c640 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a9c970 0x557f96a9ed90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5544e1d7e57c2e80 server_cookie=bcf6da10f0f614e in_seq=0 out_seq=0 2024-04-21T09:11:31.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f31ffb640 1 -- 172.21.15.66:0/4171973351 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557f96988790 con 0x557f96a9c970 2024-04-21T09:11:31.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f31ffb640 10 client.16719 handle_command_reply: tid=0 2024-04-21T09:11:31.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 client.16719 shutdown 2024-04-21T09:11:31.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 msgr2=0x557f96a29c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a29c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4f40000c00 tx=0x7f4f40005040 comp rx=0 tx=0).stop 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 shutdown_connections 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557f96a9c970 0x557f96a9ed90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f96a25e10 0x557f96a2a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f31ffb640 0 client.16719 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f96a1d290 0x557f96a29c60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 --2- 172.21.15.66:0/4171973351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f96a1c8e0 0x557f96a29720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 >> 172.21.15.66:0/4171973351 conn(0x557f96a16fb0 msgr2=0x557f96a18c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 shutdown_connections 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 1 -- 172.21.15.66:0/4171973351 wait complete. 2024-04-21T09:11:31.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.804+0000 7f4f53803740 20 client.16719 trim_cache size 0 max 16384 2024-04-21T09:11:31.811 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:11:31.811 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:11:31.811 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "93550ce6-837d-4199-ba32-c1e47a2eb0c2", 2024-04-21T09:11:31.811 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:11:31.811 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f340a3d20 msgr2=0x7f4f340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f340a3d20 0x7f4f340a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f44005fd0 tx=0x7f4f44009600 comp rx=0 tx=0).stop 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c1004c0 msgr2=0x7f4f4c19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c1004c0 0x7f4f4c19e460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005940 tx=0x7f4f3c0040c0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 shutdown_connections 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4f340a3d20 0x7f4f340a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f4c106f50 0x7f4f4c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c100e80 0x7f4f4c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 --2- 172.21.15.66:0/1900016140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f4c1004c0 0x7f4f4c19e460 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 >> 172.21.15.66:0/1900016140 conn(0x7f4f4c077450 msgr2=0x7f4f4c105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 shutdown_connections 2024-04-21T09:11:31.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.805+0000 7f4f50b32640 1 -- 172.21.15.66:0/1900016140 wait complete. 2024-04-21T09:11:31.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:11:31.816 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:11:31.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:31 smithi186 ceph-mon[31505]: pgmap v102: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:31.861 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:31 smithi083 ceph-mon[31413]: pgmap v102: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:31.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 -- 172.21.15.66:0/365896878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c1010a0 msgr2=0x7fa90c101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 --2- 172.21.15.66:0/365896878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c1010a0 0x7fa90c101500 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa900009900 tx=0x7fa90002bc30 comp rx=0 tx=0).stop 2024-04-21T09:11:31.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 -- 172.21.15.66:0/365896878 shutdown_connections 2024-04-21T09:11:31.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 --2- 172.21.15.66:0/365896878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa90c101a40 0x7fa90c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 --2- 172.21.15.66:0/365896878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c1010a0 0x7fa90c101500 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 --2- 172.21.15.66:0/365896878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90c105dd0 0x7fa90c1061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 -- 172.21.15.66:0/365896878 >> 172.21.15.66:0/365896878 conn(0x7fa90c077c40 msgr2=0x7fa90c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 -- 172.21.15.66:0/365896878 shutdown_connections 2024-04-21T09:11:31.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.961+0000 7fa9115c7640 1 -- 172.21.15.66:0/365896878 wait complete. 2024-04-21T09:11:31.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 Processor -- start 2024-04-21T09:11:31.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 -- start start 2024-04-21T09:11:31.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90c1010a0 0x7fa90c0feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa90c101a40 0x7fa90c0ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c105dd0 0x7fa90c0ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c111170 con 0x7fa90c105dd0 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c110ff0 con 0x7fa90c101a40 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.962+0000 7fa9115c7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1112f0 con 0x7fa90c1010a0 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c105dd0 0x7fa90c0ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c105dd0 0x7fa90c0ff620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54504/0 (socket says 172.21.15.66:54504) 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 -- 172.21.15.66:0/1727141746 learned_addr learned my addr 172.21.15.66:0/1727141746 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90affd640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90c1010a0 0x7fa90c0feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa90c101a40 0x7fa90c0ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 -- 172.21.15.66:0/1727141746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90c1010a0 msgr2=0x7fa90c0feba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90c1010a0 0x7fa90c0feba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 -- 172.21.15.66:0/1727141746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa90c101a40 msgr2=0x7fa90c0ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa90c101a40 0x7fa90c0ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 -- 172.21.15.66:0/1727141746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c1a6c80 con 0x7fa90c105dd0 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90affd640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90c1010a0 0x7fa90c0feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:11:31.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa90b7fe640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c105dd0 0x7fa90c0ff620 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa90400eaa0 tx=0x7fa90400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa8f3fff640 1 -- 172.21.15.66:0/1727141746 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90400cdb0 con 0x7fa90c105dd0 2024-04-21T09:11:31.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa8f3fff640 1 -- 172.21.15.66:0/1727141746 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa904005020 con 0x7fa90c105dd0 2024-04-21T09:11:31.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c1a6f70 con 0x7fa90c105dd0 2024-04-21T09:11:31.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa8f3fff640 1 -- 172.21.15.66:0/1727141746 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9040116d0 con 0x7fa90c105dd0 2024-04-21T09:11:31.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.963+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c1a7440 con 0x7fa90c105dd0 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.965+0000 7fa8f3fff640 1 -- 172.21.15.66:0/1727141746 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa9040051d0 con 0x7fa90c105dd0 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 Processor -- start 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 -- start start 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867baa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 0x555867bb5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555867b0f1e0 con 0x555867baa580 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555867aac390 con 0x555867baaea0 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa9141b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555867b0b1a0 con 0x555867bb3a80 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa8f3fff640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa8f40a3d20 0x7fa8f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867baa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867baa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54510/0 (socket says 172.21.15.66:54510) 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 -- 172.21.15.66:0/1551584988 learned_addr learned my addr 172.21.15.66:0/1551584988 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90b7fe640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa8f40a3d20 0x7fa8f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa8f3fff640 1 -- 172.21.15.66:0/1727141746 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fa9040d0e40 con 0x7fa90c105dd0 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90affd640 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 -- 172.21.15.66:0/1551584988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 msgr2=0x555867bb5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 0x555867bb5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 -- 172.21.15.66:0/1551584988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 msgr2=0x555867bb3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.966+0000 7fa90a7fc640 1 -- 172.21.15.66:0/1551584988 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558678eb7d0 con 0x555867baa580 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.967+0000 7fa90b7fe640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa8f40a3d20 0x7fa8f40a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa904098ed0 tx=0x7fa904005890 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.967+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867baa960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa90002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce036449494ffd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.967+0000 7fa8f17fa640 1 -- 172.21.15.66:0/1551584988 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900043070 con 0x555867baa580 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.967+0000 7fa8f17fa640 1 -- 172.21.15.66:0/1551584988 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90002cc90 con 0x555867baa580 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.967+0000 7fa8f17fa640 1 -- 172.21.15.66:0/1551584988 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000046c0 con 0x555867baa580 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.967+0000 7fa9141b9740 1 -- 172.21.15.66:0/1551584988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 msgr2=0x555867baa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.967+0000 7fa9141b9740 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867baa960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa90002c5d0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- 172.21.15.66:0/1551584988 shutdown_connections 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 0x555867bb5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 --2- 172.21.15.66:0/1551584988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867baa960 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- 172.21.15.66:0/1551584988 >> 172.21.15.66:0/1551584988 conn(0x555867ba4cb0 msgr2=0x555867ba5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- 172.21.15.66:0/1551584988 shutdown_connections 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- 172.21.15.66:0/1551584988 wait complete. 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- start start 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 Processor -- start 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867bb0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 0x555867bb1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555867bb6810 con 0x555867baa580 2024-04-21T09:11:31.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555867ac82f0 con 0x555867baaea0 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555867bb6990 con 0x555867bb3a80 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa9141b9740 10 client.0 _ll_register_callbacks cb 0x555867ad5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa90a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867bb0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa90a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867bb0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54522/0 (socket says 172.21.15.66:54522) 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.968+0000 7fa90a7fc640 1 -- 172.21.15.66:0/557285161 learned_addr learned my addr 172.21.15.66:0/557285161 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90affd640 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90a7fc640 1 -- 172.21.15.66:0/557285161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 msgr2=0x555867bb1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 0x555867bb1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90a7fc640 1 -- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 msgr2=0x555867bb0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90a7fc640 1 -- 172.21.15.66:0/557285161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558678213f0 con 0x555867baa580 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90affd640 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867bb0aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa900036040 tx=0x7fa900009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3303fc8c49639dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa8eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa8eeff5640 1 -- 172.21.15.66:0/557285161 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900043040 con 0x555867baa580 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90002cc90 con 0x555867baa580 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa8eeff5640 1 -- 172.21.15.66:0/557285161 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fa9000037f0 con 0x555867baa580 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa8eeff5640 1 -- 172.21.15.66:0/557285161 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9000039f0 con 0x555867baa580 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa9141b9740 10 client.16731 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5558677e10f0 con 0x555867baa580 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.969+0000 7fa8eeff5640 1 -- 172.21.15.66:0/557285161 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90002cc90 con 0x555867baa580 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa8eeff5640 1 -- 172.21.15.66:0/557285161 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa900053c00 con 0x555867baa580 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa9141b9740 10 client.16731 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa9141b9740 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa9141b9740 20 client.16731 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa9141b9740 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867bc38b0 0x555867bc5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa9141b9740 4 client.16731 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555867b1cc00 con 0x555867bc38b0 2024-04-21T09:11:31.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa90affd640 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867bc38b0 0x555867bc5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa8eeff5640 10 client.16731 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.970+0000 7fa90affd640 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867bc38b0 0x555867bc5ca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543e53263e9219a4 server_cookie=97db749b321daec3 in_seq=0 out_seq=0 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa8eeff5640 1 -- 172.21.15.66:0/557285161 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555867b1cc00 con 0x555867bc38b0 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa8eeff5640 10 client.16731 handle_command_reply: tid=0 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 client.16731 shutdown 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 msgr2=0x555867bb0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867bb0aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa900036040 tx=0x7fa900009300 comp rx=0 tx=0).stop 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 shutdown_connections 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867bc38b0 0x555867bc5ca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867bb3a80 0x555867bb1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa8eeff5640 0 client.16731 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baaea0 0x555867bb0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 --2- 172.21.15.66:0/557285161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baa580 0x555867bb0aa0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 >> 172.21.15.66:0/557285161 conn(0x555867ba4cb0 msgr2=0x555867ba6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 shutdown_connections 2024-04-21T09:11:31.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 1 -- 172.21.15.66:0/557285161 wait complete. 2024-04-21T09:11:31.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.971+0000 7fa9141b9740 20 client.16731 trim_cache size 0 max 16384 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 Processor -- start 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 -- start start 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baa580 0x555867bb31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baaea0 0x555867bafc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867bb3a80 0x555867ba3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b3d20 con 0x555867baaea0 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b15f0 con 0x555867bb3a80 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa9141b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1af420 con 0x555867baa580 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baaea0 0x555867bafc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baaea0 0x555867bafc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54538/0 (socket says 172.21.15.66:54538) 2024-04-21T09:11:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 -- 172.21.15.66:0/2262248301 learned_addr learned my addr 172.21.15.66:0/2262248301 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baa580 0x555867bb31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867bb3a80 0x555867ba3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 -- 172.21.15.66:0/2262248301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baa580 msgr2=0x555867bb31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baa580 0x555867bb31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 -- 172.21.15.66:0/2262248301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867bb3a80 msgr2=0x555867ba3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867bb3a80 0x555867ba3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.989+0000 7fa90affd640 1 -- 172.21.15.66:0/2262248301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555867bb5e70 con 0x555867baaea0 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867bb3a80 0x555867ba3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa90affd640 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baaea0 0x555867bafc00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x555867bb0300 tx=0x7fa8fc00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e699b81f55041c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa8f0ff9640 1 -- 172.21.15.66:0/2262248301 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc01e070 con 0x555867baaea0 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa8f0ff9640 1 -- 172.21.15.66:0/2262248301 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8fc01adc0 con 0x555867baaea0 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa8f0ff9640 1 -- 172.21.15.66:0/2262248301 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0119b0 con 0x555867baaea0 2024-04-21T09:11:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 -- 172.21.15.66:0/2262248301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baaea0 msgr2=0x555867bafc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baaea0 0x555867bafc00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x555867bb0300 tx=0x7fa8fc00b6d0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 -- 172.21.15.66:0/2262248301 shutdown_connections 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867bb3a80 0x555867ba3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867baaea0 0x555867bafc00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 --2- 172.21.15.66:0/2262248301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baa580 0x555867bb31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 -- 172.21.15.66:0/2262248301 >> 172.21.15.66:0/2262248301 conn(0x555867ba4cb0 msgr2=0x555867ba34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.990+0000 7fa9141b9740 1 -- 172.21.15.66:0/2262248301 shutdown_connections 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 -- 172.21.15.66:0/2262248301 wait complete. 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 -- start start 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 Processor -- start 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baa580 0x555867bc91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baaea0 0x555867bc6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867bb3a80 0x555867bc6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1afab0 con 0x555867bb3a80 2024-04-21T09:11:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b0f20 con 0x555867baa580 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b0180 con 0x555867baaea0 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa9141b9740 10 client.0 _ll_register_callbacks cb 0x7fa900005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867bb3a80 0x555867bc6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867bb3a80 0x555867bc6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54552/0 (socket says 172.21.15.66:54552) 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 -- 172.21.15.66:0/729822796 learned_addr learned my addr 172.21.15.66:0/729822796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baa580 0x555867bc91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90affd640 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baaea0 0x555867bc6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 -- 172.21.15.66:0/729822796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baaea0 msgr2=0x555867bc6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baaea0 0x555867bc6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 -- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baa580 msgr2=0x555867bc91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baa580 0x555867bc91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.991+0000 7fa90a7fc640 1 -- 172.21.15.66:0/729822796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555867bc7390 con 0x555867bb3a80 2024-04-21T09:11:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa90a7fc640 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867bb3a80 0x555867bc6b60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa900009850 tx=0x7fa900005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30aa682bc49155f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa8eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa8eeff5640 1 -- 172.21.15.66:0/729822796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900043040 con 0x555867bb3a80 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa8eeff5640 1 -- 172.21.15.66:0/729822796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fa900048c40 con 0x555867bb3a80 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc01adc0 con 0x555867bb3a80 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa8eeff5640 1 -- 172.21.15.66:0/729822796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa900068050 con 0x555867bb3a80 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 10 client.16737 fetch_fsmap learned FSMap version 25 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555867c29b40 con 0x555867bb3a80 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa8eeff5640 1 -- 172.21.15.66:0/729822796 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8fc01adc0 con 0x555867bb3a80 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa8eeff5640 1 -- 172.21.15.66:0/729822796 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9000534e0 con 0x555867bb3a80 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 10 client.16737 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 20 client.16737 populate_metadata read hostname 'smithi066' 2024-04-21T09:11:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867c2a230 0x555867c2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 4 client.16737 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.992+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555867b16a70 con 0x555867c2a230 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa90affd640 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867c2a230 0x555867c2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa8eeff5640 10 client.16737 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa90affd640 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867c2a230 0x555867c2c650 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f627dd71914769 server_cookie=dbc82f0977f2e72d in_seq=0 out_seq=0 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa8eeff5640 1 -- 172.21.15.66:0/729822796 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555867b16a70 con 0x555867c2a230 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa8eeff5640 10 client.16737 handle_command_reply: tid=0 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 client.16737 shutdown 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867bb3a80 msgr2=0x555867bc6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867bb3a80 0x555867bc6b60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa900009850 tx=0x7fa900005b50 comp rx=0 tx=0).stop 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 shutdown_connections 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555867c2a230 0x555867c2c650 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555867bb3a80 0x555867bc6b60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555867baaea0 0x555867bc6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa8eeff5640 0 client.16737 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 --2- 172.21.15.66:0/729822796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555867baa580 0x555867bc91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 >> 172.21.15.66:0/729822796 conn(0x555867ba4cb0 msgr2=0x555867bafa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 shutdown_connections 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.993+0000 7fa9141b9740 1 -- 172.21.15.66:0/729822796 wait complete. 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9141b9740 20 client.16737 trim_cache size 0 max 16384 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:11:32.000 INFO:teuthology.orchestra.run.smithi066.stdout: "93550ce6-837d-4199-ba32-c1e47a2eb0c2": { 2024-04-21T09:11:32.001 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:11:32.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "93550ce6-837d-4199-ba32-c1e47a2eb0c2", 2024-04-21T09:11:32.001 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:11:32.001 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:11:32.001 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:11:32.001 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa8f40a3d20 msgr2=0x7fa8f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa8f40a3d20 0x7fa8f40a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa904098ed0 tx=0x7fa904005890 comp rx=0 tx=0).stop 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c105dd0 msgr2=0x7fa90c0ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c105dd0 0x7fa90c0ff620 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa90400eaa0 tx=0x7fa90400ef60 comp rx=0 tx=0).stop 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 shutdown_connections 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa8f40a3d20 0x7fa8f40a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa90c105dd0 0x7fa90c0ff620 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa90c101a40 0x7fa90c0ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 --2- 172.21.15.66:0/1727141746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90c1010a0 0x7fa90c0feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 >> 172.21.15.66:0/1727141746 conn(0x7fa90c077c40 msgr2=0x7fa90c06a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:11:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 shutdown_connections 2024-04-21T09:11:32.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:11:31.994+0000 7fa9115c7640 1 -- 172.21.15.66:0/1727141746 wait complete. 2024-04-21T09:11:32.004 INFO:tasks.cephfs.filesystem:scrub status for tag:93550ce6-837d-4199-ba32-c1e47a2eb0c2 - {'path': '/', 'tag': '93550ce6-837d-4199-ba32-c1e47a2eb0c2', 'options': 'recursive,force'} 2024-04-21T09:11:32.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:32 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:11:32.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:32 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:32.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:32 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:11:32.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:32 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:32.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:32 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:11:32.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:32 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:32.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:32 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:11:32.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:32 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:32.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:32 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:11:32.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:32 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:32.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:32 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:11:32.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:32 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:11:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:33 smithi186 ceph-mon[31505]: pgmap v103: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:33.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:33 smithi083 ceph-mon[31413]: pgmap v103: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:33.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:33 smithi066 ceph-mon[27953]: pgmap v103: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:35 smithi186 ceph-mon[31505]: pgmap v104: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:35 smithi083 ceph-mon[31413]: pgmap v104: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:35.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:35 smithi066 ceph-mon[27953]: pgmap v104: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:36.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:36 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:11:36.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:36 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:11:36.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:36 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:11:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:37 smithi186 ceph-mon[31505]: pgmap v105: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:37.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:37 smithi083 ceph-mon[31413]: pgmap v105: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:37.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:37 smithi066 ceph-mon[27953]: pgmap v105: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:39.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:39 smithi186 ceph-mon[31505]: pgmap v106: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:39.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:39 smithi083 ceph-mon[31413]: pgmap v106: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:39.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:39 smithi066 ceph-mon[27953]: pgmap v106: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:41 smithi186 ceph-mon[31505]: pgmap v107: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:41.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:41 smithi083 ceph-mon[31413]: pgmap v107: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:41.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:41 smithi066 ceph-mon[27953]: pgmap v107: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:43.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:43 smithi186 ceph-mon[31505]: pgmap v108: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:43.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:43 smithi083 ceph-mon[31413]: pgmap v108: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:43.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:43 smithi066 ceph-mon[27953]: pgmap v108: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:11:45.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:45 smithi186 ceph-mon[31505]: pgmap v109: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:45.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:45 smithi083 ceph-mon[31413]: pgmap v109: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:45.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:45 smithi066 ceph-mon[27953]: pgmap v109: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:11:47.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:47 smithi186 ceph-mon[31505]: pgmap v110: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:47.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:47 smithi083 ceph-mon[31413]: pgmap v110: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:47.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:47 smithi066 ceph-mon[27953]: pgmap v110: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:49.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:49 smithi186 ceph-mon[31505]: pgmap v111: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:49.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:49 smithi083 ceph-mon[31413]: pgmap v111: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:49.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:49 smithi066 ceph-mon[27953]: pgmap v111: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:51 smithi186 ceph-mon[31505]: pgmap v112: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:51.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:51 smithi083 ceph-mon[31413]: pgmap v112: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:51.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:51 smithi066 ceph-mon[27953]: pgmap v112: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:53.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:53 smithi186 ceph-mon[31505]: pgmap v113: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:53.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:53 smithi083 ceph-mon[31413]: pgmap v113: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:53.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:53 smithi066 ceph-mon[27953]: pgmap v113: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:55.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:55 smithi186 ceph-mon[31505]: pgmap v114: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:55.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:55 smithi083 ceph-mon[31413]: pgmap v114: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:55.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:55 smithi066 ceph-mon[27953]: pgmap v114: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:57.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:57 smithi186 ceph-mon[31505]: pgmap v115: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:57.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:57 smithi083 ceph-mon[31413]: pgmap v115: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:57.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:57 smithi066 ceph-mon[27953]: pgmap v115: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:11:59.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:11:59 smithi186 ceph-mon[31505]: pgmap v116: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:59.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:11:59 smithi083 ceph-mon[31413]: pgmap v116: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:11:59.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:11:59 smithi066 ceph-mon[27953]: pgmap v116: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:01 smithi186 ceph-mon[31505]: pgmap v117: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:01.862 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:01 smithi083 ceph-mon[31413]: pgmap v117: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:01.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:01 smithi066 ceph-mon[27953]: pgmap v117: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:02.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:12:02.006 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:12:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 -- 172.21.15.66:0/3826546662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8101660 msgr2=0x7fc7b810e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/3826546662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8101660 0x7fc7b810e3b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009900 tx=0x7fc7a802bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 -- 172.21.15.66:0/3826546662 shutdown_connections 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/3826546662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7b810e8f0 0x7fc7b8110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/3826546662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8101660 0x7fc7b810e3b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/3826546662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b8100d40 0x7fc7b8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 -- 172.21.15.66:0/3826546662 >> 172.21.15.66:0/3826546662 conn(0x7fc7b8077f70 msgr2=0x7fc7b8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.123+0000 7fc7bfe81640 1 -- 172.21.15.66:0/3826546662 shutdown_connections 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 -- 172.21.15.66:0/3826546662 wait complete. 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 Processor -- start 2024-04-21T09:12:02.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 -- start start 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8100d40 0x7fc7b81a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b8101660 0x7fc7b81a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7b810e8f0 0x7fc7b819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81143f0 con 0x7fc7b8100d40 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8114270 con 0x7fc7b8101660 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bfe81640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8114570 con 0x7fc7b810e8f0 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bdc1d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8100d40 0x7fc7b81a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bdc1d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8100d40 0x7fc7b81a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52690/0 (socket says 172.21.15.66:52690) 2024-04-21T09:12:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.124+0000 7fc7bdc1d640 1 -- 172.21.15.66:0/312677178 learned_addr learned my addr 172.21.15.66:0/312677178 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7be41e640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7b810e8f0 0x7fc7b819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b8101660 0x7fc7b81a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bdc1d640 1 -- 172.21.15.66:0/312677178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7b810e8f0 msgr2=0x7fc7b819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bdc1d640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7b810e8f0 0x7fc7b819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bdc1d640 1 -- 172.21.15.66:0/312677178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b8101660 msgr2=0x7fc7b81a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bdc1d640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b8101660 0x7fc7b81a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bdc1d640 1 -- 172.21.15.66:0/312677178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b819b010 con 0x7fc7b8100d40 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bdc1d640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8100d40 0x7fc7b81a06d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400d950 tx=0x7fc7b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7aeffd640 1 -- 172.21.15.66:0/312677178 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4017070 con 0x7fc7b8100d40 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7aeffd640 1 -- 172.21.15.66:0/312677178 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b4009c30 con 0x7fc7b8100d40 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b819b300 con 0x7fc7b8100d40 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7aeffd640 1 -- 172.21.15.66:0/312677178 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400f620 con 0x7fc7b8100d40 2024-04-21T09:12:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.125+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a7520 con 0x7fc7b8100d40 2024-04-21T09:12:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.127+0000 7fc7aeffd640 1 -- 172.21.15.66:0/312677178 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc7b4010460 con 0x7fc7b8100d40 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 Processor -- start 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 -- start start 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffee580 0x560bdffee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7aeffd640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7a40a3d20 0x7fc7a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfff7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560bdfff9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560bdff531e0 con 0x560bdffeeea0 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560bdfef0390 con 0x560bdffee580 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560bdff4f1a0 con 0x560bdfff7a80 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7bd41c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfff7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.128+0000 7fc7bdc1d640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7a40a3d20 0x7fc7a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfff7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52704/0 (socket says 172.21.15.66:52704) 2024-04-21T09:12:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1144580892 learned_addr learned my addr 172.21.15.66:0/1144580892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7be41e640 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffee580 0x560bdffee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7aeffd640 1 -- 172.21.15.66:0/312677178 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc7b40d07b0 con 0x7fc7b8100d40 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1144580892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 msgr2=0x560bdfff9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560bdfff9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1144580892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffee580 msgr2=0x560bdffee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffee580 0x560bdffee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1144580892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdfd2f7d0 con 0x560bdffeeea0 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bdc1d640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7a40a3d20 0x7fc7a40a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000c00 tx=0x7fc7b4005f70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfff7540 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc7a802c170 tx=0x7fc7a8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38595a8e832bd5ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7a3fff640 1 -- 172.21.15.66:0/1144580892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8044020 con 0x560bdffeeea0 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7a3fff640 1 -- 172.21.15.66:0/1144580892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a802cc90 con 0x560bdffeeea0 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.129+0000 7fc7a3fff640 1 -- 172.21.15.66:0/1144580892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80046c0 con 0x560bdffeeea0 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1144580892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 msgr2=0x560bdfff7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfff7540 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc7a802c170 tx=0x7fc7a8005e70 comp rx=0 tx=0).stop 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1144580892 shutdown_connections 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560bdfff9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfff7540 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1144580892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffee580 0x560bdffee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1144580892 >> 172.21.15.66:0/1144580892 conn(0x560bdffe8cb0 msgr2=0x560bdffe9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1144580892 shutdown_connections 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1144580892 wait complete. 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.130+0000 7fc7c2a70740 1 -- start start 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 1 Processor -- start 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560bdfff4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560bdfff5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560bdfffa810 con 0x560bdffee580 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560bdff0c2f0 con 0x560bdffeeea0 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560bdfffa990 con 0x560bdfff7a80 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7c2a70740 10 client.0 _ll_register_callbacks cb 0x560bdff19580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7be41e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7be41e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52718/0 (socket says 172.21.15.66:52718) 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7be41e640 1 -- 172.21.15.66:0/4031880431 learned_addr learned my addr 172.21.15.66:0/4031880431 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560bdfff4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7be41e640 1 -- 172.21.15.66:0/4031880431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 msgr2=0x560bdfff5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7be41e640 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560bdfff5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7be41e640 1 -- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 msgr2=0x560bdfff4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.131+0000 7fc7be41e640 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560bdfff4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7be41e640 1 -- 172.21.15.66:0/4031880431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdfc653f0 con 0x560bdffee580 2024-04-21T09:12:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7be41e640 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff4ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00117b0 tx=0x7fc7b0011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bf4f89ad041b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7a17fa640 1 -- 172.21.15.66:0/4031880431 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0010da0 con 0x560bdffee580 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7a17fa640 1 -- 172.21.15.66:0/4031880431 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc7b0003970 con 0x560bdffee580 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a802cc90 con 0x560bdffee580 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7a17fa640 1 -- 172.21.15.66:0/4031880431 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7b0025050 con 0x560bdffee580 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7c2a70740 10 client.16749 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bdfc250f0 con 0x560bdffee580 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7a17fa640 1 -- 172.21.15.66:0/4031880431 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a802cc90 con 0x560bdffee580 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7a17fa640 1 -- 172.21.15.66:0/4031880431 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7b001c410 con 0x560bdffee580 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7c2a70740 10 client.16749 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7c2a70740 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.132+0000 7fc7c2a70740 20 client.16749 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.133+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be0007aa0 0x560be0009e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.133+0000 7fc7c2a70740 4 client.16749 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.133+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bdff60c00 con 0x560be0007aa0 2024-04-21T09:12:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.133+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be0007aa0 0x560be0009e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.133+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be0007aa0 0x560be0009e90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7542fac72c3ac2 server_cookie=85bca58f31425d0f in_seq=0 out_seq=0 2024-04-21T09:12:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.133+0000 7fc7a17fa640 10 client.16749 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7a17fa640 1 -- 172.21.15.66:0/4031880431 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bdff60c00 con 0x560be0007aa0 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7a17fa640 10 client.16749 handle_command_reply: tid=0 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 client.16749 shutdown 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 msgr2=0x560bdfff4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff4ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00117b0 tx=0x7fc7b0011c70 comp rx=0 tx=0).stop 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 shutdown_connections 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be0007aa0 0x560be0009e90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560bdfff5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7a17fa640 0 client.16749 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560bdfff4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/4031880431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff4ab0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 >> 172.21.15.66:0/4031880431 conn(0x560bdffe8cb0 msgr2=0x560bdffeaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 shutdown_connections 2024-04-21T09:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 1 -- 172.21.15.66:0/4031880431 wait complete. 2024-04-21T09:12:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.134+0000 7fc7c2a70740 20 client.16749 trim_cache size 0 max 16384 2024-04-21T09:12:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 Processor -- start 2024-04-21T09:12:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 -- start start 2024-04-21T09:12:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560be0003700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560be000f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81b3e60 con 0x560bdffee580 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81b1730 con 0x560bdffeeea0 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81af560 con 0x560bdfff7a80 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52730/0 (socket says 172.21.15.66:52730) 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 -- 172.21.15.66:0/3926388624 learned_addr learned my addr 172.21.15.66:0/3926388624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560be0003700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 -- 172.21.15.66:0/3926388624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 msgr2=0x560be000f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560be000f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 -- 172.21.15.66:0/3926388624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 msgr2=0x560be0003700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560be0003700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.152+0000 7fc7be41e640 1 -- 172.21.15.66:0/3926388624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdfff3000 con 0x560bdffee580 2024-04-21T09:12:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7be41e640 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff71e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00108b0 tx=0x7fc7b001deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3863de3967c1399 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7a37fe640 1 -- 172.21.15.66:0/3926388624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b002d070 con 0x560bdffee580 2024-04-21T09:12:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7a37fe640 1 -- 172.21.15.66:0/3926388624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0018070 con 0x560bdffee580 2024-04-21T09:12:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7a37fe640 1 -- 172.21.15.66:0/3926388624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0022040 con 0x560bdffee580 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 -- 172.21.15.66:0/3926388624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 msgr2=0x560bdfff71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff71e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00108b0 tx=0x7fc7b001deb0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 -- 172.21.15.66:0/3926388624 shutdown_connections 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdfff7a80 0x560be000f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdffeeea0 0x560be0003700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/3926388624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffee580 0x560bdfff71e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 -- 172.21.15.66:0/3926388624 >> 172.21.15.66:0/3926388624 conn(0x560bdffe8cb0 msgr2=0x560bdffefe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 -- 172.21.15.66:0/3926388624 shutdown_connections 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.153+0000 7fc7c2a70740 1 -- 172.21.15.66:0/3926388624 wait complete. 2024-04-21T09:12:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 -- start start 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 Processor -- start 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdffee580 0x560bdffffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfffb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdfff7a80 0x560bdfffb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81afbf0 con 0x560bdffeeea0 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81b1060 con 0x560bdfff7a80 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81b02c0 con 0x560bdffee580 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7c2a70740 10 client.0 _ll_register_callbacks cb 0x7fc7b00096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7bd41c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfffb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7bd41c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfffb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52742/0 (socket says 172.21.15.66:52742) 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1279238734 learned_addr learned my addr 172.21.15.66:0/1279238734 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.154+0000 7fc7be41e640 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdfff7a80 0x560bdfffb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1279238734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdffee580 msgr2=0x560bdffffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7be41e640 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdffee580 0x560bdffffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdffee580 0x560bdffffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdfff7a80 msgr2=0x560bdfffb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdfff7a80 0x560bdfffb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7bd41c640 1 -- 172.21.15.66:0/1279238734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdfffbd50 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7bd41c640 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfffb1d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8005410 tx=0x7fc7a802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128ea79d5d0260b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7a17fa640 1 -- 172.21.15.66:0/1279238734 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8044040 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7a17fa640 1 -- 172.21.15.66:0/1279238734 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc7a8004240 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7b0018070 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7a17fa640 1 -- 172.21.15.66:0/1279238734 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7a805c050 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7c2a70740 10 client.16755 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560be006de60 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7a17fa640 1 -- 172.21.15.66:0/1279238734 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0018070 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7a17fa640 1 -- 172.21.15.66:0/1279238734 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7a804e050 con 0x560bdffeeea0 2024-04-21T09:12:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7c2a70740 10 client.16755 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.155+0000 7fc7c2a70740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7c2a70740 20 client.16755 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be006e260 0x560be0070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7c2a70740 4 client.16755 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bdff5aa70 con 0x560be006e260 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7be41e640 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be006e260 0x560be0070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7be41e640 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be006e260 0x560be0070650 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6537cf26d14fdcf server_cookie=cedaba95b8baa69 in_seq=0 out_seq=0 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7a17fa640 10 client.16755 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7a17fa640 1 -- 172.21.15.66:0/1279238734 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560bdff5aa70 con 0x560be006e260 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7a17fa640 10 client.16755 handle_command_reply: tid=0 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7c2a70740 1 client.16755 shutdown 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 msgr2=0x560bdfffb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.156+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfffb1d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8005410 tx=0x7fc7a802bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 shutdown_connections 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x560be006e260 0x560be0070650 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560bdfff7a80 0x560bdfffb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560bdffeeea0 0x560bdfffb1d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7a17fa640 0 client.16755 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 --2- 172.21.15.66:0/1279238734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560bdffee580 0x560bdffffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 >> 172.21.15.66:0/1279238734 conn(0x560bdffe8cb0 msgr2=0x560bdffe74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 shutdown_connections 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 1 -- 172.21.15.66:0/1279238734 wait complete. 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7c2a70740 20 client.16755 trim_cache size 0 max 16384 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:12:02.163 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:12:02.164 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:12:02.164 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:12:02.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7a40a3d20 msgr2=0x7fc7a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7a40a3d20 0x7fc7a40a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000c00 tx=0x7fc7b4005f70 comp rx=0 tx=0).stop 2024-04-21T09:12:02.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8100d40 msgr2=0x7fc7b81a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8100d40 0x7fc7b81a06d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400d950 tx=0x7fc7b400de10 comp rx=0 tx=0).stop 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 shutdown_connections 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7a40a3d20 0x7fc7a40a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7b810e8f0 0x7fc7b819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b8101660 0x7fc7b81a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 --2- 172.21.15.66:0/312677178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7b8100d40 0x7fc7b81a06d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 >> 172.21.15.66:0/312677178 conn(0x7fc7b8077f70 msgr2=0x7fc7b8110030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 shutdown_connections 2024-04-21T09:12:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.157+0000 7fc7bfe81640 1 -- 172.21.15.66:0/312677178 wait complete. 2024-04-21T09:12:02.169 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:12:02.169 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:12:02.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.270+0000 7f9f57a99640 1 -- 172.21.15.66:0/165299766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500690e0 msgr2=0x7f9f50069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.270+0000 7f9f57a99640 1 --2- 172.21.15.66:0/165299766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500690e0 0x7f9f50069560 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009900 tx=0x7f9f4002bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:02.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- 172.21.15.66:0/165299766 shutdown_connections 2024-04-21T09:12:02.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 --2- 172.21.15.66:0/165299766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f50069aa0 0x7f9f5010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 --2- 172.21.15.66:0/165299766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500690e0 0x7f9f50069560 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 --2- 172.21.15.66:0/165299766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f50106f50 0x7f9f50107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- 172.21.15.66:0/165299766 >> 172.21.15.66:0/165299766 conn(0x7f9f50077e30 msgr2=0x7f9f50078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- 172.21.15.66:0/165299766 shutdown_connections 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- 172.21.15.66:0/165299766 wait complete. 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 Processor -- start 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- start start 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f500690e0 0x7f9f501986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f50069aa0 0x7f9f50198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f50106f50 0x7f9f5019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501121d0 con 0x7f9f50106f50 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50112050 con 0x7f9f50069aa0 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.271+0000 7f9f57a99640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50112350 con 0x7f9f500690e0 2024-04-21T09:12:02.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f50106f50 0x7f9f5019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f50106f50 0x7f9f5019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52768/0 (socket says 172.21.15.66:52768) 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 -- 172.21.15.66:0/936266272 learned_addr learned my addr 172.21.15.66:0/936266272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f55034640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f50069aa0 0x7f9f50198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 -- 172.21.15.66:0/936266272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f500690e0 msgr2=0x7f9f501986f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f500690e0 0x7f9f501986f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 -- 172.21.15.66:0/936266272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f50069aa0 msgr2=0x7f9f50198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f50069aa0 0x7f9f50198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 -- 172.21.15.66:0/936266272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5019d6e0 con 0x7f9f50106f50 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f55034640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f50069aa0 0x7f9f50198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f56036640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f50106f50 0x7f9f5019cfb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00b970 tx=0x7f9f4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f46ffd640 1 -- 172.21.15.66:0/936266272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c019070 con 0x7f9f50106f50 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5019d9d0 con 0x7f9f50106f50 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f46ffd640 1 -- 172.21.15.66:0/936266272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c014440 con 0x7f9f50106f50 2024-04-21T09:12:02.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f46ffd640 1 -- 172.21.15.66:0/936266272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0075d0 con 0x7f9f50106f50 2024-04-21T09:12:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.272+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a5260 con 0x7f9f50106f50 2024-04-21T09:12:02.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.273+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f50068de0 con 0x7f9f50106f50 2024-04-21T09:12:02.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.275+0000 7f9f46ffd640 1 -- 172.21.15.66:0/936266272 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9f4c01c070 con 0x7f9f50106f50 2024-04-21T09:12:02.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.275+0000 7f9f46ffd640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9f3c0a3d20 0x7f9f3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.276+0000 7f9f55835640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9f3c0a3d20 0x7f9f3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.276+0000 7f9f46ffd640 1 -- 172.21.15.66:0/936266272 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f9f4c0d0580 con 0x7f9f50106f50 2024-04-21T09:12:02.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.276+0000 7f9f55835640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9f3c0a3d20 0x7f9f3c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800a930 tx=0x7f9f48005c00 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.279+0000 7f9f46ffd640 1 -- 172.21.15.66:0/936266272 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f4c09c9c0 con 0x7f9f50106f50 2024-04-21T09:12:02.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.506+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f5002d5c0 con 0x7f9f50106f50 2024-04-21T09:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.507+0000 7f9f46ffd640 1 -- 172.21.15.66:0/936266272 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f9f4c09c2c0 con 0x7f9f50106f50 2024-04-21T09:12:02.508 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:12:02.508 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:12:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9f3c0a3d20 msgr2=0x7f9f3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9f3c0a3d20 0x7f9f3c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800a930 tx=0x7f9f48005c00 comp rx=0 tx=0).stop 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f50106f50 msgr2=0x7f9f5019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f50106f50 0x7f9f5019cfb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00b970 tx=0x7f9f4c00be30 comp rx=0 tx=0).stop 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 shutdown_connections 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9f3c0a3d20 0x7f9f3c0a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f50106f50 0x7f9f5019cfb0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f50069aa0 0x7f9f50198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 --2- 172.21.15.66:0/936266272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f500690e0 0x7f9f501986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 >> 172.21.15.66:0/936266272 conn(0x7f9f50077e30 msgr2=0x7f9f500feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 shutdown_connections 2024-04-21T09:12:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.509+0000 7f9f57a99640 1 -- 172.21.15.66:0/936266272 wait complete. 2024-04-21T09:12:02.521 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:12:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 -- 172.21.15.66:0/2172699888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd0101a50 msgr2=0x7f2cd0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2172699888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd0101a50 0x7f2cd0105a90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009900 tx=0x7f2cc002bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 -- 172.21.15.66:0/2172699888 shutdown_connections 2024-04-21T09:12:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2172699888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd0106170 0x7f2cd010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2172699888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd0101a50 0x7f2cd0105a90 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2172699888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd01010a0 0x7f2cd0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 -- 172.21.15.66:0/2172699888 >> 172.21.15.66:0/2172699888 conn(0x7f2cd0077eb0 msgr2=0x7f2cd00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 -- 172.21.15.66:0/2172699888 shutdown_connections 2024-04-21T09:12:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.624+0000 7f2cd7375640 1 -- 172.21.15.66:0/2172699888 wait complete. 2024-04-21T09:12:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 Processor -- start 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 -- start start 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd01010a0 0x7f2cd0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd0106170 0x7f2cd0197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd019bfe0 0x7f2cd019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0111110 con 0x7f2cd019bfe0 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0110f90 con 0x7f2cd01010a0 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd7375640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0111290 con 0x7f2cd0106170 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd019bfe0 0x7f2cd019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd5111640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd01010a0 0x7f2cd0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd019bfe0 0x7f2cd019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52788/0 (socket says 172.21.15.66:52788) 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.625+0000 7f2cd5912640 1 -- 172.21.15.66:0/2945747726 learned_addr learned my addr 172.21.15.66:0/2945747726 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd4910640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd0106170 0x7f2cd0197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd5912640 1 -- 172.21.15.66:0/2945747726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd0106170 msgr2=0x7f2cd0197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd0106170 0x7f2cd0197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd5912640 1 -- 172.21.15.66:0/2945747726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd01010a0 msgr2=0x7f2cd0197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd01010a0 0x7f2cd0197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd5912640 1 -- 172.21.15.66:0/2945747726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd019cad0 con 0x7f2cd019bfe0 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd5111640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd01010a0 0x7f2cd0197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd019bfe0 0x7f2cd019c490 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800db90 tx=0x7f2cc800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2945747726 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800e980 con 0x7f2cd019bfe0 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2945747726 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc8009580 con 0x7f2cd019bfe0 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd019cd60 con 0x7f2cd019bfe0 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2945747726 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8016650 con 0x7f2cd019bfe0 2024-04-21T09:12:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.626+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd006a7e0 con 0x7f2cd019bfe0 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.628+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2945747726 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2cc8009730 con 0x7f2cd019bfe0 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 Processor -- start 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 -- start start 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d193b0 0x556b89d19790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cc67fc640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 0x556b89d22890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d22e60 0x556b89d232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556b89c7e1f0 con 0x556b89d193b0 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b89c1b4b0 con 0x556b89d22e60 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556b89c7a1b0 con 0x556b89d19d60 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd4910640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d193b0 0x556b89d19790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd5111640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd4910640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d193b0 0x556b89d19790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52792/0 (socket says 172.21.15.66:52792) 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd4910640 1 -- 172.21.15.66:0/3770260914 learned_addr learned my addr 172.21.15.66:0/3770260914 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cc67fc640 1 -- 172.21.15.66:0/2945747726 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2cc80d20f0 con 0x7f2cd019bfe0 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd5912640 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d22e60 0x556b89d232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.629+0000 7f2cd4910640 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 0x556b89d22890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cd4910640 1 -- 172.21.15.66:0/3770260914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 msgr2=0x556b89d22890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cd4910640 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 0x556b89d22890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cd4910640 1 -- 172.21.15.66:0/3770260914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d22e60 msgr2=0x556b89d232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cd4910640 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d22e60 0x556b89d232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cd4910640 1 -- 172.21.15.66:0/3770260914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b89c0d1b0 con 0x556b89d193b0 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cd5111640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009fc0 tx=0x7f2ccc005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cd4910640 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d193b0 0x556b89d19790 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005940 tx=0x7f2cc00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9d059abf446c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cbb7fe640 1 -- 172.21.15.66:0/3770260914 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc002c7e0 con 0x556b89d193b0 2024-04-21T09:12:02.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cbb7fe640 1 -- 172.21.15.66:0/3770260914 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc0048070 con 0x556b89d193b0 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.630+0000 7f2cbb7fe640 1 -- 172.21.15.66:0/3770260914 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00048c0 con 0x556b89d193b0 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- 172.21.15.66:0/3770260914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d193b0 msgr2=0x556b89d19790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d193b0 0x556b89d19790 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005940 tx=0x7f2cc00058b0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- 172.21.15.66:0/3770260914 shutdown_connections 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d22e60 0x556b89d232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 0x556b89d22890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/3770260914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d193b0 0x556b89d19790 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- 172.21.15.66:0/3770260914 >> 172.21.15.66:0/3770260914 conn(0x556b89d13ba0 msgr2=0x556b89d13f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- 172.21.15.66:0/3770260914 shutdown_connections 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- 172.21.15.66:0/3770260914 wait complete. 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- start start 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 Processor -- start 2024-04-21T09:12:02.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 0x556b89d292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 0x556b89d26630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556b89d29d40 con 0x556b89d22e60 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b89c37500 con 0x556b89d19d60 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.631+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556b89d29ec0 con 0x556b89d193b0 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd9f64740 10 client.0 _ll_register_callbacks cb 0x556b89c05b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d26b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52802/0 (socket says 172.21.15.66:52802) 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 -- 172.21.15.66:0/1417101218 learned_addr learned my addr 172.21.15.66:0/1417101218 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd4910640 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 0x556b89d26630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5111640 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 0x556b89d292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 -- 172.21.15.66:0/1417101218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 msgr2=0x556b89d292a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 0x556b89d292a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 -- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 msgr2=0x556b89d26630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 0x556b89d26630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 -- 172.21.15.66:0/1417101218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b899903f0 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd5912640 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d26b90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2cc809db30 tx=0x7f2cc8066aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33de9e83e97816a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/1417101218 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80a21e0 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/1417101218 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2cc80cf750 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc0048070 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/1417101218 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cc8062e60 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd9f64740 10 client.16773 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.632+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b89a5a7d0 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/1417101218 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc0048070 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/1417101218 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc80a66a0 con 0x556b89d22e60 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cd9f64740 10 client.16773 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cd9f64740 10 client.16773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cd9f64740 20 client.16773 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d36f80 0x556b89d39370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cd9f64740 4 client.16773 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b89c1d930 con 0x556b89d36f80 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.633+0000 7f2cd5111640 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d36f80 0x556b89d39370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.634+0000 7f2cd5111640 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d36f80 0x556b89d39370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=291d840cfd1b803b server_cookie=a8b7364205bc2f54 in_seq=0 out_seq=0 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.634+0000 7f2cb8ff9640 10 client.16773 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.634+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/1417101218 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b89c1d930 con 0x556b89d36f80 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.634+0000 7f2cb8ff9640 10 client.16773 handle_command_reply: tid=0 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.634+0000 7f2cd9f64740 1 client.16773 shutdown 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.634+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 msgr2=0x556b89d26b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.634+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d26b90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2cc809db30 tx=0x7f2cc8066aa0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 shutdown_connections 2024-04-21T09:12:02.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d36f80 0x556b89d39370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d26b90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 0x556b89d26630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cb8ff9640 0 client.16773 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/1417101218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 0x556b89d292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 >> 172.21.15.66:0/1417101218 conn(0x556b89d13ba0 msgr2=0x556b89d158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 shutdown_connections 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 1 -- 172.21.15.66:0/1417101218 wait complete. 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.635+0000 7f2cd9f64740 20 client.16773 trim_cache size 0 max 16384 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 Processor -- start 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 -- start start 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d193b0 0x556b89d32c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 0x556b89d224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d24250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01af860 con 0x556b89d22e60 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01ad130 con 0x556b89d193b0 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.648+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01aaf60 con 0x556b89d19d60 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d24250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d24250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52816/0 (socket says 172.21.15.66:52816) 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 -- 172.21.15.66:0/2358378307 learned_addr learned my addr 172.21.15.66:0/2358378307 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5111640 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d193b0 0x556b89d32c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 -- 172.21.15.66:0/2358378307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 msgr2=0x556b89d224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 0x556b89d224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 -- 172.21.15.66:0/2358378307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d193b0 msgr2=0x556b89d32c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d193b0 0x556b89d32c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 -- 172.21.15.66:0/2358378307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b89d0f470 con 0x556b89d22e60 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5111640 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d193b0 0x556b89d32c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d24250 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8066cb0 tx=0x7f2cc809dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276293abd4101dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cbaffd640 1 -- 172.21.15.66:0/2358378307 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80b6070 con 0x556b89d22e60 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cbaffd640 1 -- 172.21.15.66:0/2358378307 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc80a21e0 con 0x556b89d22e60 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.649+0000 7f2cbaffd640 1 -- 172.21.15.66:0/2358378307 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80a6610 con 0x556b89d22e60 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2358378307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 msgr2=0x556b89d24250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d24250 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8066cb0 tx=0x7f2cc809dfb0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2358378307 shutdown_connections 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d24250 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d19d60 0x556b89d224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2358378307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d193b0 0x556b89d32c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2358378307 >> 172.21.15.66:0/2358378307 conn(0x556b89d13ba0 msgr2=0x556b89d21dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2358378307 shutdown_connections 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2358378307 wait complete. 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- start start 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 Processor -- start 2024-04-21T09:12:02.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 0x556b89d2f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 0x556b89d23840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d23ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01ab5f0 con 0x556b89d22e60 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01aca60 con 0x556b89d19d60 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01abcc0 con 0x556b89d193b0 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.650+0000 7f2cd9f64740 10 client.0 _ll_register_callbacks cb 0x7f2cc80b7790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d23ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d23ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52820/0 (socket says 172.21.15.66:52820) 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 -- 172.21.15.66:0/2639847389 learned_addr learned my addr 172.21.15.66:0/2639847389 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 -- 172.21.15.66:0/2639847389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 msgr2=0x556b89d2f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 0x556b89d2f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 -- 172.21.15.66:0/2639847389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 msgr2=0x556b89d23840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 0x556b89d23840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 -- 172.21.15.66:0/2639847389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b89d2d1b0 con 0x556b89d22e60 2024-04-21T09:12:02.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd5912640 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d23ec0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2cc809ddc0 tx=0x7f2cc8009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936eeffd62e27a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/2639847389 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80b6040 con 0x556b89d22e60 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/2639847389 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2cc80cf6c0 con 0x556b89d22e60 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc80a21e0 con 0x556b89d22e60 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/2639847389 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cc80a6aa0 con 0x556b89d22e60 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd9f64740 10 client.16779 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b89d9d5c0 con 0x556b89d22e60 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/2639847389 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc80a21e0 con 0x556b89d22e60 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.651+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/2639847389 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc800eb20 con 0x556b89d22e60 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd9f64740 10 client.16779 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd9f64740 10 client.16779 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd9f64740 20 client.16779 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d9da90 0x556b89d9fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd9f64740 4 client.16779 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b89c8bc10 con 0x556b89d9da90 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd5111640 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d9da90 0x556b89d9fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cb8ff9640 10 client.16779 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cd5111640 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d9da90 0x556b89d9fe80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3d17a99ac4dc91a server_cookie=e7498bc7fdcd7f31 in_seq=0 out_seq=0 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.652+0000 7f2cb8ff9640 1 -- 172.21.15.66:0/2639847389 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b89c8bc10 con 0x556b89d9da90 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cb8ff9640 10 client.16779 handle_command_reply: tid=0 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 client.16779 shutdown 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 msgr2=0x556b89d23ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d23ec0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2cc809ddc0 tx=0x7f2cc8009e10 comp rx=0 tx=0).stop 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 shutdown_connections 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x556b89d9da90 0x556b89d9fe80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b89d22e60 0x556b89d23ec0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b89d19d60 0x556b89d23840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cb8ff9640 0 client.16779 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:02.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 --2- 172.21.15.66:0/2639847389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556b89d193b0 0x556b89d2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 >> 172.21.15.66:0/2639847389 conn(0x556b89d13ba0 msgr2=0x556b89d22660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 shutdown_connections 2024-04-21T09:12:02.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 1 -- 172.21.15.66:0/2639847389 wait complete. 2024-04-21T09:12:02.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd9f64740 20 client.16779 trim_cache size 0 max 16384 2024-04-21T09:12:02.679 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 msgr2=0x7f2cbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009fc0 tx=0x7f2ccc005f90 comp rx=0 tx=0).stop 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd019bfe0 msgr2=0x7f2cd019c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd019bfe0 0x7f2cd019c490 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800db90 tx=0x7f2cc800dc80 comp rx=0 tx=0).stop 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 shutdown_connections 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2cbc0a3d20 0x7f2cbc0a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd019bfe0 0x7f2cd019c490 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd0106170 0x7f2cd0197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 --2- 172.21.15.66:0/2945747726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cd01010a0 0x7f2cd0197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.653+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 >> 172.21.15.66:0/2945747726 conn(0x7f2cd0077eb0 msgr2=0x7f2cd00ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.654+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 shutdown_connections 2024-04-21T09:12:02.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.654+0000 7f2cd7375640 1 -- 172.21.15.66:0/2945747726 wait complete. 2024-04-21T09:12:02.681 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 -- 172.21.15.66:0/4075247638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0102190 msgr2=0x7f74c0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 --2- 172.21.15.66:0/4075247638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0102190 0x7f74c0102570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009900 tx=0x7f74b002bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 -- 172.21.15.66:0/4075247638 shutdown_connections 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 --2- 172.21.15.66:0/4075247638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74c0107260 0x7f74c010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 --2- 172.21.15.66:0/4075247638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c0102b40 0x7f74c0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 --2- 172.21.15.66:0/4075247638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0102190 0x7f74c0102570 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 -- 172.21.15.66:0/4075247638 >> 172.21.15.66:0/4075247638 conn(0x7f74c00fd990 msgr2=0x7f74c00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 -- 172.21.15.66:0/4075247638 shutdown_connections 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.789+0000 7f74c5c32640 1 -- 172.21.15.66:0/4075247638 wait complete. 2024-04-21T09:12:02.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 Processor -- start 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 -- start start 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74c0102190 0x7f74c019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0107260 0x7f74c019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c0198620 0x7f74c0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112220 con 0x7f74c0107260 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01120a0 con 0x7f74c0198620 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74c5c32640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01123a0 con 0x7f74c0102190 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0107260 0x7f74c019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0107260 0x7f74c019e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52842/0 (socket says 172.21.15.66:52842) 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 -- 172.21.15.66:0/577076462 learned_addr learned my addr 172.21.15.66:0/577076462 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74bffff640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c0198620 0x7f74c0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74c0102190 0x7f74c019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 -- 172.21.15.66:0/577076462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74c0102190 msgr2=0x7f74c019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74c0102190 0x7f74c019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 -- 172.21.15.66:0/577076462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c0198620 msgr2=0x7f74c0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c0198620 0x7f74c0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.790+0000 7f74beffd640 1 -- 172.21.15.66:0/577076462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c0199140 con 0x7f74c0107260 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.791+0000 7f74beffd640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0107260 0x7f74c019e9b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f74b400d770 tx=0x7f74b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.791+0000 7f74bcff9640 1 -- 172.21.15.66:0/577076462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4017070 con 0x7f74c0107260 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.791+0000 7f74bcff9640 1 -- 172.21.15.66:0/577076462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b4009c30 con 0x7f74c0107260 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.791+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c0199430 con 0x7f74c0107260 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.791+0000 7f74bcff9640 1 -- 172.21.15.66:0/577076462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b400f620 con 0x7f74c0107260 2024-04-21T09:12:02.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.791+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c01a5680 con 0x7f74c0107260 2024-04-21T09:12:02.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.793+0000 7f74bcff9640 1 -- 172.21.15.66:0/577076462 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f74b401a070 con 0x7f74c0107260 2024-04-21T09:12:02.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.793+0000 7f74c881c740 1 Processor -- start 2024-04-21T09:12:02.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74c881c740 1 -- start start 2024-04-21T09:12:02.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74c881c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0e4d0 0x564818f0e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74c881c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74c881c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 0x564818f19e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74c881c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564818e731f0 con 0x564818f0ee80 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74c881c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564818e104b0 con 0x564818f0e4d0 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74c881c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564818e6f1b0 con 0x564818f17a00 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bcff9640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74a80a3d20 0x7f74a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f17430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52846/0 (socket says 172.21.15.66:52846) 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 -- 172.21.15.66:0/1691970048 learned_addr learned my addr 172.21.15.66:0/1691970048 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bffff640 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0e4d0 0x564818f0e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bffff640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74a80a3d20 0x7f74a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bcff9640 1 -- 172.21.15.66:0/577076462 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f74b40d0720 con 0x7f74c0107260 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 -- 172.21.15.66:0/1691970048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 msgr2=0x564818f19e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 0x564818f19e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 -- 172.21.15.66:0/1691970048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0e4d0 msgr2=0x564818f0e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0e4d0 0x564818f0e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.794+0000 7f74bf7fe640 1 -- 172.21.15.66:0/1691970048 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564818e021b0 con 0x564818f0ee80 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.795+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f17430 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009900 tx=0x7f74b002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe0dcb3dff3bc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.795+0000 7f74bffff640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74a80a3d20 0x7f74a80a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009900 tx=0x7f74b8009650 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.795+0000 7f74a5ffb640 1 -- 172.21.15.66:0/1691970048 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0044020 con 0x564818f0ee80 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.795+0000 7f74a5ffb640 1 -- 172.21.15.66:0/1691970048 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b002cc90 con 0x564818f0ee80 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.795+0000 7f74a5ffb640 1 -- 172.21.15.66:0/1691970048 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b00046c0 con 0x564818f0ee80 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- 172.21.15.66:0/1691970048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 msgr2=0x564818f17430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f17430 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009900 tx=0x7f74b002c650 comp rx=0 tx=0).stop 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- 172.21.15.66:0/1691970048 shutdown_connections 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 0x564818f19e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f17430 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 --2- 172.21.15.66:0/1691970048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0e4d0 0x564818f0e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- 172.21.15.66:0/1691970048 >> 172.21.15.66:0/1691970048 conn(0x564818f08ba0 msgr2=0x564818f08f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- 172.21.15.66:0/1691970048 shutdown_connections 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- 172.21.15.66:0/1691970048 wait complete. 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- start start 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 Processor -- start 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f17350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f14cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564818f1a910 con 0x564818f0ee80 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564818e2c500 con 0x564818f17a00 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564818f1aa90 con 0x564818f0e4d0 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.796+0000 7f74c881c740 10 client.0 _ll_register_callbacks cb 0x564818dfab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52850/0 (socket says 172.21.15.66:52850) 2024-04-21T09:12:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 -- 172.21.15.66:0/79617726 learned_addr learned my addr 172.21.15.66:0/79617726 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f17350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74bffff640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f14cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 -- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 msgr2=0x564818f17350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f17350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 -- 172.21.15.66:0/79617726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 msgr2=0x564818f14cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f14cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 -- 172.21.15.66:0/79617726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564818b853f0 con 0x564818f0ee80 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74bffff640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f14cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74beffd640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f146e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f74b4098ce0 tx=0x7f74b4009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6ff85b8e77ae1a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f749effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f749effd640 1 -- 172.21.15.66:0/79617726 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4021040 con 0x564818f0ee80 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f749effd640 1 -- 172.21.15.66:0/79617726 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f74b40d0330 con 0x564818f0ee80 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b002cc90 con 0x564818f0ee80 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f749effd640 1 -- 172.21.15.66:0/79617726 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74b40ba050 con 0x564818f0ee80 2024-04-21T09:12:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74c881c740 10 client.16791 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564818c4f7d0 con 0x564818f0ee80 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.797+0000 7f749effd640 1 -- 172.21.15.66:0/79617726 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b407d080 con 0x564818f0ee80 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f749effd640 1 -- 172.21.15.66:0/79617726 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74b4098060 con 0x564818f0ee80 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f74c881c740 10 client.16791 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f74c881c740 10 client.16791 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f74c881c740 20 client.16791 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f74c881c740 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f27bf0 0x564818f29fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f74c881c740 4 client.16791 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564818e12930 con 0x564818f27bf0 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.798+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f27bf0 0x564818f29fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f749effd640 10 client.16791 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:02.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f27bf0 0x564818f29fe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f122153ca4bbe277 server_cookie=11df3229e1305220 in_seq=0 out_seq=0 2024-04-21T09:12:02.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f749effd640 1 -- 172.21.15.66:0/79617726 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564818e12930 con 0x564818f27bf0 2024-04-21T09:12:02.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f749effd640 10 client.16791 handle_command_reply: tid=0 2024-04-21T09:12:02.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 client.16791 shutdown 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 msgr2=0x564818f146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f146e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f74b4098ce0 tx=0x7f74b4009de0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 shutdown_connections 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f14cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f27bf0 0x564818f29fe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f146e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 --2- 172.21.15.66:0/79617726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f17350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f749effd640 0 client.16791 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.799+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 >> 172.21.15.66:0/79617726 conn(0x564818f08ba0 msgr2=0x564818f0a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.800+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 shutdown_connections 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.800+0000 7f74c881c740 1 -- 172.21.15.66:0/79617726 wait complete. 2024-04-21T09:12:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.800+0000 7f74c881c740 20 client.16791 trim_cache size 0 max 16384 2024-04-21T09:12:02.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 Processor -- start 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 -- start start 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0e4d0 0x564818f237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0ee80 0x564818f16e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 0x564818f17350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01b1c00 con 0x564818f0e4d0 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01af4d0 con 0x564818f0ee80 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74c881c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ad300 con 0x564818f17a00 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0e4d0 0x564818f237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0e4d0 0x564818f237d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52852/0 (socket says 172.21.15.66:52852) 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 -- 172.21.15.66:0/3712224564 learned_addr learned my addr 172.21.15.66:0/3712224564 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74beffd640 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0ee80 0x564818f16e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 -- 172.21.15.66:0/3712224564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 msgr2=0x564818f17350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 0x564818f17350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 -- 172.21.15.66:0/3712224564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0ee80 msgr2=0x564818f16e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0ee80 0x564818f16e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.813+0000 7f74bf7fe640 1 -- 172.21.15.66:0/3712224564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564818f2a350 con 0x564818f0e4d0 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0e4d0 0x564818f237d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x564818f0dfa0 tx=0x7f74b002c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864cc98e1d584259 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74a57fa640 1 -- 172.21.15.66:0/3712224564 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0044040 con 0x564818f0e4d0 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74a57fa640 1 -- 172.21.15.66:0/3712224564 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b0005620 con 0x564818f0e4d0 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74a57fa640 1 -- 172.21.15.66:0/3712224564 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0003850 con 0x564818f0e4d0 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 -- 172.21.15.66:0/3712224564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0e4d0 msgr2=0x564818f237d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0e4d0 0x564818f237d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x564818f0dfa0 tx=0x7f74b002c740 comp rx=0 tx=0).stop 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 -- 172.21.15.66:0/3712224564 shutdown_connections 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f17a00 0x564818f17350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f0ee80 0x564818f16e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 --2- 172.21.15.66:0/3712224564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0e4d0 0x564818f237d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 -- 172.21.15.66:0/3712224564 >> 172.21.15.66:0/3712224564 conn(0x564818f08ba0 msgr2=0x564818f0fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 -- 172.21.15.66:0/3712224564 shutdown_connections 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.814+0000 7f74c881c740 1 -- 172.21.15.66:0/3712224564 wait complete. 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 -- start start 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 Processor -- start 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f20130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f11140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ad990 con 0x564818f0ee80 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01aee00 con 0x564818f17a00 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ae060 con 0x564818f0e4d0 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74c881c740 10 client.0 _ll_register_callbacks cb 0x7f74b409be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f11140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f11140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52860/0 (socket says 172.21.15.66:52860) 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 -- 172.21.15.66:0/875007069 learned_addr learned my addr 172.21.15.66:0/875007069 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74bffff640 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f20130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 -- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 msgr2=0x564818f20130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f20130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 -- 172.21.15.66:0/875007069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 msgr2=0x564818f116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.815+0000 7f74beffd640 1 -- 172.21.15.66:0/875007069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564818f11ed0 con 0x564818f0ee80 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74beffd640 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f11140 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f74b409a5c0 tx=0x7f74b409ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b66e58d736db68 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f749effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f749effd640 1 -- 172.21.15.66:0/875007069 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4014070 con 0x564818f0ee80 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f749effd640 1 -- 172.21.15.66:0/875007069 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f74b40a0270 con 0x564818f0ee80 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b0005620 con 0x564818f0ee80 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f749effd640 1 -- 172.21.15.66:0/875007069 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74b4021040 con 0x564818f0ee80 2024-04-21T09:12:02.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 10 client.16797 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564818f8e0a0 con 0x564818f0ee80 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f749effd640 1 -- 172.21.15.66:0/875007069 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b0005620 con 0x564818f0ee80 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f749effd640 1 -- 172.21.15.66:0/875007069 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74b409b1d0 con 0x564818f0ee80 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 10 client.16797 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 10 client.16797 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 20 client.16797 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f8e7e0 0x564818f90c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 4 client.16797 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564818e80c10 con 0x564818f8e7e0 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.816+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f8e7e0 0x564818f90c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f749effd640 10 client.16797 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74bf7fe640 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f8e7e0 0x564818f90c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=997bd3633af96e3e server_cookie=ddbdecb8490d931 in_seq=0 out_seq=0 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f749effd640 1 -- 172.21.15.66:0/875007069 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564818e80c10 con 0x564818f8e7e0 2024-04-21T09:12:02.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f749effd640 10 client.16797 handle_command_reply: tid=0 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 client.16797 shutdown 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 msgr2=0x564818f11140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f11140 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f74b409a5c0 tx=0x7f74b409ca90 comp rx=0 tx=0).stop 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 shutdown_connections 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564818f17a00 0x564818f116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x564818f8e7e0 0x564818f90c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564818f0ee80 0x564818f11140 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 --2- 172.21.15.66:0/875007069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564818f0e4d0 0x564818f20130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f749effd640 0 client.16797 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 >> 172.21.15.66:0/875007069 conn(0x564818f08ba0 msgr2=0x564818f0f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 shutdown_connections 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.817+0000 7f74c881c740 1 -- 172.21.15.66:0/875007069 wait complete. 2024-04-21T09:12:02.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c881c740 20 client.16797 trim_cache size 0 max 16384 2024-04-21T09:12:02.825 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:12:02.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74a80a3d20 msgr2=0x7f74a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74a80a3d20 0x7f74a80a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009900 tx=0x7f74b8009650 comp rx=0 tx=0).stop 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0107260 msgr2=0x7f74c019e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0107260 0x7f74c019e9b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f74b400d770 tx=0x7f74b400dc30 comp rx=0 tx=0).stop 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 shutdown_connections 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74a80a3d20 0x7f74a80a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c0198620 0x7f74c0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c0107260 0x7f74c019e9b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 --2- 172.21.15.66:0/577076462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74c0102190 0x7f74c019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 >> 172.21.15.66:0/577076462 conn(0x7f74c00fd990 msgr2=0x7f74c00fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 shutdown_connections 2024-04-21T09:12:02.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.818+0000 7f74c5c32640 1 -- 172.21.15.66:0/577076462 wait complete. 2024-04-21T09:12:02.829 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:12:02.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 -- 172.21.15.66:0/3975809777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098107270 msgr2=0x7f109810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 --2- 172.21.15.66:0/3975809777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098107270 0x7f109810ee40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1090009ca0 tx=0x7f109002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 -- 172.21.15.66:0/3975809777 shutdown_connections 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 --2- 172.21.15.66:0/3975809777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1098107270 0x7f109810ee40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 --2- 172.21.15.66:0/3975809777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f109806b1f0 0x7f1098106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 --2- 172.21.15.66:0/3975809777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f109806a840 0x7f109806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 -- 172.21.15.66:0/3975809777 >> 172.21.15.66:0/3975809777 conn(0x7f1098077eb0 msgr2=0x7f10980782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.932+0000 7f109e52b640 1 -- 172.21.15.66:0/3975809777 shutdown_connections 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 -- 172.21.15.66:0/3975809777 wait complete. 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 Processor -- start 2024-04-21T09:12:02.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 -- start start 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f109806a840 0x7f10981a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098107270 0x7f10981a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f109819a710 0x7f109819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10981121d0 con 0x7f109819a710 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1098112050 con 0x7f1098107270 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.933+0000 7f109e52b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1098112350 con 0x7f109806a840 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f109819a710 0x7f109819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f109819a710 0x7f109819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52872/0 (socket says 172.21.15.66:52872) 2024-04-21T09:12:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 -- 172.21.15.66:0/2677935630 learned_addr learned my addr 172.21.15.66:0/2677935630 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f1097fff640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f109806a840 0x7f10981a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 -- 172.21.15.66:0/2677935630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f109806a840 msgr2=0x7f10981a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f109806a840 0x7f10981a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 -- 172.21.15.66:0/2677935630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098107270 msgr2=0x7f10981a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098107270 0x7f10981a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 -- 172.21.15.66:0/2677935630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109819b310 con 0x7f109819a710 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f1097fff640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f109806a840 0x7f10981a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109cac8640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f109819a710 0x7f109819ab90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1090005e30 tx=0x7f1090032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f10957fa640 1 -- 172.21.15.66:0/2677935630 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090041020 con 0x7f109819a710 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f10957fa640 1 -- 172.21.15.66:0/2677935630 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1090032970 con 0x7f109819a710 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109819b5a0 con 0x7f109819a710 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f10957fa640 1 -- 172.21.15.66:0/2677935630 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090039040 con 0x7f109819a710 2024-04-21T09:12:02.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.934+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981a76e0 con 0x7f109819a710 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.936+0000 7f10957fa640 1 -- 172.21.15.66:0/2677935630 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1090044020 con 0x7f109819a710 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 Processor -- start 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 -- start start 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x5567224158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556722415e80 0x55672241e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55672241ea00 0x556722420e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55672237a1f0 con 0x556722415e80 2024-04-21T09:12:02.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567223174b0 con 0x5567224154d0 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10a11f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5567223761b0 con 0x55672241ea00 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10957fa640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f10800a3d20 0x7f10800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f109cac8640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f10800a3d20 0x7f10800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556722415e80 0x55672241e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10977fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x5567224158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556722415e80 0x55672241e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52886/0 (socket says 172.21.15.66:52886) 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10977fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x5567224158b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:57826/0 (socket says 172.21.15.66:57826) 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 -- 172.21.15.66:0/1579040213 learned_addr learned my addr 172.21.15.66:0/1579040213 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10957fa640 1 -- 172.21.15.66:0/2677935630 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f10900f28a0 con 0x7f109819a710 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10977fe640 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55672241ea00 0x556722420e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 -- 172.21.15.66:0/1579040213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55672241ea00 msgr2=0x556722420e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55672241ea00 0x556722420e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 -- 172.21.15.66:0/1579040213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 msgr2=0x5567224158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x5567224158b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f1097fff640 1 -- 172.21.15.66:0/1579040213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567223091b0 con 0x556722415e80 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.937+0000 7f10977fe640 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x5567224158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.938+0000 7f109cac8640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f10800a3d20 0x7f10800a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10900c6d20 tx=0x7f10900ba240 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.938+0000 7f1097fff640 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556722415e80 0x55672241e430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f108c009c60 tx=0x7f108c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf542cc1615598a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.938+0000 7f107e7fc640 1 -- 172.21.15.66:0/1579040213 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c005020 con 0x556722415e80 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.938+0000 7f107e7fc640 1 -- 172.21.15.66:0/1579040213 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c0047a0 con 0x556722415e80 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.938+0000 7f107e7fc640 1 -- 172.21.15.66:0/1579040213 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c012690 con 0x556722415e80 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.938+0000 7f10a11f6740 1 -- 172.21.15.66:0/1579040213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556722415e80 msgr2=0x55672241e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.938+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556722415e80 0x55672241e430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f108c009c60 tx=0x7f108c010740 comp rx=0 tx=0).stop 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- 172.21.15.66:0/1579040213 shutdown_connections 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55672241ea00 0x556722420e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556722415e80 0x55672241e430 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1579040213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x5567224158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- 172.21.15.66:0/1579040213 >> 172.21.15.66:0/1579040213 conn(0x55672240fba0 msgr2=0x55672240ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- 172.21.15.66:0/1579040213 shutdown_connections 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- 172.21.15.66:0/1579040213 wait complete. 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- start start 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 Processor -- start 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567224154d0 0x55672241e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556722415e80 0x55672241b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x55672241be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556722421970 con 0x55672241ea00 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556722333500 con 0x556722415e80 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556722421af0 con 0x5567224154d0 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10a11f6740 10 client.0 _ll_register_callbacks cb 0x556722301b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.939+0000 7f10977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x55672241be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x55672241be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52890/0 (socket says 172.21.15.66:52890) 2024-04-21T09:12:02.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 -- 172.21.15.66:0/1254113125 learned_addr learned my addr 172.21.15.66:0/1254113125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567224154d0 0x55672241e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f1097fff640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556722415e80 0x55672241b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 -- 172.21.15.66:0/1254113125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567224154d0 msgr2=0x55672241e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567224154d0 0x55672241e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 -- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556722415e80 msgr2=0x55672241b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556722415e80 0x55672241b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 -- 172.21.15.66:0/1254113125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55672208c3f0 con 0x55672241ea00 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f1097fff640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556722415e80 0x55672241b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10977fe640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x55672241be70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f10880148f0 tx=0x7f1088005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a962dd628de70d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10777fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10777fe640 1 -- 172.21.15.66:0/1254113125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088004260 con 0x55672241ea00 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10777fe640 1 -- 172.21.15.66:0/1254113125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1088004400 con 0x55672241ea00 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108c0047a0 con 0x55672241ea00 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10777fe640 1 -- 172.21.15.66:0/1254113125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1088025050 con 0x55672241ea00 2024-04-21T09:12:02.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10a11f6740 10 client.16809 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5567221567d0 con 0x55672241ea00 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.940+0000 7f10777fe640 1 -- 172.21.15.66:0/1254113125 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c0047a0 con 0x55672241ea00 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f10777fe640 1 -- 172.21.15.66:0/1254113125 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10880054d0 con 0x55672241ea00 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f10a11f6740 10 client.16809 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f10a11f6740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f10a11f6740 20 client.16809 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f10a11f6740 4 client.16809 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556722319930 con 0x55672242ec30 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.941+0000 7f1097fff640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f1097fff640 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224310b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e1c5f13d919795 server_cookie=e517891343f6159e in_seq=0 out_seq=0 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10777fe640 10 client.16809 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10777fe640 1 -- 172.21.15.66:0/1254113125 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556722319930 con 0x55672242ec30 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10777fe640 10 client.16809 handle_command_reply: tid=0 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10a11f6740 1 client.16809 shutdown 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 msgr2=0x55672241be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x55672241be70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f10880148f0 tx=0x7f1088005f50 comp rx=0 tx=0).stop 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 shutdown_connections 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224310b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.943+0000 7f10777fe640 0 client.16809 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.942+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x55672241be70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.943+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556722415e80 0x55672241b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.943+0000 7f10a11f6740 1 --2- 172.21.15.66:0/1254113125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567224154d0 0x55672241e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.943+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 >> 172.21.15.66:0/1254113125 conn(0x55672240fba0 msgr2=0x5567224118f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.943+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 shutdown_connections 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.943+0000 7f10a11f6740 1 -- 172.21.15.66:0/1254113125 wait complete. 2024-04-21T09:12:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.943+0000 7f10a11f6740 20 client.16809 trim_cache size 0 max 16384 2024-04-21T09:12:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 Processor -- start 2024-04-21T09:12:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 -- start start 2024-04-21T09:12:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x55672242a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x55672241b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x556722411880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b3e40 con 0x55672241ea00 2024-04-21T09:12:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b1710 con 0x5567224154d0 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10a11f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f10981af540 con 0x556722415e80 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x556722411880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x556722411880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52898/0 (socket says 172.21.15.66:52898) 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.956+0000 7f10977fe640 1 -- 172.21.15.66:0/3028576313 learned_addr learned my addr 172.21.15.66:0/3028576313 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f10977fe640 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x55672242a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f1097fff640 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x55672241b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f10977fe640 1 -- 172.21.15.66:0/3028576313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 msgr2=0x55672241b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f10977fe640 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x55672241b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f10977fe640 1 -- 172.21.15.66:0/3028576313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 msgr2=0x55672242a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f10977fe640 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x55672242a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f10977fe640 1 -- 172.21.15.66:0/3028576313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556722411dc0 con 0x55672241ea00 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f10977fe640 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x556722411880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1088004700 tx=0x7f108800b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0da2518a00dfb5c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f107dffb640 1 -- 172.21.15.66:0/3028576313 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088004260 con 0x55672241ea00 2024-04-21T09:12:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f107dffb640 1 -- 172.21.15.66:0/3028576313 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1088022020 con 0x55672241ea00 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.957+0000 7f107dffb640 1 -- 172.21.15.66:0/3028576313 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088005070 con 0x55672241ea00 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- 172.21.15.66:0/3028576313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 msgr2=0x556722411880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x556722411880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1088004700 tx=0x7f108800b600 comp rx=0 tx=0).stop 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- 172.21.15.66:0/3028576313 shutdown_connections 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x556722411880 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x55672241b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3028576313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x55672242a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- 172.21.15.66:0/3028576313 >> 172.21.15.66:0/3028576313 conn(0x55672240fba0 msgr2=0x55672240ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- 172.21.15.66:0/3028576313 shutdown_connections 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- 172.21.15.66:0/3028576313 wait complete. 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- start start 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 Processor -- start 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x556722426d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x556722427260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x5567224179c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10981afbd0 con 0x55672241ea00 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b1040 con 0x5567224154d0 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b02a0 con 0x556722415e80 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.958+0000 7f10a11f6740 10 client.0 _ll_register_callbacks cb 0x7f1088014410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x5567224179c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x5567224179c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52902/0 (socket says 172.21.15.66:52902) 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 -- 172.21.15.66:0/3151912746 learned_addr learned my addr 172.21.15.66:0/3151912746 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x556722426d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f1097fff640 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x556722427260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 -- 172.21.15.66:0/3151912746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 msgr2=0x556722427260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x556722427260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 -- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 msgr2=0x556722426d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x556722426d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 -- 172.21.15.66:0/3151912746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556722417f80 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10977fe640 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x5567224179c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x556722418bd0 tx=0x7f108800fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1ffaeef2a9336a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10777fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10777fe640 1 -- 172.21.15.66:0/3151912746 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108800fba0 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10777fe640 1 -- 172.21.15.66:0/3151912746 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1088004e10 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1088022020 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10777fe640 1 -- 172.21.15.66:0/3151912746 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1088041050 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10a11f6740 10 client.16815 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.959+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556722494a50 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10777fe640 1 -- 172.21.15.66:0/3151912746 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1088022020 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10777fe640 1 -- 172.21.15.66:0/3151912746 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1088003890 con 0x55672241ea00 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10a11f6740 10 client.16815 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10a11f6740 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10a11f6740 20 client.16815 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10a11f6740 4 client.16815 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556722387c10 con 0x55672242ec30 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f1097fff640 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10777fe640 10 client.16815 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f1097fff640 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224971f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0cd65f2168d0016 server_cookie=1ead8f992ea76ffe in_seq=0 out_seq=0 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.960+0000 7f10777fe640 1 -- 172.21.15.66:0/3151912746 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556722387c10 con 0x55672242ec30 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10777fe640 10 client.16815 handle_command_reply: tid=0 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 client.16815 shutdown 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 msgr2=0x5567224179c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x5567224179c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x556722418bd0 tx=0x7f108800fea0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 shutdown_connections 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55672242ec30 0x5567224971f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55672241ea00 0x5567224179c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556722415e80 0x556722427260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 --2- 172.21.15.66:0/3151912746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567224154d0 0x556722426d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10777fe640 0 client.16815 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:02.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 >> 172.21.15.66:0/3151912746 conn(0x55672240fba0 msgr2=0x55672240e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 shutdown_connections 2024-04-21T09:12:02.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 1 -- 172.21.15.66:0/3151912746 wait complete. 2024-04-21T09:12:02.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f10a11f6740 20 client.16815 trim_cache size 0 max 16384 2024-04-21T09:12:02.968 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:12:02.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f10800a3d20 msgr2=0x7f10800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f109e52b640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f10800a3d20 0x7f10800a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10900c6d20 tx=0x7f10900ba240 comp rx=0 tx=0).stop 2024-04-21T09:12:02.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f109819a710 msgr2=0x7f109819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:02.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.961+0000 7f109e52b640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f109819a710 0x7f109819ab90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1090005e30 tx=0x7f1090032420 comp rx=0 tx=0).stop 2024-04-21T09:12:02.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 shutdown_connections 2024-04-21T09:12:02.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f10800a3d20 0x7f10800a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f109819a710 0x7f109819ab90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098107270 0x7f10981a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 --2- 172.21.15.66:0/2677935630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f109806a840 0x7f10981a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:02.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 >> 172.21.15.66:0/2677935630 conn(0x7f1098077eb0 msgr2=0x7f109810d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:02.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 shutdown_connections 2024-04-21T09:12:02.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:02.962+0000 7f109e52b640 1 -- 172.21.15.66:0/2677935630 wait complete. 2024-04-21T09:12:03.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:03 smithi186 ceph-mon[31505]: pgmap v118: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:03.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:03 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/936266272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:12:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:03 smithi083 ceph-mon[31413]: pgmap v118: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:03.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:03 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/936266272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:12:03.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:03 smithi066 ceph-mon[27953]: pgmap v118: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:03.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:03 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/936266272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:12:03.974 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:12:03.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:12:03.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:12:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 -- 172.21.15.66:0/3672721935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101a40 msgr2=0x7f5518105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 --2- 172.21.15.66:0/3672721935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101a40 0x7f5518105a80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f550002bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 -- 172.21.15.66:0/3672721935 shutdown_connections 2024-04-21T09:12:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 --2- 172.21.15.66:0/3672721935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5518106160 0x7f551810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 --2- 172.21.15.66:0/3672721935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101a40 0x7f5518105a80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 --2- 172.21.15.66:0/3672721935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5518101090 0x7f5518101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 -- 172.21.15.66:0/3672721935 >> 172.21.15.66:0/3672721935 conn(0x7f5518077f50 msgr2=0x7f5518078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 -- 172.21.15.66:0/3672721935 shutdown_connections 2024-04-21T09:12:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.093+0000 7f551d49a640 1 -- 172.21.15.66:0/3672721935 wait complete. 2024-04-21T09:12:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 Processor -- start 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 -- start start 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101090 0x7f5518198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5518106160 0x7f5518198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f551819cfe0 0x7f551819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55181121e0 con 0x7f5518101090 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5518112060 con 0x7f551819cfe0 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f551d49a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5518112360 con 0x7f5518106160 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f5516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101090 0x7f5518198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f5516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101090 0x7f5518198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52922/0 (socket says 172.21.15.66:52922) 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f5516ffd640 1 -- 172.21.15.66:0/1587427710 learned_addr learned my addr 172.21.15.66:0/1587427710 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f55167fc640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5518106160 0x7f5518198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.094+0000 7f55177fe640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f551819cfe0 0x7f551819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f5516ffd640 1 -- 172.21.15.66:0/1587427710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5518106160 msgr2=0x7f5518198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f5516ffd640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5518106160 0x7f5518198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f5516ffd640 1 -- 172.21.15.66:0/1587427710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f551819cfe0 msgr2=0x7f551819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f5516ffd640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f551819cfe0 0x7f551819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f5516ffd640 1 -- 172.21.15.66:0/1587427710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551819dad0 con 0x7f5518101090 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f5516ffd640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101090 0x7f5518198720 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f550800db80 tx=0x7f550800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f550ffff640 1 -- 172.21.15.66:0/1587427710 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508017070 con 0x7f5518101090 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f550ffff640 1 -- 172.21.15.66:0/1587427710 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5508003b70 con 0x7f5518101090 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551819ddc0 con 0x7f5518101090 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f550ffff640 1 -- 172.21.15.66:0/1587427710 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55080106c0 con 0x7f5518101090 2024-04-21T09:12:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.095+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55181a5580 con 0x7f5518101090 2024-04-21T09:12:04.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.097+0000 7f552016d740 1 Processor -- start 2024-04-21T09:12:04.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f552016d740 1 -- start start 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f550ffff640 1 -- 172.21.15.66:0/1587427710 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5508004020 con 0x7f5518101090 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f552016d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de8e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f552016d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8e9170 0x5555de8f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f552016d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8f2270 0x5555de8f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f552016d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5555de84cf00 con 0x5555de8f2270 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f552016d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5555de7ea590 con 0x5555de8e9170 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f552016d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5555de848ec0 con 0x5555de8e87c0 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f55167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8f2270 0x5555de8f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f55167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8f2270 0x5555de8f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52928/0 (socket says 172.21.15.66:52928) 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f55167fc640 1 -- 172.21.15.66:0/3196301590 learned_addr learned my addr 172.21.15.66:0/3196301590 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.098+0000 7f5516ffd640 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8e9170 0x5555de8f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f550ffff640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55040a3d20 0x7f55040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f55167fc640 1 -- 172.21.15.66:0/3196301590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 msgr2=0x5555de8e8ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f55167fc640 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de8e8ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f55167fc640 1 -- 172.21.15.66:0/3196301590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8e9170 msgr2=0x5555de8f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f55167fc640 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8e9170 0x5555de8f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f55167fc640 1 -- 172.21.15.66:0/3196301590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555de6277e0 con 0x5555de8f2270 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f55177fe640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55040a3d20 0x7f55040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f550ffff640 1 -- 172.21.15.66:0/1587427710 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f55080d0af0 con 0x7f5518101090 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f55167fc640 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8f2270 0x5555de8f26d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f550002c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493439ace29d84b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f550d7fa640 1 -- 172.21.15.66:0/3196301590 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500045020 con 0x5555de8f2270 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f550d7fa640 1 -- 172.21.15.66:0/3196301590 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550002c7e0 con 0x5555de8f2270 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.099+0000 7f550d7fa640 1 -- 172.21.15.66:0/3196301590 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55000046c0 con 0x5555de8f2270 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 -- 172.21.15.66:0/3196301590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8f2270 msgr2=0x5555de8f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f55177fe640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55040a3d20 0x7f55040a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5510004500 tx=0x7f551000b2e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8f2270 0x5555de8f26d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f550002c610 comp rx=0 tx=0).stop 2024-04-21T09:12:04.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 -- 172.21.15.66:0/3196301590 shutdown_connections 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8f2270 0x5555de8f26d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8e9170 0x5555de8f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 --2- 172.21.15.66:0/3196301590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de8e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 -- 172.21.15.66:0/3196301590 >> 172.21.15.66:0/3196301590 conn(0x5555de8e2fb0 msgr2=0x5555de8e3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 -- 172.21.15.66:0/3196301590 shutdown_connections 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 -- 172.21.15.66:0/3196301590 wait complete. 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.100+0000 7f552016d740 1 -- start start 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 1 Processor -- start 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e87c0 0x5555de8f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e9170 0x5555de8f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5555de8f9150 con 0x5555de8e87c0 2024-04-21T09:12:04.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5555de806580 con 0x5555de8f2270 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5555de8f92d0 con 0x5555de8e9170 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f552016d740 10 client.0 _ll_register_callbacks cb 0x5555de80c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e87c0 0x5555de8f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e87c0 0x5555de8f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52940/0 (socket says 172.21.15.66:52940) 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 -- 172.21.15.66:0/3125039120 learned_addr learned my addr 172.21.15.66:0/3125039120 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f55177fe640 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 -- 172.21.15.66:0/3125039120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e9170 msgr2=0x5555de8f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e9170 0x5555de8f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 -- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 msgr2=0x5555de8f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 -- 172.21.15.66:0/3125039120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555de687f70 con 0x5555de8e87c0 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f5516ffd640 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e87c0 0x5555de8f5df0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f550809c420 tx=0x7f550800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3575ae22acbef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.101+0000 7f54fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f54fe7fc640 1 -- 172.21.15.66:0/3125039120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550809c260 con 0x5555de8e87c0 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f54fe7fc640 1 -- 172.21.15.66:0/3125039120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f5508003d20 con 0x5555de8e87c0 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550002c7e0 con 0x5555de8e87c0 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f54fe7fc640 1 -- 172.21.15.66:0/3125039120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f550809c080 con 0x5555de8e87c0 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 10 client.16827 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555de51f170 con 0x5555de8e87c0 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f54fe7fc640 1 -- 172.21.15.66:0/3125039120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55080d6060 con 0x5555de8e87c0 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f54fe7fc640 1 -- 172.21.15.66:0/3125039120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5508023060 con 0x5555de8e87c0 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 10 client.16827 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 20 client.16827 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de906790 0x5555de908bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 4 client.16827 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.102+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f550002c7e0 con 0x5555de906790 2024-04-21T09:12:04.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.103+0000 7f55167fc640 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de906790 0x5555de908bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.103+0000 7f55167fc640 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de906790 0x5555de908bc0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76fb017ad68fee2 server_cookie=9bde62c95ad8162c in_seq=0 out_seq=0 2024-04-21T09:12:04.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.103+0000 7f54fe7fc640 10 client.16827 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f54fe7fc640 1 -- 172.21.15.66:0/3125039120 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f550002c7e0 con 0x5555de906790 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f54fe7fc640 10 client.16827 handle_command_reply: tid=0 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 client.16827 shutdown 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e87c0 msgr2=0x5555de8f5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e87c0 0x5555de8f5df0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f550809c420 tx=0x7f550800de80 comp rx=0 tx=0).stop 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 shutdown_connections 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de906790 0x5555de908bc0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e9170 0x5555de8f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f54fe7fc640 0 client.16827 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 --2- 172.21.15.66:0/3125039120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e87c0 0x5555de8f5df0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 >> 172.21.15.66:0/3125039120 conn(0x5555de8e2fb0 msgr2=0x5555de8e4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 shutdown_connections 2024-04-21T09:12:04.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 1 -- 172.21.15.66:0/3125039120 wait complete. 2024-04-21T09:12:04.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.104+0000 7f552016d740 20 client.16827 trim_cache size 0 max 16384 2024-04-21T09:12:04.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 Processor -- start 2024-04-21T09:12:04.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 -- start start 2024-04-21T09:12:04.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de8f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de902050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55181af870 con 0x5555de8e9170 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55181ad140 con 0x5555de8f2270 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f552016d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f55181aaf70 con 0x5555de8e87c0 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de902050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de902050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52954/0 (socket says 172.21.15.66:52954) 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 -- 172.21.15.66:0/1874467920 learned_addr learned my addr 172.21.15.66:0/1874467920 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55177fe640 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 -- 172.21.15.66:0/1874467920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 msgr2=0x5555de8f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de8f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 -- 172.21.15.66:0/1874467920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 msgr2=0x5555de8f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.122+0000 7f55167fc640 1 -- 172.21.15.66:0/1874467920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555de8e83f0 con 0x5555de8e9170 2024-04-21T09:12:04.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f55167fc640 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de902050 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f550002c780 tx=0x7f5500005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1964b85de34e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f550cff9640 1 -- 172.21.15.66:0/1874467920 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500045040 con 0x5555de8e9170 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f550cff9640 1 -- 172.21.15.66:0/1874467920 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5500052070 con 0x5555de8e9170 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f550cff9640 1 -- 172.21.15.66:0/1874467920 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500003850 con 0x5555de8e9170 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f552016d740 1 -- 172.21.15.66:0/1874467920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 msgr2=0x5555de902050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f552016d740 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de902050 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f550002c780 tx=0x7f5500005380 comp rx=0 tx=0).stop 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f552016d740 1 -- 172.21.15.66:0/1874467920 shutdown_connections 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f552016d740 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de8f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f552016d740 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de902050 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f552016d740 1 --2- 172.21.15.66:0/1874467920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de8f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.123+0000 7f552016d740 1 -- 172.21.15.66:0/1874467920 >> 172.21.15.66:0/1874467920 conn(0x5555de8e2fb0 msgr2=0x5555de8e4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 -- 172.21.15.66:0/1874467920 shutdown_connections 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 -- 172.21.15.66:0/1874467920 wait complete. 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 -- start start 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 Processor -- start 2024-04-21T09:12:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de90dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de90e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de96bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55181ab600 con 0x5555de8e9170 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55181aca70 con 0x5555de8f2270 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f55181abcd0 con 0x5555de8e87c0 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f552016d740 10 client.0 _ll_register_callbacks cb 0x5555de80c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de90e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de90e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52966/0 (socket says 172.21.15.66:52966) 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 -- 172.21.15.66:0/2795538247 learned_addr learned my addr 172.21.15.66:0/2795538247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 -- 172.21.15.66:0/2795538247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 msgr2=0x5555de90dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de90dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 -- 172.21.15.66:0/2795538247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 msgr2=0x5555de96bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de96bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.124+0000 7f55167fc640 1 -- 172.21.15.66:0/2795538247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555de96c240 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f55167fc640 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de90e260 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5500005ec0 tx=0x7f5500000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2c236958e40a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f54fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f54fe7fc640 1 -- 172.21.15.66:0/2795538247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500045040 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f54fe7fc640 1 -- 172.21.15.66:0/2795538247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f550002c7e0 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5500052070 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f54fe7fc640 1 -- 172.21.15.66:0/2795538247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f550002ca20 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 10 client.16833 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555de96d380 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f54fe7fc640 1 -- 172.21.15.66:0/2795538247 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5500052070 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f54fe7fc640 1 -- 172.21.15.66:0/2795538247 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5500041480 con 0x5555de8e9170 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 10 client.16833 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 20 client.16833 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de96d6b0 0x5555de96fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 4 client.16833 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:12:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.125+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555de85a920 con 0x5555de96d6b0 2024-04-21T09:12:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.126+0000 7f5516ffd640 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de96d6b0 0x5555de96fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.126+0000 7f54fe7fc640 10 client.16833 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.126+0000 7f5516ffd640 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de96d6b0 0x5555de96fac0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e2ddb94b0b10c2 server_cookie=8034f047a6445a38 in_seq=0 out_seq=0 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f54fe7fc640 1 -- 172.21.15.66:0/2795538247 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555de85a920 con 0x5555de96d6b0 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f54fe7fc640 10 client.16833 handle_command_reply: tid=0 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 client.16833 shutdown 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 msgr2=0x5555de90e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de90e260 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5500005ec0 tx=0x7f5500000bc0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 shutdown_connections 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5555de96d6b0 0x5555de96fac0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555de8f2270 0x5555de96bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5555de8e9170 0x5555de90e260 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f54fe7fc640 0 client.16833 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 --2- 172.21.15.66:0/2795538247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555de8e87c0 0x5555de90dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.127+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 >> 172.21.15.66:0/2795538247 conn(0x5555de8e2fb0 msgr2=0x5555de8e2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 shutdown_connections 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f552016d740 1 -- 172.21.15.66:0/2795538247 wait complete. 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f552016d740 20 client.16833 trim_cache size 0 max 16384 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "cf1dfe25-5f21-443f-bd4c-0367d0c4aaf6", 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:12:04.133 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:12:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55040a3d20 msgr2=0x7f55040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55040a3d20 0x7f55040a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5510004500 tx=0x7f551000b2e0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101090 msgr2=0x7f5518198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101090 0x7f5518198720 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f550800db80 tx=0x7f550800b630 comp rx=0 tx=0).stop 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 shutdown_connections 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f55040a3d20 0x7f55040a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f551819cfe0 0x7f551819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5518106160 0x7f5518198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 --2- 172.21.15.66:0/1587427710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5518101090 0x7f5518198720 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 >> 172.21.15.66:0/1587427710 conn(0x7f5518077f50 msgr2=0x7f55180ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 shutdown_connections 2024-04-21T09:12:04.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.128+0000 7f551d49a640 1 -- 172.21.15.66:0/1587427710 wait complete. 2024-04-21T09:12:04.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:12:04.140 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 -- 172.21.15.66:0/413573524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc06b290 msgr2=0x7f2ffc106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 --2- 172.21.15.66:0/413573524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc06b290 0x7f2ffc106c30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009900 tx=0x7f2fec02bc50 comp rx=0 tx=0).stop 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 -- 172.21.15.66:0/413573524 shutdown_connections 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 --2- 172.21.15.66:0/413573524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ffc107310 0x7f2ffc10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 --2- 172.21.15.66:0/413573524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc06b290 0x7f2ffc106c30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 --2- 172.21.15.66:0/413573524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc06a8e0 0x7f2ffc06acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 -- 172.21.15.66:0/413573524 >> 172.21.15.66:0/413573524 conn(0x7f2ffc0771d0 msgr2=0x7f2ffc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 -- 172.21.15.66:0/413573524 shutdown_connections 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.244+0000 7f30041ce640 1 -- 172.21.15.66:0/413573524 wait complete. 2024-04-21T09:12:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 Processor -- start 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 -- start start 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ffc06a8e0 0x7f2ffc19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc107310 0x7f2ffc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc198590 0x7f2ffc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc112250 con 0x7f2ffc107310 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1120d0 con 0x7f2ffc198590 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f30041ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1123d0 con 0x7f2ffc06a8e0 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc107310 0x7f2ffc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc107310 0x7f2ffc19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52986/0 (socket says 172.21.15.66:52986) 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 -- 172.21.15.66:0/1329213641 learned_addr learned my addr 172.21.15.66:0/1329213641 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f300276b640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc198590 0x7f2ffc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ffc06a8e0 0x7f2ffc19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 -- 172.21.15.66:0/1329213641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ffc06a8e0 msgr2=0x7f2ffc19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ffc06a8e0 0x7f2ffc19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 -- 172.21.15.66:0/1329213641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc198590 msgr2=0x7f2ffc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc198590 0x7f2ffc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.245+0000 7f3001769640 1 -- 172.21.15.66:0/1329213641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc1990b0 con 0x7f2ffc107310 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.246+0000 7f3001769640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc107310 0x7f2ffc19e920 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2fec000c00 tx=0x7f2fec0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.246+0000 7f2ff2ffd640 1 -- 172.21.15.66:0/1329213641 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec043020 con 0x7f2ffc107310 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.246+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc199340 con 0x7f2ffc107310 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.246+0000 7f2ff2ffd640 1 -- 172.21.15.66:0/1329213641 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec02ca90 con 0x7f2ffc107310 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.246+0000 7f2ff2ffd640 1 -- 172.21.15.66:0/1329213641 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec003ac0 con 0x7f2ffc107310 2024-04-21T09:12:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.246+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a5660 con 0x7f2ffc107310 2024-04-21T09:12:04.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.248+0000 7f2ff2ffd640 1 -- 172.21.15.66:0/1329213641 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2fec046020 con 0x7f2ffc107310 2024-04-21T09:12:04.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.248+0000 7f3006dbf740 1 Processor -- start 2024-04-21T09:12:04.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.248+0000 7f3006dbf740 1 -- start start 2024-04-21T09:12:04.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.248+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34bc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.248+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bc910 0x55f5d34c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34c9e90 0x55f5d34cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f3006dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5d34211e0 con 0x55f5d34bbf60 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f3006dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5d33be390 con 0x55f5d34bc910 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f3006dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f5d341d1a0 con 0x55f5d34c9e90 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f2ff2ffd640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2fe80a3d20 0x7f2fe80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34bc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34bc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53000/0 (socket says 172.21.15.66:53000) 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 -- 172.21.15.66:0/1715598313 learned_addr learned my addr 172.21.15.66:0/1715598313 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bc910 0x55f5d34c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2fe80a3d20 0x7f2fe80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f2ff2ffd640 1 -- 172.21.15.66:0/1329213641 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2fec0f2b80 con 0x7f2ffc107310 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f3001769640 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34c9e90 0x55f5d34cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 -- 172.21.15.66:0/1715598313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34c9e90 msgr2=0x55f5d34cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34c9e90 0x55f5d34cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 -- 172.21.15.66:0/1715598313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bc910 msgr2=0x55f5d34c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bc910 0x55f5d34c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 -- 172.21.15.66:0/1715598313 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d31fd7d0 con 0x55f5d34bbf60 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bc910 0x55f5d34c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34bc340 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400aa30 tx=0x7f2ff400aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892dcddd1797f123 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f2fe7fff640 1 -- 172.21.15.66:0/1715598313 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff401d030 con 0x55f5d34bbf60 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f2fe7fff640 1 -- 172.21.15.66:0/1715598313 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff4015d20 con 0x55f5d34bbf60 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f2fe7fff640 1 -- 172.21.15.66:0/1715598313 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4019e50 con 0x55f5d34bbf60 2024-04-21T09:12:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.249+0000 7f300276b640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2fe80a3d20 0x7f2fe80a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4020000 tx=0x7f2ff401e000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 -- 172.21.15.66:0/1715598313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 msgr2=0x55f5d34bc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34bc340 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400aa30 tx=0x7f2ff400aef0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 -- 172.21.15.66:0/1715598313 shutdown_connections 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34c9e90 0x55f5d34cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bc910 0x55f5d34c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1715598313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34bc340 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 -- 172.21.15.66:0/1715598313 >> 172.21.15.66:0/1715598313 conn(0x55f5d34b6cb0 msgr2=0x55f5d34b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 -- 172.21.15.66:0/1715598313 shutdown_connections 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.250+0000 7f3006dbf740 1 -- 172.21.15.66:0/1715598313 wait complete. 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 -- start start 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 Processor -- start 2024-04-21T09:12:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bbf60 0x55f5d34c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34c9e90 0x55f5d34c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5d34ccc50 con 0x55f5d34c9e90 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5d33da2f0 con 0x55f5d34bbf60 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f5d34ccdd0 con 0x55f5d34bc910 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3006dbf740 10 client.0 _ll_register_callbacks cb 0x55f5d33e7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f300276b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34c9e90 0x55f5d34c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f300276b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34c9e90 0x55f5d34c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53010/0 (socket says 172.21.15.66:53010) 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f3001f6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bbf60 0x55f5d34c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.251+0000 7f300276b640 1 -- 172.21.15.66:0/1858751014 learned_addr learned my addr 172.21.15.66:0/1858751014 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f300276b640 1 -- 172.21.15.66:0/1858751014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 msgr2=0x55f5d34c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f300276b640 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f300276b640 1 -- 172.21.15.66:0/1858751014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bbf60 msgr2=0x55f5d34c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f300276b640 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bbf60 0x55f5d34c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f300276b640 1 -- 172.21.15.66:0/1858751014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d31333f0 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f300276b640 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34c9e90 0x55f5d34c7930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400bfd0 tx=0x7f2ff4019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3988cb3821896626 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f2fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f2fe57fa640 1 -- 172.21.15.66:0/1858751014 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff402f060 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f2fe57fa640 1 -- 172.21.15.66:0/1858751014 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2ff4003a10 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff4015d20 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f2fe57fa640 1 -- 172.21.15.66:0/1858751014 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ff4036050 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f3006dbf740 10 client.16845 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5d30f30f0 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f2fe57fa640 1 -- 172.21.15.66:0/1858751014 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff4015d20 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.252+0000 7f2fe57fa640 1 -- 172.21.15.66:0/1858751014 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ff4004050 con 0x55f5d34c9e90 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3006dbf740 10 client.16845 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3006dbf740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3006dbf740 20 client.16845 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d34d9e70 0x55f5d34dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3006dbf740 4 client.16845 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5d342ec00 con 0x55f5d34d9e70 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d34d9e70 0x55f5d34dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d34d9e70 0x55f5d34dc260 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd17f125298e89 server_cookie=44cec099b053860c in_seq=0 out_seq=0 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.253+0000 7f2fe57fa640 10 client.16845 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f2fe57fa640 1 -- 172.21.15.66:0/1858751014 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5d342ec00 con 0x55f5d34d9e70 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f2fe57fa640 10 client.16845 handle_command_reply: tid=0 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 client.16845 shutdown 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34c9e90 msgr2=0x55f5d34c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34c9e90 0x55f5d34c7930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400bfd0 tx=0x7f2ff4019640 comp rx=0 tx=0).stop 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 shutdown_connections 2024-04-21T09:12:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d34d9e70 0x55f5d34dc260 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34c9e90 0x55f5d34c7930 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1858751014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34bbf60 0x55f5d34c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f2fe57fa640 0 client.16845 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 >> 172.21.15.66:0/1858751014 conn(0x55f5d34b6cb0 msgr2=0x55f5d34b8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 shutdown_connections 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 1 -- 172.21.15.66:0/1858751014 wait complete. 2024-04-21T09:12:04.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.254+0000 7f3006dbf740 20 client.16845 trim_cache size 0 max 16384 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 Processor -- start 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 -- start start 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1afa80 con 0x55f5d34bbf60 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ad350 con 0x55f5d34c9e90 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3006dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ab180 con 0x55f5d34bc910 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53018/0 (socket says 172.21.15.66:53018) 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 -- 172.21.15.66:0/1174670884 learned_addr learned my addr 172.21.15.66:0/1174670884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001769640 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f300276b640 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 -- 172.21.15.66:0/1174670884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 msgr2=0x55f5d34d5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34d5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 -- 172.21.15.66:0/1174670884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 msgr2=0x55f5d34c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 -- 172.21.15.66:0/1174670884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d34bb670 con 0x55f5d34bbf60 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f300276b640 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f3001f6a640 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c95e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55f5d34dc940 tx=0x7f2ff8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d076f58f236423 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.272+0000 7f2fe77fe640 1 -- 172.21.15.66:0/1174670884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8004230 con 0x55f5d34bbf60 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f2fe77fe640 1 -- 172.21.15.66:0/1174670884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff80043d0 con 0x55f5d34bbf60 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f2fe77fe640 1 -- 172.21.15.66:0/1174670884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8003b50 con 0x55f5d34bbf60 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 -- 172.21.15.66:0/1174670884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 msgr2=0x55f5d34c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c95e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55f5d34dc940 tx=0x7f2ff8005f90 comp rx=0 tx=0).stop 2024-04-21T09:12:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 -- 172.21.15.66:0/1174670884 shutdown_connections 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34d5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 --2- 172.21.15.66:0/1174670884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c95e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 -- 172.21.15.66:0/1174670884 >> 172.21.15.66:0/1174670884 conn(0x55f5d34b6cb0 msgr2=0x55f5d34b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 -- 172.21.15.66:0/1174670884 shutdown_connections 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 -- 172.21.15.66:0/1174670884 wait complete. 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 -- start start 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.273+0000 7f3006dbf740 1 Processor -- start 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3006dbf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3006dbf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ab810 con 0x55f5d34bbf60 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3006dbf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1acc80 con 0x55f5d34c9e90 2024-04-21T09:12:04.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3006dbf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1abee0 con 0x55f5d34bc910 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3006dbf740 10 client.0 _ll_register_callbacks cb 0x7f2ff8009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53028/0 (socket says 172.21.15.66:53028) 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 -- 172.21.15.66:0/3889558054 learned_addr learned my addr 172.21.15.66:0/3889558054 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f300276b640 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 -- 172.21.15.66:0/3889558054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 msgr2=0x55f5d34c41a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34c41a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 -- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 msgr2=0x55f5d34c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.274+0000 7f3001f6a640 1 -- 172.21.15.66:0/3889558054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d353f340 con 0x55f5d34bbf60 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3001f6a640 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c3c60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009dc0 tx=0x7f2ff80037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f33023d7ee5761d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f2fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f2fe57fa640 1 -- 172.21.15.66:0/3889558054 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8005600 con 0x55f5d34bbf60 2024-04-21T09:12:04.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f2fe57fa640 1 -- 172.21.15.66:0/3889558054 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2ff8021160 con 0x55f5d34bbf60 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff80043d0 con 0x55f5d34bbf60 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f2fe57fa640 1 -- 172.21.15.66:0/3889558054 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ff803b050 con 0x55f5d34bbf60 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 10 client.16851 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5d35405a0 con 0x55f5d34bbf60 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f2fe57fa640 1 -- 172.21.15.66:0/3889558054 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff80043d0 con 0x55f5d34bbf60 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f2fe57fa640 1 -- 172.21.15.66:0/3889558054 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ff80038d0 con 0x55f5d34bbf60 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 10 client.16851 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 20 client.16851 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d3540c90 0x55f5d35430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 4 client.16851 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5d3428a70 con 0x55f5d3540c90 2024-04-21T09:12:04.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.275+0000 7f3001769640 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d3540c90 0x55f5d35430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f2fe57fa640 10 client.16851 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3001769640 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d3540c90 0x55f5d35430b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc76303a350188b server_cookie=afd00ac0c7a6a052 in_seq=0 out_seq=0 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f2fe57fa640 1 -- 172.21.15.66:0/3889558054 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f5d3428a70 con 0x55f5d3540c90 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f2fe57fa640 10 client.16851 handle_command_reply: tid=0 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 client.16851 shutdown 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 msgr2=0x55f5d34c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c3c60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009dc0 tx=0x7f2ff80037d0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 shutdown_connections 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f5d3540c90 0x55f5d35430b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34c9e90 0x55f5d34c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5d34bc910 0x55f5d34c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f2fe57fa640 0 client.16851 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 --2- 172.21.15.66:0/3889558054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d34bbf60 0x55f5d34c3c60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 >> 172.21.15.66:0/3889558054 conn(0x55f5d34b6cb0 msgr2=0x55f5d34b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 shutdown_connections 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 1 -- 172.21.15.66:0/3889558054 wait complete. 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.276+0000 7f3006dbf740 20 client.16851 trim_cache size 0 max 16384 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stdout: "cf1dfe25-5f21-443f-bd4c-0367d0c4aaf6": { 2024-04-21T09:12:04.284 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:12:04.285 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "cf1dfe25-5f21-443f-bd4c-0367d0c4aaf6", 2024-04-21T09:12:04.285 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:12:04.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:12:04.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:12:04.285 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2fe80a3d20 msgr2=0x7f2fe80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2fe80a3d20 0x7f2fe80a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4020000 tx=0x7f2ff401e000 comp rx=0 tx=0).stop 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc107310 msgr2=0x7f2ffc19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc107310 0x7f2ffc19e920 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2fec000c00 tx=0x7f2fec0045e0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 shutdown_connections 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f2fe80a3d20 0x7f2fe80a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc198590 0x7f2ffc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc107310 0x7f2ffc19e920 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 --2- 172.21.15.66:0/1329213641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ffc06a8e0 0x7f2ffc19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 >> 172.21.15.66:0/1329213641 conn(0x7f2ffc0771d0 msgr2=0x7f2ffc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 shutdown_connections 2024-04-21T09:12:04.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:04.277+0000 7f30041ce640 1 -- 172.21.15.66:0/1329213641 wait complete. 2024-04-21T09:12:04.288 INFO:tasks.cephfs.filesystem:scrub status for tag:cf1dfe25-5f21-443f-bd4c-0367d0c4aaf6 - {'path': '/', 'tag': 'cf1dfe25-5f21-443f-bd4c-0367d0c4aaf6', 'options': 'recursive,force'} 2024-04-21T09:12:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:05 smithi186 ceph-mon[31505]: pgmap v119: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:12:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:05 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:12:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:05 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:05 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:12:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:05 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:05 smithi083 ceph-mon[31413]: pgmap v119: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:12:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:05 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:12:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:05 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:05 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:12:05.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:05 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:05.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:05 smithi066 ceph-mon[27953]: pgmap v119: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:12:05.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:05 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:12:05.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:05 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:05.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:05 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:12:05.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:05 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:06.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:06 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:12:06.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:06 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:12:06.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:06 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:12:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:07 smithi186 ceph-mon[31505]: pgmap v120: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:12:07.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:07 smithi083 ceph-mon[31413]: pgmap v120: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:12:07.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:07 smithi066 ceph-mon[27953]: pgmap v120: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:12:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:09 smithi186 ceph-mon[31505]: pgmap v121: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:12:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:09 smithi083 ceph-mon[31413]: pgmap v121: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:12:09.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:09 smithi066 ceph-mon[27953]: pgmap v121: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:12:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:11 smithi186 ceph-mon[31505]: pgmap v122: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:12:11.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:11 smithi083 ceph-mon[31413]: pgmap v122: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:12:11.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:11 smithi066 ceph-mon[27953]: pgmap v122: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:12:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:13 smithi186 ceph-mon[31505]: pgmap v123: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:12:13.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:13 smithi083 ceph-mon[31413]: pgmap v123: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:12:13.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:13 smithi066 ceph-mon[27953]: pgmap v123: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-21T09:12:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:15 smithi186 ceph-mon[31505]: pgmap v124: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:12:15.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:15 smithi083 ceph-mon[31413]: pgmap v124: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:12:15.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:15 smithi066 ceph-mon[27953]: pgmap v124: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:12:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:17 smithi186 ceph-mon[31505]: pgmap v125: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T09:12:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:17 smithi083 ceph-mon[31413]: pgmap v125: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T09:12:17.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:17 smithi066 ceph-mon[27953]: pgmap v125: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T09:12:19.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:19 smithi186 ceph-mon[31505]: pgmap v126: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:12:19.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:19 smithi083 ceph-mon[31413]: pgmap v126: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:12:19.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:19 smithi066 ceph-mon[27953]: pgmap v126: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:12:20.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:20 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:12:20.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:20 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:12:20.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:20 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:12:21.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:21 smithi083 ceph-mon[31413]: pgmap v127: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:21.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:21 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:12:21.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:21 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:12:21.761 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:21 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:12:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:21 smithi186 ceph-mon[31505]: pgmap v127: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:21 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:12:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:21 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:12:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:21 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:12:21.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:21 smithi066 ceph-mon[27953]: pgmap v127: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:21.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:21 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:12:21.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:21 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:12:21.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:21 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:12:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:23 smithi186 ceph-mon[31505]: pgmap v128: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:23 smithi083 ceph-mon[31413]: pgmap v128: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:23.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:23 smithi066 ceph-mon[27953]: pgmap v128: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:25.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:25 smithi186 ceph-mon[31505]: pgmap v129: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:25.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:25 smithi083 ceph-mon[31413]: pgmap v129: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:25.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:25 smithi066 ceph-mon[27953]: pgmap v129: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:27 smithi186 ceph-mon[31505]: pgmap v130: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:27.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:27 smithi083 ceph-mon[31413]: pgmap v130: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:27.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:27 smithi066 ceph-mon[27953]: pgmap v130: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:29.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:29 smithi186 ceph-mon[31505]: pgmap v131: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:29.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:29 smithi083 ceph-mon[31413]: pgmap v131: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:29.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:29 smithi066 ceph-mon[27953]: pgmap v131: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:31.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:31 smithi186 ceph-mon[31505]: pgmap v132: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:31.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:31 smithi083 ceph-mon[31413]: pgmap v132: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:31.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:31 smithi066 ceph-mon[27953]: pgmap v132: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:33.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:33 smithi186 ceph-mon[31505]: pgmap v133: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:34.214 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:33 smithi066 ceph-mon[27953]: pgmap v133: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:34.215 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:33 smithi083 ceph-mon[31413]: pgmap v133: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:34.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:12:34.289 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:12:34.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.407+0000 7fc87d65c640 1 -- 172.21.15.66:0/2272868856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc878107270 msgr2=0x7fc87810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.407+0000 7fc87d65c640 1 --2- 172.21.15.66:0/2272868856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc878107270 0x7fc87810ee40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc870009ca0 tx=0x7fc87002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.407+0000 7fc87d65c640 1 -- 172.21.15.66:0/2272868856 shutdown_connections 2024-04-21T09:12:34.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.407+0000 7fc87d65c640 1 --2- 172.21.15.66:0/2272868856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc878107270 0x7fc87810ee40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.407+0000 7fc87d65c640 1 --2- 172.21.15.66:0/2272868856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc878102b50 0x7fc878106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.407+0000 7fc87d65c640 1 --2- 172.21.15.66:0/2272868856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8781021a0 0x7fc878102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.407+0000 7fc87d65c640 1 -- 172.21.15.66:0/2272868856 >> 172.21.15.66:0/2272868856 conn(0x7fc8780793e0 msgr2=0x7fc8780797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 -- 172.21.15.66:0/2272868856 shutdown_connections 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 -- 172.21.15.66:0/2272868856 wait complete. 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 Processor -- start 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 -- start start 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8781021a0 0x7fc878198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc878107270 0x7fc878198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87819cf90 0x7fc87819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781121e0 con 0x7fc8781021a0 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.408+0000 7fc87d65c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc878112060 con 0x7fc878107270 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc87d65c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc878112360 con 0x7fc87819cf90 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8781021a0 0x7fc878198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8781021a0 0x7fc878198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47812/0 (socket says 172.21.15.66:47812) 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 -- 172.21.15.66:0/1956049983 learned_addr learned my addr 172.21.15.66:0/1956049983 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc8777fe640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87819cf90 0x7fc87819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc878107270 0x7fc878198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 -- 172.21.15.66:0/1956049983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87819cf90 msgr2=0x7fc87819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87819cf90 0x7fc87819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 -- 172.21.15.66:0/1956049983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc878107270 msgr2=0x7fc878198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc878107270 0x7fc878198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 -- 172.21.15.66:0/1956049983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87819da80 con 0x7fc8781021a0 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc8777fe640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87819cf90 0x7fc87819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc878107270 0x7fc878198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8781021a0 0x7fc878198720 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc86800d770 tx=0x7fc86800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc85ffff640 1 -- 172.21.15.66:0/1956049983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868017070 con 0x7fc8781021a0 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87819dd70 con 0x7fc8781021a0 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc85ffff640 1 -- 172.21.15.66:0/1956049983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8680054d0 con 0x7fc8781021a0 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc85ffff640 1 -- 172.21.15.66:0/1956049983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868004c30 con 0x7fc8781021a0 2024-04-21T09:12:34.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.409+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8781a5560 con 0x7fc8781021a0 2024-04-21T09:12:34.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc85ffff640 1 -- 172.21.15.66:0/1956049983 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc868004e10 con 0x7fc8781021a0 2024-04-21T09:12:34.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 Processor -- start 2024-04-21T09:12:34.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 -- start start 2024-04-21T09:12:34.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5754540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5756e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5650b56b01e0 con 0x5650b574b580 2024-04-21T09:12:34.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5650b564d390 con 0x5650b5754a80 2024-04-21T09:12:34.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc88024d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5650b56ac1a0 con 0x5650b574bea0 2024-04-21T09:12:34.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc85ffff640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8600a3d20 0x7fc8600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.412+0000 7fc8777fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47826/0 (socket says 172.21.15.66:47826) 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 -- 172.21.15.66:0/165364944 learned_addr learned my addr 172.21.15.66:0/165364944 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc85ffff640 1 -- 172.21.15.66:0/1956049983 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc8680d06f0 con 0x7fc8781021a0 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8600a3d20 0x7fc8600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8767fc640 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5756e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc876ffd640 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5754540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 -- 172.21.15.66:0/165364944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 msgr2=0x5650b5754540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5754540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 -- 172.21.15.66:0/165364944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 msgr2=0x5650b5756e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5756e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 -- 172.21.15.66:0/165364944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b548c7d0 con 0x5650b574b580 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574b960 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc870005fe0 tx=0x7fc8700324c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370e084b184d8273 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc8777fe640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8600a3d20 0x7fc8600a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc870032460 tx=0x7fc870041000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc85d7fa640 1 -- 172.21.15.66:0/165364944 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870051070 con 0x5650b574b580 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc85d7fa640 1 -- 172.21.15.66:0/165364944 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc870048070 con 0x5650b574b580 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.413+0000 7fc85d7fa640 1 -- 172.21.15.66:0/165364944 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870039040 con 0x5650b574b580 2024-04-21T09:12:34.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 -- 172.21.15.66:0/165364944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 msgr2=0x5650b574b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574b960 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc870005fe0 tx=0x7fc8700324c0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 -- 172.21.15.66:0/165364944 shutdown_connections 2024-04-21T09:12:34.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5756e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5754540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 --2- 172.21.15.66:0/165364944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574b960 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 -- 172.21.15.66:0/165364944 >> 172.21.15.66:0/165364944 conn(0x5650b5745cb0 msgr2=0x5650b5746090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 -- 172.21.15.66:0/165364944 shutdown_connections 2024-04-21T09:12:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 -- 172.21.15.66:0/165364944 wait complete. 2024-04-21T09:12:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.414+0000 7fc88024d740 1 -- start start 2024-04-21T09:12:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 1 Processor -- start 2024-04-21T09:12:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b5751ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5752010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5752550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5650b5757810 con 0x5650b574b580 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5650b56692f0 con 0x5650b5754a80 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5650b5757990 con 0x5650b574bea0 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc88024d740 10 client.0 _ll_register_callbacks cb 0x5650b5676580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b5751ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b5751ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47834/0 (socket says 172.21.15.66:47834) 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 -- 172.21.15.66:0/2175093881 learned_addr learned my addr 172.21.15.66:0/2175093881 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc8767fc640 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5752010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc8777fe640 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5752550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 -- 172.21.15.66:0/2175093881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 msgr2=0x5650b5752010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5752010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 -- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 msgr2=0x5650b5752550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5752550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 -- 172.21.15.66:0/2175093881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b53c23f0 con 0x5650b574b580 2024-04-21T09:12:34.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc876ffd640 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b5751ad0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cdb30 tx=0x7fc86809c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624804704318456e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.415+0000 7fc85aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc85aff5640 1 -- 172.21.15.66:0/2175093881 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8680b5060 con 0x5650b574b580 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc85aff5640 1 -- 172.21.15.66:0/2175093881 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc8680cdd50 con 0x5650b574b580 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc870048070 con 0x5650b574b580 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc85aff5640 1 -- 172.21.15.66:0/2175093881 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc868020040 con 0x5650b574b580 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 10 client.16863 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650b53820f0 con 0x5650b574b580 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc85aff5640 1 -- 172.21.15.66:0/2175093881 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc870048070 con 0x5650b574b580 2024-04-21T09:12:34.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc85aff5640 1 -- 172.21.15.66:0/2175093881 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8680d0390 con 0x5650b574b580 2024-04-21T09:12:34.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 10 client.16863 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:34.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:34.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 20 client.16863 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:34.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57649e0 0x5650b5766e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 4 client.16863 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:34.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b56bdc00 con 0x5650b57649e0 2024-04-21T09:12:34.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.416+0000 7fc8767fc640 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57649e0 0x5650b5766e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.417+0000 7fc85aff5640 10 client.16863 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.417+0000 7fc8767fc640 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57649e0 0x5650b5766e50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570291787730162e server_cookie=6640cde7b81aac8c in_seq=0 out_seq=0 2024-04-21T09:12:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc85aff5640 1 -- 172.21.15.66:0/2175093881 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650b56bdc00 con 0x5650b57649e0 2024-04-21T09:12:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc85aff5640 10 client.16863 handle_command_reply: tid=0 2024-04-21T09:12:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 client.16863 shutdown 2024-04-21T09:12:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 msgr2=0x5650b5751ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b5751ad0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cdb30 tx=0x7fc86809c320 comp rx=0 tx=0).stop 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 shutdown_connections 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57649e0 0x5650b5766e50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b5752550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b5752010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 --2- 172.21.15.66:0/2175093881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b5751ad0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc85aff5640 0 client.16863 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 >> 172.21.15.66:0/2175093881 conn(0x5650b5745cb0 msgr2=0x5650b5747a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 shutdown_connections 2024-04-21T09:12:34.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 1 -- 172.21.15.66:0/2175093881 wait complete. 2024-04-21T09:12:34.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.418+0000 7fc88024d740 20 client.16863 trim_cache size 0 max 16384 2024-04-21T09:12:34.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 Processor -- start 2024-04-21T09:12:34.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 -- start start 2024-04-21T09:12:34.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b574b580 0x5650b5754200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574bea0 0x5650b57606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b5754a80 0x5650b5750c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781b1c10 con 0x5650b574bea0 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781af4e0 con 0x5650b574b580 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc88024d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781ad310 con 0x5650b5754a80 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574bea0 0x5650b57606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574bea0 0x5650b57606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47848/0 (socket says 172.21.15.66:47848) 2024-04-21T09:12:34.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 -- 172.21.15.66:0/1053629134 learned_addr learned my addr 172.21.15.66:0/1053629134 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b574b580 0x5650b5754200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 -- 172.21.15.66:0/1053629134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b5754a80 msgr2=0x5650b5750c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b5754a80 0x5650b5750c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 -- 172.21.15.66:0/1053629134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b574b580 msgr2=0x5650b5754200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b574b580 0x5650b5754200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 -- 172.21.15.66:0/1053629134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b574bcd0 con 0x5650b574bea0 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b574b580 0x5650b5754200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.436+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574bea0 0x5650b57606d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc878101820 tx=0x7fc86c009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0315b73716b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc85cff9640 1 -- 172.21.15.66:0/1053629134 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c014ac0 con 0x5650b574bea0 2024-04-21T09:12:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc85cff9640 1 -- 172.21.15.66:0/1053629134 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c014c60 con 0x5650b574bea0 2024-04-21T09:12:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc85cff9640 1 -- 172.21.15.66:0/1053629134 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0035f0 con 0x5650b574bea0 2024-04-21T09:12:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 -- 172.21.15.66:0/1053629134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574bea0 msgr2=0x5650b57606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574bea0 0x5650b57606d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc878101820 tx=0x7fc86c009450 comp rx=0 tx=0).stop 2024-04-21T09:12:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 -- 172.21.15.66:0/1053629134 shutdown_connections 2024-04-21T09:12:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b5754a80 0x5650b5750c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574bea0 0x5650b57606d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 --2- 172.21.15.66:0/1053629134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b574b580 0x5650b5754200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 -- 172.21.15.66:0/1053629134 >> 172.21.15.66:0/1053629134 conn(0x5650b5745cb0 msgr2=0x5650b5751a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 -- 172.21.15.66:0/1053629134 shutdown_connections 2024-04-21T09:12:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.437+0000 7fc88024d740 1 -- 172.21.15.66:0/1053629134 wait complete. 2024-04-21T09:12:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 -- start start 2024-04-21T09:12:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 Processor -- start 2024-04-21T09:12:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b574ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b574f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781ad9a0 con 0x5650b574b580 2024-04-21T09:12:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781aee10 con 0x5650b5754a80 2024-04-21T09:12:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc8781ae070 con 0x5650b574bea0 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc88024d740 10 client.0 _ll_register_callbacks cb 0x7fc868098640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47864/0 (socket says 172.21.15.66:47864) 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 -- 172.21.15.66:0/1533497936 learned_addr learned my addr 172.21.15.66:0/1533497936 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc8777fe640 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b574f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b574ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 -- 172.21.15.66:0/1533497936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 msgr2=0x5650b574ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b574ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 -- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 msgr2=0x5650b574f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b574f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 -- 172.21.15.66:0/1533497936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b57ca1e0 con 0x5650b574b580 2024-04-21T09:12:34.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc876ffd640 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574e9f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc86809a5d0 tx=0x7fc8680bb3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9240fec5092ad582 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.438+0000 7fc85aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:34.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc85aff5640 1 -- 172.21.15.66:0/1533497936 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86809c0f0 con 0x5650b574b580 2024-04-21T09:12:34.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc85aff5640 1 -- 172.21.15.66:0/1533497936 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc8680a02b0 con 0x5650b574b580 2024-04-21T09:12:34.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc86c014c60 con 0x5650b574b580 2024-04-21T09:12:34.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc85aff5640 1 -- 172.21.15.66:0/1533497936 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8680a04b0 con 0x5650b574b580 2024-04-21T09:12:34.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 10 client.16869 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:34.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650b57cb440 con 0x5650b574b580 2024-04-21T09:12:34.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc85aff5640 1 -- 172.21.15.66:0/1533497936 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c014c60 con 0x5650b574b580 2024-04-21T09:12:34.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc85aff5640 1 -- 172.21.15.66:0/1533497936 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8680d0390 con 0x5650b574b580 2024-04-21T09:12:34.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 10 client.16869 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 20 client.16869 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57cbb30 0x5650b57cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 4 client.16869 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650b56b7a70 con 0x5650b57cbb30 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.439+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57cbb30 0x5650b57cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc85aff5640 10 client.16869 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc8767fc640 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57cbb30 0x5650b57cdf50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c4194777a2b573 server_cookie=36d1188357fd1c0a in_seq=0 out_seq=0 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc85aff5640 1 -- 172.21.15.66:0/1533497936 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5650b56b7a70 con 0x5650b57cbb30 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc85aff5640 10 client.16869 handle_command_reply: tid=0 2024-04-21T09:12:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 client.16869 shutdown 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 msgr2=0x5650b574e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574e9f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc86809a5d0 tx=0x7fc8680bb3a0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 shutdown_connections 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5650b57cbb30 0x5650b57cdf50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b5754a80 0x5650b574f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650b574bea0 0x5650b574ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc85aff5640 0 client.16869 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 --2- 172.21.15.66:0/1533497936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650b574b580 0x5650b574e9f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 >> 172.21.15.66:0/1533497936 conn(0x5650b5745cb0 msgr2=0x5650b5747820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 shutdown_connections 2024-04-21T09:12:34.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 1 -- 172.21.15.66:0/1533497936 wait complete. 2024-04-21T09:12:34.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.440+0000 7fc88024d740 20 client.16869 trim_cache size 0 max 16384 2024-04-21T09:12:34.457 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:12:34.458 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:12:34.458 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:12:34.458 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8600a3d20 msgr2=0x7fc8600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8600a3d20 0x7fc8600a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc870032460 tx=0x7fc870041000 comp rx=0 tx=0).stop 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8781021a0 msgr2=0x7fc878198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8781021a0 0x7fc878198720 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc86800d770 tx=0x7fc86800dc30 comp rx=0 tx=0).stop 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 shutdown_connections 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8600a3d20 0x7fc8600a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87819cf90 0x7fc87819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc878107270 0x7fc878198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 --2- 172.21.15.66:0/1956049983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8781021a0 0x7fc878198720 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 >> 172.21.15.66:0/1956049983 conn(0x7fc8780793e0 msgr2=0x7fc8780ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 shutdown_connections 2024-04-21T09:12:34.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.441+0000 7fc87d65c640 1 -- 172.21.15.66:0/1956049983 wait complete. 2024-04-21T09:12:34.461 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:12:34.461 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:12:34.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.563+0000 7f30927ed640 1 -- 172.21.15.66:0/761656173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c101b60 msgr2=0x7f308c10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.563+0000 7f30927ed640 1 --2- 172.21.15.66:0/761656173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c101b60 0x7f308c10dc90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3084009cd0 tx=0x7f308402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.563+0000 7f30927ed640 1 -- 172.21.15.66:0/761656173 shutdown_connections 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 --2- 172.21.15.66:0/761656173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c101b60 0x7f308c10dc90 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 --2- 172.21.15.66:0/761656173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f308c1011a0 0x7f308c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 --2- 172.21.15.66:0/761656173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c105e30 0x7f308c106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 -- 172.21.15.66:0/761656173 >> 172.21.15.66:0/761656173 conn(0x7f308c077e30 msgr2=0x7f308c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 -- 172.21.15.66:0/761656173 shutdown_connections 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 -- 172.21.15.66:0/761656173 wait complete. 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 Processor -- start 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 -- start start 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c1011a0 0x7f308c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c101b60 0x7f308c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f308c105e30 0x7f308c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f308c112240 con 0x7f308c1011a0 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f308c1120c0 con 0x7f308c101b60 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.564+0000 7f30927ed640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f308c1123c0 con 0x7f308c105e30 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c1011a0 0x7f308c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c1011a0 0x7f308c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47894/0 (socket says 172.21.15.66:47894) 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 -- 172.21.15.66:0/3055926051 learned_addr learned my addr 172.21.15.66:0/3055926051 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f3090d8a640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f308c105e30 0x7f308c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308b7fe640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c101b60 0x7f308c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 -- 172.21.15.66:0/3055926051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f308c105e30 msgr2=0x7f308c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f308c105e30 0x7f308c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 -- 172.21.15.66:0/3055926051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c101b60 msgr2=0x7f308c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c101b60 0x7f308c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 -- 172.21.15.66:0/3055926051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c19d6c0 con 0x7f308c1011a0 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308b7fe640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c101b60 0x7f308c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f3090d8a640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f308c105e30 0x7f308c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f308bfff640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c1011a0 0x7f308c198720 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c890 tx=0x7f307c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f30897fa640 1 -- 172.21.15.66:0/3055926051 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c004220 con 0x7f308c1011a0 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f30897fa640 1 -- 172.21.15.66:0/3055926051 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f307c0043c0 con 0x7f308c1011a0 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c19d9b0 con 0x7f308c1011a0 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f30897fa640 1 -- 172.21.15.66:0/3055926051 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c004a00 con 0x7f308c1011a0 2024-04-21T09:12:34.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.565+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c1a5240 con 0x7f308c1011a0 2024-04-21T09:12:34.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.566+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c068de0 con 0x7f308c1011a0 2024-04-21T09:12:34.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.568+0000 7f30897fa640 1 -- 172.21.15.66:0/3055926051 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f307c0079b0 con 0x7f308c1011a0 2024-04-21T09:12:34.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.568+0000 7f30897fa640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30740a3da0 0x7f30740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.568+0000 7f308b7fe640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30740a3da0 0x7f30740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.568+0000 7f30897fa640 1 -- 172.21.15.66:0/3055926051 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f307c0d08f0 con 0x7f308c1011a0 2024-04-21T09:12:34.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.569+0000 7f308b7fe640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30740a3da0 0x7f30740a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f308c100d10 tx=0x7f3080005e50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.573+0000 7f30897fa640 1 -- 172.21.15.66:0/3055926051 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f307c014030 con 0x7f308c1011a0 2024-04-21T09:12:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.799+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f308c109360 con 0x7f308c1011a0 2024-04-21T09:12:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.800+0000 7f30897fa640 1 -- 172.21.15.66:0/3055926051 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f307c09cc20 con 0x7f308c1011a0 2024-04-21T09:12:34.801 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:12:34.801 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:12:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30740a3da0 msgr2=0x7f30740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30740a3da0 0x7f30740a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f308c100d10 tx=0x7f3080005e50 comp rx=0 tx=0).stop 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c1011a0 msgr2=0x7f308c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c1011a0 0x7f308c198720 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c890 tx=0x7f307c00cd50 comp rx=0 tx=0).stop 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 shutdown_connections 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30740a3da0 0x7f30740a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f308c105e30 0x7f308c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c101b60 0x7f308c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 --2- 172.21.15.66:0/3055926051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f308c1011a0 0x7f308c198720 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 >> 172.21.15.66:0/3055926051 conn(0x7f308c077e30 msgr2=0x7f308c0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 shutdown_connections 2024-04-21T09:12:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.802+0000 7f30927ed640 1 -- 172.21.15.66:0/3055926051 wait complete. 2024-04-21T09:12:34.814 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 -- 172.21.15.66:0/2775778490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 msgr2=0x7f8c50102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2775778490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c50102570 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 -- 172.21.15.66:0/2775778490 shutdown_connections 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2775778490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c50107260 0x7f8c5010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2775778490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50102b40 0x7f8c50106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2775778490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c50102570 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 -- 172.21.15.66:0/2775778490 >> 172.21.15.66:0/2775778490 conn(0x7f8c500fd990 msgr2=0x7f8c500ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.917+0000 7f8c575e6640 1 -- 172.21.15.66:0/2775778490 shutdown_connections 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 -- 172.21.15.66:0/2775778490 wait complete. 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 Processor -- start 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 -- start start 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c5019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c50107260 0x7f8c5019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50198620 0x7f8c50198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50112220 con 0x7f8c50102190 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501120a0 con 0x7f8c50198620 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c575e6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501123a0 con 0x7f8c50107260 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c55382640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c5019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c55382640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c5019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55134/0 (socket says 172.21.15.66:55134) 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.918+0000 7f8c55382640 1 -- 172.21.15.66:0/2124410148 learned_addr learned my addr 172.21.15.66:0/2124410148 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c55382640 1 -- 172.21.15.66:0/2124410148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c50107260 msgr2=0x7f8c5019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c55382640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c50107260 0x7f8c5019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c55382640 1 -- 172.21.15.66:0/2124410148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50198620 msgr2=0x7f8c50198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c55382640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50198620 0x7f8c50198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c55382640 1 -- 172.21.15.66:0/2124410148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c50199140 con 0x7f8c50102190 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c55382640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c5019e470 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005940 tx=0x7f8c4c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c467fc640 1 -- 172.21.15.66:0/2124410148 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c043020 con 0x7f8c50102190 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c467fc640 1 -- 172.21.15.66:0/2124410148 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c02cbe0 con 0x7f8c50102190 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c501993d0 con 0x7f8c50102190 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c467fc640 1 -- 172.21.15.66:0/2124410148 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c003bf0 con 0x7f8c50102190 2024-04-21T09:12:34.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.919+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c501a5670 con 0x7f8c50102190 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.921+0000 7f8c467fc640 1 -- 172.21.15.66:0/2124410148 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8c4c046020 con 0x7f8c50102190 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 Processor -- start 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 -- start start 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 0x5577b83c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5577b831c1f0 con 0x5577b83b74d0 2024-04-21T09:12:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577b82b94b0 con 0x5577b83c0a00 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5577b83181b0 con 0x5577b83b7e80 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c467fc640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c3c0a3da0 0x7f8c3c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55146/0 (socket says 172.21.15.66:55146) 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 -- 172.21.15.66:0/2934158721 learned_addr learned my addr 172.21.15.66:0/2934158721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c54b81640 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c467fc640 1 -- 172.21.15.66:0/2124410148 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8c4c0f2d10 con 0x7f8c50102190 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c3c0a3da0 0x7f8c3c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 -- 172.21.15.66:0/2934158721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 msgr2=0x5577b83c0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 0x5577b83c0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 -- 172.21.15.66:0/2934158721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 msgr2=0x5577b83c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.922+0000 7f8c55b83640 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.923+0000 7f8c55b83640 1 -- 172.21.15.66:0/2934158721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b82ab1b0 con 0x5577b83b74d0 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.923+0000 7f8c55b83640 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83b78b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8c480126c0 tx=0x7f8c48012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1303f8fe87908a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.923+0000 7f8c3b7fe640 1 -- 172.21.15.66:0/2934158721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48012050 con 0x5577b83b74d0 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.923+0000 7f8c3b7fe640 1 -- 172.21.15.66:0/2934158721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c48013040 con 0x5577b83b74d0 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.923+0000 7f8c3b7fe640 1 -- 172.21.15.66:0/2934158721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4801b070 con 0x5577b83b74d0 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.923+0000 7f8c55b83640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c3c0a3da0 0x7f8c3c0a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8c4801d000 tx=0x7f8c4801b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2934158721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 msgr2=0x5577b83b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83b78b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8c480126c0 tx=0x7f8c48012b80 comp rx=0 tx=0).stop 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2934158721 shutdown_connections 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 0x5577b83c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2934158721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83b78b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2934158721 >> 172.21.15.66:0/2934158721 conn(0x5577b83b1ba0 msgr2=0x5577b83b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2934158721 shutdown_connections 2024-04-21T09:12:34.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2934158721 wait complete. 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 -- start start 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 Processor -- start 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.924+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 0x5577b83bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5577b83c3970 con 0x5577b83b7e80 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577b82d5500 con 0x5577b83b74d0 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5577b83c3af0 con 0x5577b83c0a00 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c5a1d2740 10 client.0 _ll_register_callbacks cb 0x5577b82a3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83bd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55150/0 (socket says 172.21.15.66:55150) 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 -- 172.21.15.66:0/682238556 learned_addr learned my addr 172.21.15.66:0/682238556 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c55382640 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 -- 172.21.15.66:0/682238556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 msgr2=0x5577b83bdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 0x5577b83bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 -- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 msgr2=0x5577b83c0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83c0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 -- 172.21.15.66:0/682238556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b802e3f0 con 0x5577b83b7e80 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c54b81640 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83bd710 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8c400038a0 tx=0x7f8c40003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f8afe0f3e454b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c38ff9640 1 -- 172.21.15.66:0/682238556 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4001a470 con 0x5577b83b7e80 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c38ff9640 1 -- 172.21.15.66:0/682238556 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8c4001a610 con 0x5577b83b7e80 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c48013040 con 0x5577b83b7e80 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c38ff9640 1 -- 172.21.15.66:0/682238556 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c40023050 con 0x5577b83b7e80 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c5a1d2740 10 client.16887 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.925+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577b80f87d0 con 0x5577b83b7e80 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c38ff9640 1 -- 172.21.15.66:0/682238556 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c48013040 con 0x5577b83b7e80 2024-04-21T09:12:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c38ff9640 1 -- 172.21.15.66:0/682238556 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c400118d0 con 0x5577b83b7e80 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c5a1d2740 10 client.16887 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c5a1d2740 10 client.16887 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c5a1d2740 20 client.16887 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b83d0d40 0x5577b83d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c5a1d2740 4 client.16887 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b82bb930 con 0x5577b83d0d40 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.926+0000 7f8c55382640 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b83d0d40 0x5577b83d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c55382640 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b83d0d40 0x5577b83d3190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ed340b3832efcd8 server_cookie=87eefec6977aa3b in_seq=0 out_seq=0 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c38ff9640 10 client.16887 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c38ff9640 1 -- 172.21.15.66:0/682238556 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577b82bb930 con 0x5577b83d0d40 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c38ff9640 10 client.16887 handle_command_reply: tid=0 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 client.16887 shutdown 2024-04-21T09:12:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 msgr2=0x5577b83bd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83bd710 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8c400038a0 tx=0x7f8c40003ca0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 shutdown_connections 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b83d0d40 0x5577b83d3190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 0x5577b83bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83bd710 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/682238556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c38ff9640 0 client.16887 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.927+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 >> 172.21.15.66:0/682238556 conn(0x5577b83b1ba0 msgr2=0x5577b83b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.928+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 shutdown_connections 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.928+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/682238556 wait complete. 2024-04-21T09:12:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.928+0000 7f8c5a1d2740 20 client.16887 trim_cache size 0 max 16384 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 Processor -- start 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 -- start start 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 0x5577b83d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b1c30 con 0x5577b83b7e80 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501af500 con 0x5577b83b74d0 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501ad330 con 0x5577b83c0a00 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c54b81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c54b81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83c0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55154/0 (socket says 172.21.15.66:55154) 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c54b81640 1 -- 172.21.15.66:0/2881449996 learned_addr learned my addr 172.21.15.66:0/2881449996 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.941+0000 7f8c55382640 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c54b81640 1 -- 172.21.15.66:0/2881449996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 msgr2=0x5577b83d8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c54b81640 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 0x5577b83d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c54b81640 1 -- 172.21.15.66:0/2881449996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 msgr2=0x5577b83bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c54b81640 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c54b81640 1 -- 172.21.15.66:0/2881449996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b83be090 con 0x5577b83b7e80 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c54b81640 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83c0380 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8c40003870 tx=0x7f8c40011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15558194ec070b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c3affd640 1 -- 172.21.15.66:0/2881449996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40003ef0 con 0x5577b83b7e80 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c3affd640 1 -- 172.21.15.66:0/2881449996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c40015070 con 0x5577b83b7e80 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c3affd640 1 -- 172.21.15.66:0/2881449996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4001a6d0 con 0x5577b83b7e80 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2881449996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 msgr2=0x5577b83c0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83c0380 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8c40003870 tx=0x7f8c40011f90 comp rx=0 tx=0).stop 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2881449996 shutdown_connections 2024-04-21T09:12:34.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83c0a00 0x5577b83d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b7e80 0x5577b83c0380 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.942+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/2881449996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83b74d0 0x5577b83bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2881449996 >> 172.21.15.66:0/2881449996 conn(0x5577b83b1ba0 msgr2=0x5577b83b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2881449996 shutdown_connections 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/2881449996 wait complete. 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 -- start start 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 Processor -- start 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 0x5577b83c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501ad9c0 con 0x5577b83b74d0 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501aee30 con 0x5577b83c0a00 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501ae090 con 0x5577b83b7e80 2024-04-21T09:12:34.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c5a1d2740 10 client.0 _ll_register_callbacks cb 0x7f8c40004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c55382640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c55382640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55166/0 (socket says 172.21.15.66:55166) 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.943+0000 7f8c55382640 1 -- 172.21.15.66:0/81567213 learned_addr learned my addr 172.21.15.66:0/81567213 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c55b83640 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c55382640 1 -- 172.21.15.66:0/81567213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 msgr2=0x5577b83c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c55382640 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 0x5577b83c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c55382640 1 -- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 msgr2=0x5577b83c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c55382640 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c55382640 1 -- 172.21.15.66:0/81567213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b83c6510 con 0x5577b83b74d0 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c55382640 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83c8d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02cdd0 tx=0x7f8c4c0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4289b02ab0f3f6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c38ff9640 1 -- 172.21.15.66:0/81567213 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0c76c0 con 0x5577b83b74d0 2024-04-21T09:12:34.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c38ff9640 1 -- 172.21.15.66:0/81567213 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f8c4c0bcd50 con 0x5577b83b74d0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c40015070 con 0x5577b83b74d0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c38ff9640 1 -- 172.21.15.66:0/81567213 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c0054d0 con 0x5577b83b74d0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c5a1d2740 10 client.16893 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577b8437da0 con 0x5577b83b74d0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.944+0000 7f8c38ff9640 1 -- 172.21.15.66:0/81567213 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c40015070 con 0x5577b83b74d0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c38ff9640 1 -- 172.21.15.66:0/81567213 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c4c0c7230 con 0x5577b83b74d0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 10 client.16893 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 10 client.16893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 20 client.16893 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b84384d0 0x5577b843a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 4 client.16893 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577b8329c10 con 0x5577b84384d0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c54b81640 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b84384d0 0x5577b843a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:34.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c38ff9640 10 client.16893 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c54b81640 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b84384d0 0x5577b843a8f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a8887f3690b4088 server_cookie=a5a300c5ca8600ae in_seq=0 out_seq=0 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c38ff9640 1 -- 172.21.15.66:0/81567213 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577b8329c10 con 0x5577b84384d0 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c38ff9640 10 client.16893 handle_command_reply: tid=0 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 1 client.16893 shutdown 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 msgr2=0x5577b83c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.945+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83c8d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02cdd0 tx=0x7f8c4c0df040 comp rx=0 tx=0).stop 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 shutdown_connections 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577b84384d0 0x5577b843a8f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b83c0a00 0x5577b83c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577b83b7e80 0x5577b83c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c38ff9640 0 client.16893 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 --2- 172.21.15.66:0/81567213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577b83b74d0 0x5577b83c8d40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 >> 172.21.15.66:0/81567213 conn(0x5577b83b1ba0 msgr2=0x5577b83b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 shutdown_connections 2024-04-21T09:12:34.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 1 -- 172.21.15.66:0/81567213 wait complete. 2024-04-21T09:12:34.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c5a1d2740 20 client.16893 trim_cache size 0 max 16384 2024-04-21T09:12:34.953 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:12:34.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c3c0a3da0 msgr2=0x7f8c3c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c3c0a3da0 0x7f8c3c0a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8c4801d000 tx=0x7f8c4801b000 comp rx=0 tx=0).stop 2024-04-21T09:12:34.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 msgr2=0x7f8c5019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:34.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c5019e470 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005940 tx=0x7f8c4c02c7e0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 shutdown_connections 2024-04-21T09:12:34.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c3c0a3da0 0x7f8c3c0a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50198620 0x7f8c50198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c50107260 0x7f8c5019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 --2- 172.21.15.66:0/2124410148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c50102190 0x7f8c5019e470 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:34.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 >> 172.21.15.66:0/2124410148 conn(0x7f8c500fd990 msgr2=0x7f8c500fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:34.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 shutdown_connections 2024-04-21T09:12:34.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:34.946+0000 7f8c575e6640 1 -- 172.21.15.66:0/2124410148 wait complete. 2024-04-21T09:12:34.958 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:12:35.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 -- 172.21.15.66:0/3267967759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee806a840 msgr2=0x7f7ee806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 --2- 172.21.15.66:0/3267967759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee806a840 0x7f7ee806ac20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed802bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:35.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 -- 172.21.15.66:0/3267967759 shutdown_connections 2024-04-21T09:12:35.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 --2- 172.21.15.66:0/3267967759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8107270 0x7f7ee810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 --2- 172.21.15.66:0/3267967759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee806b1f0 0x7f7ee8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 --2- 172.21.15.66:0/3267967759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee806a840 0x7f7ee806ac20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 -- 172.21.15.66:0/3267967759 >> 172.21.15.66:0/3267967759 conn(0x7f7ee8077eb0 msgr2=0x7f7ee80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 -- 172.21.15.66:0/3267967759 shutdown_connections 2024-04-21T09:12:35.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.063+0000 7f7eec855640 1 -- 172.21.15.66:0/3267967759 wait complete. 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 Processor -- start 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 -- start start 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee806a840 0x7f7ee8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee8107270 0x7f7ee8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee819cff0 0x7f7ee819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81121b0 con 0x7f7ee819cff0 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8112030 con 0x7f7ee806a840 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7eec855640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8112330 con 0x7f7ee8107270 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7ee6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee819cff0 0x7f7ee819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7ee659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee806a840 0x7f7ee8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7ee6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee819cff0 0x7f7ee819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55194/0 (socket says 172.21.15.66:55194) 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.064+0000 7f7ee6d9d640 1 -- 172.21.15.66:0/1982039472 learned_addr learned my addr 172.21.15.66:0/1982039472 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee8107270 0x7f7ee8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee6d9d640 1 -- 172.21.15.66:0/1982039472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee8107270 msgr2=0x7f7ee8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee6d9d640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee8107270 0x7f7ee8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee6d9d640 1 -- 172.21.15.66:0/1982039472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee806a840 msgr2=0x7f7ee8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee6d9d640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee806a840 0x7f7ee8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee6d9d640 1 -- 172.21.15.66:0/1982039472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee819dae0 con 0x7f7ee819cff0 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee8107270 0x7f7ee8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ee6d9d640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee819cff0 0x7f7ee819d4a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009fd0 tx=0x7f7ee000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ed77fe640 1 -- 172.21.15.66:0/1982039472 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0017070 con 0x7f7ee819cff0 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ed77fe640 1 -- 172.21.15.66:0/1982039472 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee0003e20 con 0x7f7ee819cff0 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee819ddd0 con 0x7f7ee819cff0 2024-04-21T09:12:35.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7ed77fe640 1 -- 172.21.15.66:0/1982039472 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0010460 con 0x7f7ee819cff0 2024-04-21T09:12:35.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.065+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a5570 con 0x7f7ee819cff0 2024-04-21T09:12:35.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.067+0000 7f7ed77fe640 1 -- 172.21.15.66:0/1982039472 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7ee0003970 con 0x7f7ee819cff0 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 Processor -- start 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 -- start start 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f869710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869ce0 0x56100f872820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f8732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56100f7ce1f0 con 0x56100f869330 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56100f76b4b0 con 0x56100f872e40 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7eef448740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56100f7ca1b0 con 0x56100f869ce0 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f869710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f869710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55204/0 (socket says 172.21.15.66:55204) 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3211287244 learned_addr learned my addr 172.21.15.66:0/3211287244 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ed77fe640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7ed00a3d20 0x7f7ed00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f8732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee6d9d640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7ed00a3d20 0x7f7ed00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee659c640 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869ce0 0x56100f872820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3211287244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869ce0 msgr2=0x56100f872820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869ce0 0x56100f872820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3211287244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 msgr2=0x56100f8732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f8732c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.068+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3211287244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56100f75d1b0 con 0x56100f869330 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.069+0000 7f7ed77fe640 1 -- 172.21.15.66:0/1982039472 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7ee00d08c0 con 0x7f7ee819cff0 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.069+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f869710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009e20 tx=0x7f7edc0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d290f3fa0ac2dcbe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.069+0000 7f7ed4ff9640 1 -- 172.21.15.66:0/3211287244 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc017070 con 0x56100f869330 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.069+0000 7f7ed4ff9640 1 -- 172.21.15.66:0/3211287244 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7edc0054d0 con 0x56100f869330 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.069+0000 7f7ed4ff9640 1 -- 172.21.15.66:0/3211287244 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc013600 con 0x56100f869330 2024-04-21T09:12:35.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- 172.21.15.66:0/3211287244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 msgr2=0x56100f869710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7ee6d9d640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7ed00a3d20 0x7f7ed00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7ee009c480 tx=0x7f7ee0004480 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f869710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009e20 tx=0x7f7edc0095d0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- 172.21.15.66:0/3211287244 shutdown_connections 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f8732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869ce0 0x56100f872820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 --2- 172.21.15.66:0/3211287244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f869710 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- 172.21.15.66:0/3211287244 >> 172.21.15.66:0/3211287244 conn(0x56100f863ba0 msgr2=0x56100f863f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- 172.21.15.66:0/3211287244 shutdown_connections 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- 172.21.15.66:0/3211287244 wait complete. 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- start start 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 Processor -- start 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f879280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f876660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f872e40 0x56100f876c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56100f879dc0 con 0x56100f869330 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56100f787500 con 0x56100f869ce0 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.070+0000 7f7eef448740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56100f879f40 con 0x56100f872e40 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7eef448740 10 client.0 _ll_register_callbacks cb 0x56100f755b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f879280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f879280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55216/0 (socket says 172.21.15.66:55216) 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3427719818 learned_addr learned my addr 172.21.15.66:0/3427719818 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee659c640 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f876660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f872e40 0x56100f876c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f872e40 msgr2=0x56100f876c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f872e40 0x56100f876c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3427719818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 msgr2=0x56100f876660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f876660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/3427719818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56100f4e03f0 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f879280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7edc004ca0 tx=0x7f7edc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1ac143c6a57d5d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ecdffb640 1 -- 172.21.15.66:0/3427719818 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc017040 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ecdffb640 1 -- 172.21.15.66:0/3427719818 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7edc01b400 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7edc0054d0 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7ecdffb640 1 -- 172.21.15.66:0/3427719818 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7edc01b600 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7eef448740 10 client.16905 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.071+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56100f5aa7d0 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7ecdffb640 1 -- 172.21.15.66:0/3427719818 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7edc0054d0 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7ecdffb640 1 -- 172.21.15.66:0/3427719818 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7edc013590 con 0x56100f869330 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7eef448740 10 client.16905 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7eef448740 10 client.16905 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:12:35.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7eef448740 20 client.16905 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7eef448740 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f886f90 0x56100f889380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7eef448740 4 client.16905 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56100f76d930 con 0x56100f886f90 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.072+0000 7f7ee659c640 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f886f90 0x56100f889380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.073+0000 7f7ecdffb640 10 client.16905 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.073+0000 7f7ee659c640 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f886f90 0x56100f889380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfc5f07659478b5e server_cookie=875db51014bfdd05 in_seq=0 out_seq=0 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.073+0000 7f7ecdffb640 1 -- 172.21.15.66:0/3427719818 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56100f76d930 con 0x56100f886f90 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.073+0000 7f7ecdffb640 10 client.16905 handle_command_reply: tid=0 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 client.16905 shutdown 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 msgr2=0x56100f879280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f879280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7edc004ca0 tx=0x7f7edc009f70 comp rx=0 tx=0).stop 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 shutdown_connections 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f872e40 0x56100f876c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f876660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f886f90 0x56100f889380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 --2- 172.21.15.66:0/3427719818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869330 0x56100f879280 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7ecdffb640 0 client.16905 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:35.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 >> 172.21.15.66:0/3427719818 conn(0x56100f863ba0 msgr2=0x56100f8658f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 shutdown_connections 2024-04-21T09:12:35.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 1 -- 172.21.15.66:0/3427719818 wait complete. 2024-04-21T09:12:35.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.074+0000 7f7eef448740 20 client.16905 trim_cache size 0 max 16384 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 Processor -- start 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 -- start start 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f882cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f874230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f872e40 0x56100f8758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81afa10 con 0x56100f872e40 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81ad2e0 con 0x56100f869ce0 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7eef448740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81ab110 con 0x56100f869330 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.087+0000 7f7ee5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f872e40 0x56100f8758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f872e40 0x56100f8758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55218/0 (socket says 172.21.15.66:55218) 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/2973202928 learned_addr learned my addr 172.21.15.66:0/2973202928 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee659c640 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f874230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f882cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/2973202928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 msgr2=0x56100f882cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f882cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/2973202928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 msgr2=0x56100f874230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f874230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 -- 172.21.15.66:0/2973202928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56100f869a80 con 0x56100f872e40 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f872e40 0x56100f8758d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7edc013b00 tx=0x7f7edc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe33e51f89437223 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ecffff640 1 -- 172.21.15.66:0/2973202928 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc017070 con 0x56100f872e40 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ecffff640 1 -- 172.21.15.66:0/2973202928 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7edc027040 con 0x56100f872e40 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.088+0000 7f7ecffff640 1 -- 172.21.15.66:0/2973202928 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0052a0 con 0x56100f872e40 2024-04-21T09:12:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- 172.21.15.66:0/2973202928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f872e40 msgr2=0x56100f8758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f872e40 0x56100f8758d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7edc013b00 tx=0x7f7edc009bb0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- 172.21.15.66:0/2973202928 shutdown_connections 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f872e40 0x56100f8758d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f869ce0 0x56100f874230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 --2- 172.21.15.66:0/2973202928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f882cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- 172.21.15.66:0/2973202928 >> 172.21.15.66:0/2973202928 conn(0x56100f863ba0 msgr2=0x56100f862c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- 172.21.15.66:0/2973202928 shutdown_connections 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- 172.21.15.66:0/2973202928 wait complete. 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- start start 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 Processor -- start 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f8783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869ce0 0x56100f878910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f878e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81ab7a0 con 0x56100f869ce0 2024-04-21T09:12:35.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81acc10 con 0x56100f872e40 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81abe70 con 0x56100f869330 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7eef448740 10 client.0 _ll_register_callbacks cb 0x7f7edc010f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.089+0000 7f7ee659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869ce0 0x56100f878910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869ce0 0x56100f878910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55224/0 (socket says 172.21.15.66:55224) 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 -- 172.21.15.66:0/1732166671 learned_addr learned my addr 172.21.15.66:0/1732166671 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f8783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f878e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 -- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 msgr2=0x56100f8783d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f8783d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 -- 172.21.15.66:0/1732166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 msgr2=0x56100f878e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f878e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 -- 172.21.15.66:0/1732166671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56100f8ec550 con 0x56100f869ce0 2024-04-21T09:12:35.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f878e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ee659c640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869ce0 0x56100f878910 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7ed802c170 tx=0x7f7ed8004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752d8e402efa712 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ecdffb640 1 -- 172.21.15.66:0/1732166671 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed802c9f0 con 0x56100f869ce0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7edc027040 con 0x56100f869ce0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ecdffb640 1 -- 172.21.15.66:0/1732166671 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7ed802cb90 con 0x56100f869ce0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ecdffb640 1 -- 172.21.15.66:0/1732166671 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ed8058050 con 0x56100f869ce0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7eef448740 10 client.16911 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56100f8ed810 con 0x56100f869ce0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.090+0000 7f7ecdffb640 1 -- 172.21.15.66:0/1732166671 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7edc027040 con 0x56100f869ce0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7ecdffb640 1 -- 172.21.15.66:0/1732166671 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ed8004b70 con 0x56100f869ce0 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7eef448740 10 client.16911 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7eef448740 10 client.16911 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:12:35.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7eef448740 20 client.16911 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7eef448740 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f8edf60 0x56100f8f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7eef448740 4 client.16911 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56100f7dbc10 con 0x56100f8edf60 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.091+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f8edf60 0x56100f8f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7ecdffb640 10 client.16911 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7ee5d9b640 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f8edf60 0x56100f8f0380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a36eb576e4e8cce server_cookie=5b509e182635a0d9 in_seq=0 out_seq=0 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7ecdffb640 1 -- 172.21.15.66:0/1732166671 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56100f7dbc10 con 0x56100f8edf60 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7ecdffb640 10 client.16911 handle_command_reply: tid=0 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 client.16911 shutdown 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869ce0 msgr2=0x56100f878910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869ce0 0x56100f878910 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7ed802c170 tx=0x7f7ed8004370 comp rx=0 tx=0).stop 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 shutdown_connections 2024-04-21T09:12:35.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56100f872e40 0x56100f878e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x56100f8edf60 0x56100f8f0380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56100f869ce0 0x56100f878910 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 --2- 172.21.15.66:0/1732166671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56100f869330 0x56100f8783d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7ecdffb640 0 client.16911 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 >> 172.21.15.66:0/1732166671 conn(0x56100f863ba0 msgr2=0x56100f870b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 shutdown_connections 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 1 -- 172.21.15.66:0/1732166671 wait complete. 2024-04-21T09:12:35.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.092+0000 7f7eef448740 20 client.16911 trim_cache size 0 max 16384 2024-04-21T09:12:35.101 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:12:35.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7ed00a3d20 msgr2=0x7f7ed00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7ed00a3d20 0x7f7ed00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7ee009c480 tx=0x7f7ee0004480 comp rx=0 tx=0).stop 2024-04-21T09:12:35.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee819cff0 msgr2=0x7f7ee819d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee819cff0 0x7f7ee819d4a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009fd0 tx=0x7f7ee000b260 comp rx=0 tx=0).stop 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 shutdown_connections 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7ed00a3d20 0x7f7ed00a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee819cff0 0x7f7ee819d4a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee8107270 0x7f7ee8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 --2- 172.21.15.66:0/1982039472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee806a840 0x7f7ee8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 >> 172.21.15.66:0/1982039472 conn(0x7f7ee8077eb0 msgr2=0x7f7ee8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 shutdown_connections 2024-04-21T09:12:35.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.093+0000 7f7eec855640 1 -- 172.21.15.66:0/1982039472 wait complete. 2024-04-21T09:12:35.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:12:35.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 -- 172.21.15.66:0/1029069375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338406b1f0 msgr2=0x7f3384106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 --2- 172.21.15.66:0/1029069375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338406b1f0 0x7f3384106b90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3374009900 tx=0x7f337402bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:35.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 -- 172.21.15.66:0/1029069375 shutdown_connections 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 --2- 172.21.15.66:0/1029069375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3384107270 0x7f338410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 --2- 172.21.15.66:0/1029069375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338406b1f0 0x7f3384106b90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 --2- 172.21.15.66:0/1029069375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f338406a840 0x7f338406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 -- 172.21.15.66:0/1029069375 >> 172.21.15.66:0/1029069375 conn(0x7f3384077eb0 msgr2=0x7f33840782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 -- 172.21.15.66:0/1029069375 shutdown_connections 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.207+0000 7f338c611640 1 -- 172.21.15.66:0/1029069375 wait complete. 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 Processor -- start 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 -- start start 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f338406a840 0x7f3384198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384107270 0x7f3384198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338419cff0 0x7f338419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33841121b0 con 0x7f338419cff0 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3384112030 con 0x7f338406a840 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338c611640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3384112330 con 0x7f3384107270 2024-04-21T09:12:35.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338abae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338419cff0 0x7f338419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338abae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338419cff0 0x7f338419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55242/0 (socket says 172.21.15.66:55242) 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338a3ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f338406a840 0x7f3384198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338abae640 1 -- 172.21.15.66:0/1937698444 learned_addr learned my addr 172.21.15.66:0/1937698444 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f3389bac640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384107270 0x7f3384198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.208+0000 7f338abae640 1 -- 172.21.15.66:0/1937698444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384107270 msgr2=0x7f3384198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338abae640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384107270 0x7f3384198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338abae640 1 -- 172.21.15.66:0/1937698444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f338406a840 msgr2=0x7f3384198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338abae640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f338406a840 0x7f3384198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338abae640 1 -- 172.21.15.66:0/1937698444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338419dae0 con 0x7f338419cff0 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f3389bac640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384107270 0x7f3384198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338a3ad640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f338406a840 0x7f3384198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338abae640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338419cff0 0x7f338419d4a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f337c00eb10 tx=0x7f337c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f337b7fe640 1 -- 172.21.15.66:0/1937698444 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c004b20 con 0x7f338419cff0 2024-04-21T09:12:35.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338419ddd0 con 0x7f338419cff0 2024-04-21T09:12:35.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f337b7fe640 1 -- 172.21.15.66:0/1937698444 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f337c004cc0 con 0x7f338419cff0 2024-04-21T09:12:35.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f337b7fe640 1 -- 172.21.15.66:0/1937698444 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c010670 con 0x7f338419cff0 2024-04-21T09:12:35.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.209+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33841a5570 con 0x7f338419cff0 2024-04-21T09:12:35.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.211+0000 7f337b7fe640 1 -- 172.21.15.66:0/1937698444 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f337c005020 con 0x7f338419cff0 2024-04-21T09:12:35.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 Processor -- start 2024-04-21T09:12:35.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 -- start start 2024-04-21T09:12:35.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f54d0 0x55bf842f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f5e80 0x55bf842fe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842fea00 0x55bf84300e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bf8425a1f0 con 0x55bf842f54d0 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bf841f74b0 con 0x55bf842f5e80 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338f201740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf842561b0 con 0x55bf842fea00 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f337b7fe640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f33700a3d20 0x7f33700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f3389bac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f54d0 0x55bf842f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f3389bac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f54d0 0x55bf842f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55248/0 (socket says 172.21.15.66:55248) 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f3389bac640 1 -- 172.21.15.66:0/35110328 learned_addr learned my addr 172.21.15.66:0/35110328 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338a3ad640 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f5e80 0x55bf842fe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.212+0000 7f338abae640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f33700a3d20 0x7f33700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f337b7fe640 1 -- 172.21.15.66:0/1937698444 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f337c0d1120 con 0x7f338419cff0 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3389bac640 1 -- 172.21.15.66:0/35110328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842fea00 msgr2=0x55bf84300e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3389bac640 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842fea00 0x55bf84300e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3389bac640 1 -- 172.21.15.66:0/35110328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f5e80 msgr2=0x55bf842fe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3389bac640 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f5e80 0x55bf842fe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3389bac640 1 -- 172.21.15.66:0/35110328 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf841e91b0 con 0x55bf842f54d0 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f338a3ad640 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f5e80 0x55bf842fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f338abae640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f33700a3d20 0x7f33700a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f337c09ca50 tx=0x7f337c0677f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3389bac640 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f54d0 0x55bf842f58b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3374006010 tx=0x7f3374005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447540d9ca155830 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3378bfe640 1 -- 172.21.15.66:0/35110328 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374004030 con 0x55bf842f54d0 2024-04-21T09:12:35.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3378bfe640 1 -- 172.21.15.66:0/35110328 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374044070 con 0x55bf842f54d0 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.213+0000 7f3378bfe640 1 -- 172.21.15.66:0/35110328 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33740052d0 con 0x55bf842f54d0 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 -- 172.21.15.66:0/35110328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f54d0 msgr2=0x55bf842f58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f54d0 0x55bf842f58b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3374006010 tx=0x7f3374005de0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 -- 172.21.15.66:0/35110328 shutdown_connections 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842fea00 0x55bf84300e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f5e80 0x55bf842fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 --2- 172.21.15.66:0/35110328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f54d0 0x55bf842f58b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 -- 172.21.15.66:0/35110328 >> 172.21.15.66:0/35110328 conn(0x55bf842efba0 msgr2=0x55bf842eff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 -- 172.21.15.66:0/35110328 shutdown_connections 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 -- 172.21.15.66:0/35110328 wait complete. 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 -- start start 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.214+0000 7f338f201740 1 Processor -- start 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf842fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf842fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bf84301910 con 0x55bf842fea00 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bf84213500 con 0x55bf842f54d0 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf84301a90 con 0x55bf842f5e80 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 10 client.0 _ll_register_callbacks cb 0x55bf841e1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55254/0 (socket says 172.21.15.66:55254) 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 -- 172.21.15.66:0/359019250 learned_addr learned my addr 172.21.15.66:0/359019250 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf842fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338a3ad640 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf842fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 -- 172.21.15.66:0/359019250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 msgr2=0x55bf842fb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf842fb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 -- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 msgr2=0x55bf842fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf842fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 -- 172.21.15.66:0/359019250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf83f6c3f0 con 0x55bf842fea00 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f3389bac640 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fbbc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f33740046f0 tx=0x7f337402cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4f47d7acb0b644 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f336dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f336dffb640 1 -- 172.21.15.66:0/359019250 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374004460 con 0x55bf842fea00 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f336dffb640 1 -- 172.21.15.66:0/359019250 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f3374004750 con 0x55bf842fea00 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.215+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3374044070 con 0x55bf842fea00 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f336dffb640 1 -- 172.21.15.66:0/359019250 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f337405a050 con 0x55bf842fea00 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 10 client.16923 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf840367d0 con 0x55bf842fea00 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f336dffb640 1 -- 172.21.15.66:0/359019250 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374044070 con 0x55bf842fea00 2024-04-21T09:12:35.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f336dffb640 1 -- 172.21.15.66:0/359019250 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3374055030 con 0x55bf842fea00 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 10 client.16923 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 20 client.16923 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf8430ea00 0x55bf84310df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 4 client.16923 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf841f9930 con 0x55bf8430ea00 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.216+0000 7f338a3ad640 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf8430ea00 0x55bf84310df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.217+0000 7f336dffb640 10 client.16923 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.217+0000 7f338a3ad640 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf8430ea00 0x55bf84310df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689ee9980f7a017f server_cookie=13d9c9aaaf26e0cc in_seq=0 out_seq=0 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.217+0000 7f336dffb640 1 -- 172.21.15.66:0/359019250 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf841f9930 con 0x55bf8430ea00 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.217+0000 7f336dffb640 10 client.16923 handle_command_reply: tid=0 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.217+0000 7f338f201740 1 client.16923 shutdown 2024-04-21T09:12:35.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 msgr2=0x55bf842fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fbbc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f33740046f0 tx=0x7f337402cce0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 shutdown_connections 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf8430ea00 0x55bf84310df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fbbc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf842fb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 --2- 172.21.15.66:0/359019250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf842fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f336dffb640 0 client.16923 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 >> 172.21.15.66:0/359019250 conn(0x55bf842efba0 msgr2=0x55bf842f18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 shutdown_connections 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 1 -- 172.21.15.66:0/359019250 wait complete. 2024-04-21T09:12:35.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.218+0000 7f338f201740 20 client.16923 trim_cache size 0 max 16384 2024-04-21T09:12:35.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 Processor -- start 2024-04-21T09:12:35.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 -- start start 2024-04-21T09:12:35.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f54d0 0x55bf8430a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f5e80 0x55bf843070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842fea00 0x55bf842eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b1ae0 con 0x55bf842f5e80 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33841af3b0 con 0x55bf842fea00 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338f201740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f33841ad1e0 con 0x55bf842f54d0 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338a3ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f5e80 0x55bf843070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338a3ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f5e80 0x55bf843070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55270/0 (socket says 172.21.15.66:55270) 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f338a3ad640 1 -- 172.21.15.66:0/3300332869 learned_addr learned my addr 172.21.15.66:0/3300332869 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.231+0000 7f3389bac640 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842fea00 0x55bf842eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f3389bac640 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f54d0 0x55bf8430a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f338a3ad640 1 -- 172.21.15.66:0/3300332869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f54d0 msgr2=0x55bf8430a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f338a3ad640 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f54d0 0x55bf8430a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f338a3ad640 1 -- 172.21.15.66:0/3300332869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842fea00 msgr2=0x55bf842eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f338a3ad640 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842fea00 0x55bf842eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f338a3ad640 1 -- 172.21.15.66:0/3300332869 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf842fbee0 con 0x55bf842f5e80 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f3389bac640 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842fea00 0x55bf842eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f338a3ad640 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f5e80 0x55bf843070c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x55bf842f4fa0 tx=0x7f338000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db142016a048b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f336ffff640 1 -- 172.21.15.66:0/3300332869 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380004220 con 0x55bf842f5e80 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f336ffff640 1 -- 172.21.15.66:0/3300332869 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3380023070 con 0x55bf842f5e80 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.232+0000 7f336ffff640 1 -- 172.21.15.66:0/3300332869 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338001f790 con 0x55bf842f5e80 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- 172.21.15.66:0/3300332869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f5e80 msgr2=0x55bf843070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f5e80 0x55bf843070c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x55bf842f4fa0 tx=0x7f338000b390 comp rx=0 tx=0).stop 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- 172.21.15.66:0/3300332869 shutdown_connections 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842fea00 0x55bf842eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842f5e80 0x55bf843070c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 --2- 172.21.15.66:0/3300332869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f54d0 0x55bf8430a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- 172.21.15.66:0/3300332869 >> 172.21.15.66:0/3300332869 conn(0x55bf842efba0 msgr2=0x55bf842ee370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- 172.21.15.66:0/3300332869 shutdown_connections 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- 172.21.15.66:0/3300332869 wait complete. 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- start start 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 Processor -- start 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf843713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf84371900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33841ad870 con 0x55bf842fea00 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33841aece0 con 0x55bf842f54d0 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f33841adf40 con 0x55bf842f5e80 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338f201740 10 client.0 _ll_register_callbacks cb 0x7f337402c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f3389bac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f3389bac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55280/0 (socket says 172.21.15.66:55280) 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f3389bac640 1 -- 172.21.15.66:0/3312206055 learned_addr learned my addr 172.21.15.66:0/3312206055 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f3389bac640 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf843713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.233+0000 7f338a3ad640 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf84371900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f3389bac640 1 -- 172.21.15.66:0/3312206055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 msgr2=0x55bf84371900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f3389bac640 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf84371900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f3389bac640 1 -- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 msgr2=0x55bf843713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f3389bac640 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf843713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f3389bac640 1 -- 172.21.15.66:0/3312206055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf842fdb10 con 0x55bf842fea00 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f3389bac640 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fd420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3374009850 tx=0x7f337402c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1e0ab4602aca5a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f336dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f336dffb640 1 -- 172.21.15.66:0/3312206055 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374004260 con 0x55bf842fea00 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3380023070 con 0x55bf842fea00 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f336dffb640 1 -- 172.21.15.66:0/3312206055 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f3374004400 con 0x55bf842fea00 2024-04-21T09:12:35.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f336dffb640 1 -- 172.21.15.66:0/3312206055 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3374003800 con 0x55bf842fea00 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f338f201740 10 client.16929 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf84378bc0 con 0x55bf842fea00 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f336dffb640 1 -- 172.21.15.66:0/3312206055 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3380023070 con 0x55bf842fea00 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.234+0000 7f336dffb640 1 -- 172.21.15.66:0/3312206055 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3374055040 con 0x55bf842fea00 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 10 client.16929 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 20 client.16929 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf84379230 0x55bf8437b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 4 client.16929 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf84267c10 con 0x55bf84379230 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338a3ad640 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf84379230 0x55bf8437b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338a3ad640 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf84379230 0x55bf8437b650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c1ebcaf3b8110e server_cookie=7f57a95c2b03151b in_seq=0 out_seq=0 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f336dffb640 10 client.16929 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:35.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f336dffb640 1 -- 172.21.15.66:0/3312206055 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf84267c10 con 0x55bf84379230 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f336dffb640 10 client.16929 handle_command_reply: tid=0 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 1 client.16929 shutdown 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 msgr2=0x55bf842fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.235+0000 7f338f201740 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fd420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3374009850 tx=0x7f337402c2b0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 shutdown_connections 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bf84379230 0x55bf8437b650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf842fea00 0x55bf842fd420 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf842f5e80 0x55bf84371900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f336dffb640 0 client.16929 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 --2- 172.21.15.66:0/3312206055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf842f54d0 0x55bf843713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 >> 172.21.15.66:0/3312206055 conn(0x55bf842efba0 msgr2=0x55bf842eec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 shutdown_connections 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 1 -- 172.21.15.66:0/3312206055 wait complete. 2024-04-21T09:12:35.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338f201740 20 client.16929 trim_cache size 0 max 16384 2024-04-21T09:12:35.244 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f33700a3d20 msgr2=0x7f33700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f33700a3d20 0x7f33700a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f337c09ca50 tx=0x7f337c0677f0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338419cff0 msgr2=0x7f338419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338419cff0 0x7f338419d4a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f337c00eb10 tx=0x7f337c00c2d0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 shutdown_connections 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f33700a3d20 0x7f33700a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f338419cff0 0x7f338419d4a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384107270 0x7f3384198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 --2- 172.21.15.66:0/1937698444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f338406a840 0x7f3384198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 >> 172.21.15.66:0/1937698444 conn(0x7f3384077eb0 msgr2=0x7f33840ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 shutdown_connections 2024-04-21T09:12:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:35.236+0000 7f338c611640 1 -- 172.21.15.66:0/1937698444 wait complete. 2024-04-21T09:12:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:35 smithi186 ceph-mon[31505]: pgmap v134: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:35 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3055926051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:12:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:35 smithi083 ceph-mon[31413]: pgmap v134: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:35 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3055926051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:12:35.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:35 smithi066 ceph-mon[27953]: pgmap v134: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:35.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:35 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3055926051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:12:36.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:12:36.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:12:36.248 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:12:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 -- 172.21.15.66:0/3188941189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f490406b1f0 msgr2=0x7f4904106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 --2- 172.21.15.66:0/3188941189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f490406b1f0 0x7f4904106b90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402bc50 comp rx=0 tx=0).stop 2024-04-21T09:12:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 -- 172.21.15.66:0/3188941189 shutdown_connections 2024-04-21T09:12:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 --2- 172.21.15.66:0/3188941189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4904107270 0x7f490410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 --2- 172.21.15.66:0/3188941189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f490406b1f0 0x7f4904106b90 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 --2- 172.21.15.66:0/3188941189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f490406a840 0x7f490406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 -- 172.21.15.66:0/3188941189 >> 172.21.15.66:0/3188941189 conn(0x7f4904077eb0 msgr2=0x7f49040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 -- 172.21.15.66:0/3188941189 shutdown_connections 2024-04-21T09:12:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.366+0000 7f4909c9a640 1 -- 172.21.15.66:0/3188941189 wait complete. 2024-04-21T09:12:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 Processor -- start 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 -- start start 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f490406a840 0x7f49041a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4904107270 0x7f49041a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f490419a7d0 0x7f490419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4904112230 con 0x7f4904107270 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49041120b0 con 0x7f490419a7d0 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4909c9a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f49041123b0 con 0x7f490406a840 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4904107270 0x7f49041a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4904107270 0x7f49041a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55292/0 (socket says 172.21.15.66:55292) 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.367+0000 7f4902ffd640 1 -- 172.21.15.66:0/614438233 learned_addr learned my addr 172.21.15.66:0/614438233 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4903fff640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f490419a7d0 0x7f490419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4902ffd640 1 -- 172.21.15.66:0/614438233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f490406a840 msgr2=0x7f49041a0660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4902ffd640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f490406a840 0x7f49041a0660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4902ffd640 1 -- 172.21.15.66:0/614438233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f490419a7d0 msgr2=0x7f490419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4902ffd640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f490419a7d0 0x7f490419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4902ffd640 1 -- 172.21.15.66:0/614438233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490419b360 con 0x7f4904107270 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4902ffd640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4904107270 0x7f49041a0ba0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f48f4036040 tx=0x7f48f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4900ff9640 1 -- 172.21.15.66:0/614438233 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4043020 con 0x7f4904107270 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490419b5f0 con 0x7f4904107270 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4900ff9640 1 -- 172.21.15.66:0/614438233 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f402cb90 con 0x7f4904107270 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4900ff9640 1 -- 172.21.15.66:0/614438233 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4003bf0 con 0x7f4904107270 2024-04-21T09:12:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.368+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49041a77d0 con 0x7f4904107270 2024-04-21T09:12:36.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.371+0000 7f490c96a740 1 Processor -- start 2024-04-21T09:12:36.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.371+0000 7f4900ff9640 1 -- 172.21.15.66:0/614438233 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f48f4046020 con 0x7f4904107270 2024-04-21T09:12:36.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.371+0000 7f490c96a740 1 -- start start 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f490c96a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3c8e0 0x55ac0be3ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f490c96a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 0x55ac0be45840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f490c96a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be48260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f490c96a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ac0bda0f00 con 0x55ac0be3c8e0 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f490c96a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac0bd3e590 con 0x55ac0be45e10 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f490c96a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ac0bd9cec0 con 0x55ac0be3d290 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3c8e0 0x55ac0be3ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3c8e0 0x55ac0be3ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55298/0 (socket says 172.21.15.66:55298) 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 -- 172.21.15.66:0/504752263 learned_addr learned my addr 172.21.15.66:0/504752263 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 -- 172.21.15.66:0/504752263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 msgr2=0x55ac0be45840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 0x55ac0be45840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 -- 172.21.15.66:0/504752263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 msgr2=0x55ac0be48260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4900ff9640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4902ffd640 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be48260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be48260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 -- 172.21.15.66:0/504752263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0bb7b7e0 con 0x55ac0be3c8e0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4902ffd640 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be48260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4900ff9640 1 -- 172.21.15.66:0/614438233 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f48f40f2c50 con 0x7f4904107270 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.372+0000 7f4903fff640 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3c8e0 0x55ac0be3ccc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009eb0 tx=0x7f48fc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8dee6ad60a09b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f48e9ffb640 1 -- 172.21.15.66:0/504752263 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011a00 con 0x55ac0be3c8e0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f48e9ffb640 1 -- 172.21.15.66:0/504752263 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc011d00 con 0x55ac0be3c8e0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f48e9ffb640 1 -- 172.21.15.66:0/504752263 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc01c3c0 con 0x55ac0be3c8e0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f490c96a740 1 -- 172.21.15.66:0/504752263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3c8e0 msgr2=0x55ac0be3ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f4903fff640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48fc01e000 tx=0x7f48fc021000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f490c96a740 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3c8e0 0x55ac0be3ccc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009eb0 tx=0x7f48fc00f730 comp rx=0 tx=0).stop 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f490c96a740 1 -- 172.21.15.66:0/504752263 shutdown_connections 2024-04-21T09:12:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f490c96a740 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be48260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f490c96a740 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 0x55ac0be45840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.373+0000 7f490c96a740 1 --2- 172.21.15.66:0/504752263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3c8e0 0x55ac0be3ccc0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 -- 172.21.15.66:0/504752263 >> 172.21.15.66:0/504752263 conn(0x55ac0be36fb0 msgr2=0x55ac0be37390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 -- 172.21.15.66:0/504752263 shutdown_connections 2024-04-21T09:12:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 -- 172.21.15.66:0/504752263 wait complete. 2024-04-21T09:12:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 -- start start 2024-04-21T09:12:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 Processor -- start 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3c8e0 0x55ac0be42e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 0x55ac0be43360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ac0be48d80 con 0x55ac0be45e10 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac0bd5a580 con 0x55ac0be3c8e0 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ac0be48f00 con 0x55ac0be3d290 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.374+0000 7f490c96a740 10 client.0 _ll_register_callbacks cb 0x55ac0bd60650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55300/0 (socket says 172.21.15.66:55300) 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 -- 172.21.15.66:0/2186176942 learned_addr learned my addr 172.21.15.66:0/2186176942 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f49037fe640 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3c8e0 0x55ac0be42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 -- 172.21.15.66:0/2186176942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 msgr2=0x55ac0be43360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 0x55ac0be43360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 -- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3c8e0 msgr2=0x55ac0be42e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3c8e0 0x55ac0be42e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 -- 172.21.15.66:0/2186176942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0bbdbf70 con 0x55ac0be45e10 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f4903fff640 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be438a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f48fc010a40 tx=0x7f48fc01df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ba0dc770d768ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f48e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f48e2ffd640 1 -- 172.21.15.66:0/2186176942 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011930 con 0x55ac0be45e10 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f48e2ffd640 1 -- 172.21.15.66:0/2186176942 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f48fc0103d0 con 0x55ac0be45e10 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48fc011d00 con 0x55ac0be45e10 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f48e2ffd640 1 -- 172.21.15.66:0/2186176942 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48fc03d050 con 0x55ac0be45e10 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f490c96a740 10 client.16941 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.375+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac0ba73170 con 0x55ac0be45e10 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f48e2ffd640 1 -- 172.21.15.66:0/2186176942 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc011d00 con 0x55ac0be45e10 2024-04-21T09:12:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f48e2ffd640 1 -- 172.21.15.66:0/2186176942 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f48fc038030 con 0x55ac0be45e10 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f490c96a740 10 client.16941 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f490c96a740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f490c96a740 20 client.16941 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f490c96a740 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0be56220 0x55ac0be58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f490c96a740 4 client.16941 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac0bdae920 con 0x55ac0be56220 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.376+0000 7f49037fe640 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0be56220 0x55ac0be58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.377+0000 7f49037fe640 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0be56220 0x55ac0be58610 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5dce3435a28a7df server_cookie=6a3bdfd558daa96e in_seq=0 out_seq=0 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.377+0000 7f48e2ffd640 10 client.16941 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.377+0000 7f48e2ffd640 1 -- 172.21.15.66:0/2186176942 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac0bdae920 con 0x55ac0be56220 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.377+0000 7f48e2ffd640 10 client.16941 handle_command_reply: tid=0 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.377+0000 7f490c96a740 1 client.16941 shutdown 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 msgr2=0x55ac0be438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be438a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f48fc010a40 tx=0x7f48fc01df90 comp rx=0 tx=0).stop 2024-04-21T09:12:36.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 shutdown_connections 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0be56220 0x55ac0be58610 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be438a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3d290 0x55ac0be43360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f48e2ffd640 0 client.16941 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 --2- 172.21.15.66:0/2186176942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3c8e0 0x55ac0be42e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 >> 172.21.15.66:0/2186176942 conn(0x55ac0be36fb0 msgr2=0x55ac0be38d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 shutdown_connections 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 1 -- 172.21.15.66:0/2186176942 wait complete. 2024-04-21T09:12:36.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.378+0000 7f490c96a740 20 client.16941 trim_cache size 0 max 16384 2024-04-21T09:12:36.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.395+0000 7f490c96a740 1 Processor -- start 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.395+0000 7f490c96a740 1 -- start start 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f490c96a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 0x55ac0be51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f490c96a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3d290 0x55ac0be425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f490c96a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be44c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f490c96a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b3d90 con 0x55ac0be45e10 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f490c96a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b1660 con 0x55ac0be3d290 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f490c96a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f49041af490 con 0x55ac0be3c8e0 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be44c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be44c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55314/0 (socket says 172.21.15.66:55314) 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 -- 172.21.15.66:0/2413882917 learned_addr learned my addr 172.21.15.66:0/2413882917 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 -- 172.21.15.66:0/2413882917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 msgr2=0x55ac0be51c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 0x55ac0be51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 -- 172.21.15.66:0/2413882917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3d290 msgr2=0x55ac0be425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4902ffd640 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3d290 0x55ac0be425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3d290 0x55ac0be425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 -- 172.21.15.66:0/2413882917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0be41370 con 0x55ac0be45e10 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4902ffd640 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3d290 0x55ac0be425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f4903fff640 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be44c50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f48fc010e50 tx=0x7f48fc01de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1635d065f6f7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f48e97fa640 1 -- 172.21.15.66:0/2413882917 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc045030 con 0x55ac0be45e10 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f48e97fa640 1 -- 172.21.15.66:0/2413882917 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc010a70 con 0x55ac0be45e10 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.396+0000 7f48e97fa640 1 -- 172.21.15.66:0/2413882917 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc010660 con 0x55ac0be45e10 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 -- 172.21.15.66:0/2413882917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 msgr2=0x55ac0be44c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be44c50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f48fc010e50 tx=0x7f48fc01de80 comp rx=0 tx=0).stop 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 -- 172.21.15.66:0/2413882917 shutdown_connections 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be45e10 0x55ac0be44c50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be3d290 0x55ac0be425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 --2- 172.21.15.66:0/2413882917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 0x55ac0be51c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 -- 172.21.15.66:0/2413882917 >> 172.21.15.66:0/2413882917 conn(0x55ac0be36fb0 msgr2=0x55ac0be44540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 -- 172.21.15.66:0/2413882917 shutdown_connections 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 -- 172.21.15.66:0/2413882917 wait complete. 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.397+0000 7f490c96a740 1 -- start start 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 1 Processor -- start 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 0x55ac0be49bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3d290 0x55ac0be4a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49041afb20 con 0x55ac0be3d290 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b0f90 con 0x55ac0be45e10 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b01f0 con 0x55ac0be3c8e0 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f490c96a740 10 client.0 _ll_register_callbacks cb 0x7f48fc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3d290 0x55ac0be4a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3d290 0x55ac0be4a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55324/0 (socket says 172.21.15.66:55324) 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 -- 172.21.15.66:0/4191102956 learned_addr learned my addr 172.21.15.66:0/4191102956 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4903fff640 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 -- 172.21.15.66:0/4191102956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 msgr2=0x55ac0be49bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 0x55ac0be49bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 -- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 msgr2=0x55ac0be4a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be4a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 -- 172.21.15.66:0/4191102956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0bebb5c0 con 0x55ac0be3d290 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f4902ffd640 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3d290 0x55ac0be4a110 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f48f40bb360 tx=0x7f48f40be820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5718fdd7f7204860 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.398+0000 7f48e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f48e2ffd640 1 -- 172.21.15.66:0/4191102956 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f40c7600 con 0x55ac0be3d290 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f48e2ffd640 1 -- 172.21.15.66:0/4191102956 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f48f40bebf0 con 0x55ac0be3d290 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48fc010a70 con 0x55ac0be3d290 2024-04-21T09:12:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f48e2ffd640 1 -- 172.21.15.66:0/4191102956 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48f40be990 con 0x55ac0be3d290 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 10 client.16947 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac0bebc8d0 con 0x55ac0be3d290 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f48e2ffd640 1 -- 172.21.15.66:0/4191102956 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc010a70 con 0x55ac0be3d290 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f48e2ffd640 1 -- 172.21.15.66:0/4191102956 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f48f40f0220 con 0x55ac0be3d290 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 10 client.16947 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 20 client.16947 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0bebcfa0 0x55ac0bebf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 4 client.16947 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.399+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac0bda8790 con 0x55ac0bebcfa0 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.400+0000 7f49037fe640 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0bebcfa0 0x55ac0bebf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.400+0000 7f48e2ffd640 10 client.16947 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.400+0000 7f49037fe640 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0bebcfa0 0x55ac0bebf3c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5d6b82f705dd0f server_cookie=b05c25cf21fe9ced in_seq=0 out_seq=0 2024-04-21T09:12:36.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f48e2ffd640 1 -- 172.21.15.66:0/4191102956 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac0bda8790 con 0x55ac0bebcfa0 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f48e2ffd640 10 client.16947 handle_command_reply: tid=0 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 client.16947 shutdown 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3d290 msgr2=0x55ac0be4a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3d290 0x55ac0be4a110 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f48f40bb360 tx=0x7f48f40be820 comp rx=0 tx=0).stop 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 shutdown_connections 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ac0bebcfa0 0x55ac0bebf3c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac0be45e10 0x55ac0be4a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac0be3d290 0x55ac0be4a110 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f48e2ffd640 0 client.16947 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 --2- 172.21.15.66:0/4191102956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0be3c8e0 0x55ac0be49bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 >> 172.21.15.66:0/4191102956 conn(0x55ac0be36fb0 msgr2=0x55ac0be36010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 shutdown_connections 2024-04-21T09:12:36.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 1 -- 172.21.15.66:0/4191102956 wait complete. 2024-04-21T09:12:36.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.401+0000 7f490c96a740 20 client.16947 trim_cache size 0 max 16384 2024-04-21T09:12:36.408 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:12:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:12:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "17ae514c-b326-452a-a634-e45e160e0f69", 2024-04-21T09:12:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:12:36.408 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f48ec0a3d20 msgr2=0x7f48ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48fc01e000 tx=0x7f48fc021000 comp rx=0 tx=0).stop 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4904107270 msgr2=0x7f49041a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4904107270 0x7f49041a0ba0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f48f4036040 tx=0x7f48f40045e0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 shutdown_connections 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f490419a7d0 0x7f490419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4904107270 0x7f49041a0ba0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 --2- 172.21.15.66:0/614438233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f490406a840 0x7f49041a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 >> 172.21.15.66:0/614438233 conn(0x7f4904077eb0 msgr2=0x7f4904108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 shutdown_connections 2024-04-21T09:12:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.402+0000 7f4909c9a640 1 -- 172.21.15.66:0/614438233 wait complete. 2024-04-21T09:12:36.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:12:36.414 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:12:36.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 -- 172.21.15.66:0/2992839624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b8101a30 msgr2=0x7ff7b8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/2992839624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b8101a30 0x7ff7b8105a70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009900 tx=0x7ff7a002bc30 comp rx=0 tx=0).stop 2024-04-21T09:12:36.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 -- 172.21.15.66:0/2992839624 shutdown_connections 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/2992839624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7b8106150 0x7ff7b810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/2992839624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b8101a30 0x7ff7b8105a70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/2992839624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7b8101080 0x7ff7b8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 -- 172.21.15.66:0/2992839624 >> 172.21.15.66:0/2992839624 conn(0x7ff7b8078000 msgr2=0x7ff7b8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 -- 172.21.15.66:0/2992839624 shutdown_connections 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 -- 172.21.15.66:0/2992839624 wait complete. 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 Processor -- start 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.519+0000 7ff7bdf32640 1 -- start start 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7b8101080 0x7ff7b8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7b8106150 0x7ff7b8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b819d000 0x7ff7b819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b8111070 con 0x7ff7b819d000 2024-04-21T09:12:36.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b8110ef0 con 0x7ff7b8106150 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81111f0 con 0x7ff7b8101080 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b819d000 0x7ff7b819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b819d000 0x7ff7b819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55344/0 (socket says 172.21.15.66:55344) 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 -- 172.21.15.66:0/3992969802 learned_addr learned my addr 172.21.15.66:0/3992969802 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7affff640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7b8106150 0x7ff7b8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7b8101080 0x7ff7b8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 -- 172.21.15.66:0/3992969802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7b8101080 msgr2=0x7ff7b8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7b8101080 0x7ff7b8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 -- 172.21.15.66:0/3992969802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7b8106150 msgr2=0x7ff7b8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7b8106150 0x7ff7b8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 -- 172.21.15.66:0/3992969802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b819daf0 con 0x7ff7b819d000 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b819d000 0x7ff7b819d4b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009fd0 tx=0x7ff7b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b57fa640 1 -- 172.21.15.66:0/3992969802 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0017070 con 0x7ff7b819d000 2024-04-21T09:12:36.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b57fa640 1 -- 172.21.15.66:0/3992969802 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7b0003e20 con 0x7ff7b819d000 2024-04-21T09:12:36.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b819dde0 con 0x7ff7b819d000 2024-04-21T09:12:36.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7b57fa640 1 -- 172.21.15.66:0/3992969802 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0010460 con 0x7ff7b819d000 2024-04-21T09:12:36.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.520+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b81a5550 con 0x7ff7b819d000 2024-04-21T09:12:36.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7b57fa640 1 -- 172.21.15.66:0/3992969802 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff7b0003970 con 0x7ff7b819d000 2024-04-21T09:12:36.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 Processor -- start 2024-04-21T09:12:36.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 -- start start 2024-04-21T09:12:36.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7b57fa640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7a40a3d20 0x7ff7a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52580 0x5634e5a52960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52ea0 0x5634e5a5b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a5de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5634e59b71e0 con 0x5634e5a52580 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634e5954390 con 0x5634e5a5ba80 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5634e59b31a0 con 0x5634e5a52ea0 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7affff640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7a40a3d20 0x7ff7a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7b57fa640 1 -- 172.21.15.66:0/3992969802 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff7b00d08b0 con 0x7ff7b819d000 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52580 0x5634e5a52960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52580 0x5634e5a52960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55360/0 (socket says 172.21.15.66:55360) 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.523+0000 7ff7b77fe640 1 -- 172.21.15.66:0/2044004835 learned_addr learned my addr 172.21.15.66:0/2044004835 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a5de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7b77fe640 1 -- 172.21.15.66:0/2044004835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52ea0 msgr2=0x5634e5a5b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52ea0 0x5634e5a5b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7b77fe640 1 -- 172.21.15.66:0/2044004835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 msgr2=0x5634e5a5de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a5de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7b77fe640 1 -- 172.21.15.66:0/2044004835 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e57937d0 con 0x5634e5a52580 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7affff640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7a40a3d20 0x7ff7a40a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009230 tx=0x7ff7a0004040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52580 0x5634e5a52960 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800dcb0 tx=0x7ff7a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4234bf742138fe0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7adffb640 1 -- 172.21.15.66:0/2044004835 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8011440 con 0x5634e5a52580 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7adffb640 1 -- 172.21.15.66:0/2044004835 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a8011a90 con 0x5634e5a52580 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.524+0000 7ff7adffb640 1 -- 172.21.15.66:0/2044004835 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a801b9a0 con 0x5634e5a52580 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2044004835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52580 msgr2=0x5634e5a52960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52580 0x5634e5a52960 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800dcb0 tx=0x7ff7a800f730 comp rx=0 tx=0).stop 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2044004835 shutdown_connections 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a5de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52ea0 0x5634e5a5b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2044004835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52580 0x5634e5a52960 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2044004835 >> 172.21.15.66:0/2044004835 conn(0x5634e5a4ccb0 msgr2=0x5634e5a4d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2044004835 shutdown_connections 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2044004835 wait complete. 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 -- start start 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.525+0000 7ff7c0c02740 1 Processor -- start 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a58ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a59020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a59560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5634e5a5e870 con 0x5634e5a52ea0 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634e59702f0 con 0x5634e5a5ba80 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5634e5a5e9f0 con 0x5634e5a52580 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 10 client.0 _ll_register_callbacks cb 0x5634e597d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a59020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a59020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55372/0 (socket says 172.21.15.66:55372) 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 -- 172.21.15.66:0/7437493 learned_addr learned my addr 172.21.15.66:0/7437493 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a59560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 -- 172.21.15.66:0/7437493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 msgr2=0x5634e5a58ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a58ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 -- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 msgr2=0x5634e5a59560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a59560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 -- 172.21.15.66:0/7437493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e56c93f0 con 0x5634e5a52ea0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7affff640 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a59020 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00096c0 tx=0x7ff7a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a181796f709399f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff796ffd640 1 -- 172.21.15.66:0/7437493 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0050020 con 0x5634e5a52ea0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff796ffd640 1 -- 172.21.15.66:0/7437493 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff7a00053d0 con 0x5634e5a52ea0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.526+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a8011a90 con 0x5634e5a52ea0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff796ffd640 1 -- 172.21.15.66:0/7437493 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7a005b050 con 0x5634e5a52ea0 2024-04-21T09:12:36.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 10 client.16959 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634e56890f0 con 0x5634e5a52ea0 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff796ffd640 1 -- 172.21.15.66:0/7437493 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a8011a90 con 0x5634e5a52ea0 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff796ffd640 1 -- 172.21.15.66:0/7437493 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7a00037b0 con 0x5634e5a52ea0 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 10 client.16959 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 20 client.16959 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5a6bb90 0x5634e5a6e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 4 client.16959 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634e59c4c00 con 0x5634e5a6bb90 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.527+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5a6bb90 0x5634e5a6e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.528+0000 7ff796ffd640 10 client.16959 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.528+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5a6bb90 0x5634e5a6e010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748250bb5cfd0874 server_cookie=569d021c6178912e in_seq=0 out_seq=0 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.528+0000 7ff796ffd640 1 -- 172.21.15.66:0/7437493 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634e59c4c00 con 0x5634e5a6bb90 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.528+0000 7ff796ffd640 10 client.16959 handle_command_reply: tid=0 2024-04-21T09:12:36.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.528+0000 7ff7c0c02740 1 client.16959 shutdown 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 msgr2=0x5634e5a59020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a59020 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00096c0 tx=0x7ff7a0009260 comp rx=0 tx=0).stop 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 shutdown_connections 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5a6bb90 0x5634e5a6e010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a59560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a59020 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/7437493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a58ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff796ffd640 0 client.16959 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 >> 172.21.15.66:0/7437493 conn(0x5634e5a4ccb0 msgr2=0x5634e5a4ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 shutdown_connections 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 1 -- 172.21.15.66:0/7437493 wait complete. 2024-04-21T09:12:36.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.529+0000 7ff7c0c02740 20 client.16959 trim_cache size 0 max 16384 2024-04-21T09:12:36.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.546+0000 7ff7c0c02740 1 Processor -- start 2024-04-21T09:12:36.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.546+0000 7ff7c0c02740 1 -- start start 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a5b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a57a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a67730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81b1be0 con 0x5634e5a52ea0 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81af4b0 con 0x5634e5a5ba80 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81ad2e0 con 0x5634e5a52580 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a57a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a57a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55376/0 (socket says 172.21.15.66:55376) 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 -- 172.21.15.66:0/1109307794 learned_addr learned my addr 172.21.15.66:0/1109307794 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a67730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a5b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 -- 172.21.15.66:0/1109307794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 msgr2=0x5634e5a5b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a5b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 -- 172.21.15.66:0/1109307794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 msgr2=0x5634e5a67730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a67730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 -- 172.21.15.66:0/1109307794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e5a57030 con 0x5634e5a52ea0 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a67730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a5b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7affff640 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a57a20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff7a002c170 tx=0x7ff7a0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea1c88064618df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7ad7fa640 1 -- 172.21.15.66:0/1109307794 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0050040 con 0x5634e5a52ea0 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7ad7fa640 1 -- 172.21.15.66:0/1109307794 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a0047070 con 0x5634e5a52ea0 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.547+0000 7ff7ad7fa640 1 -- 172.21.15.66:0/1109307794 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0004920 con 0x5634e5a52ea0 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 -- 172.21.15.66:0/1109307794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 msgr2=0x5634e5a57a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a57a20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff7a002c170 tx=0x7ff7a0005dd0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 -- 172.21.15.66:0/1109307794 shutdown_connections 2024-04-21T09:12:36.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a5ba80 0x5634e5a67730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a57a20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/1109307794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a52580 0x5634e5a5b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 -- 172.21.15.66:0/1109307794 >> 172.21.15.66:0/1109307794 conn(0x5634e5a4ccb0 msgr2=0x5634e5a59330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 -- 172.21.15.66:0/1109307794 shutdown_connections 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 -- 172.21.15.66:0/1109307794 wait complete. 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 -- start start 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.548+0000 7ff7c0c02740 1 Processor -- start 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a52580 0x5634e5a60220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a5ba80 0x5634e5ace520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81ad970 con 0x5634e5a52ea0 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81aede0 con 0x5634e5a52580 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81ae040 con 0x5634e5a5ba80 2024-04-21T09:12:36.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 10 client.0 _ll_register_callbacks cb 0x7ff7a002c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a60760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55390/0 (socket says 172.21.15.66:55390) 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 -- 172.21.15.66:0/2836407016 learned_addr learned my addr 172.21.15.66:0/2836407016 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7b7fff640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a5ba80 0x5634e5ace520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a52580 0x5634e5a60220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 -- 172.21.15.66:0/2836407016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a5ba80 msgr2=0x5634e5ace520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a5ba80 0x5634e5ace520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 -- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a52580 msgr2=0x5634e5a60220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a52580 0x5634e5a60220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 -- 172.21.15.66:0/2836407016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e5acebe0 con 0x5634e5a52ea0 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a52580 0x5634e5a60220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7affff640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a60760 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0042040 tx=0x7ff7a0004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba4188ed602a11e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:12:36.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff796ffd640 1 -- 172.21.15.66:0/2836407016 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0050040 con 0x5634e5a52ea0 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff796ffd640 1 -- 172.21.15.66:0/2836407016 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff7a00037f0 con 0x5634e5a52ea0 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.549+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a0047070 con 0x5634e5a52ea0 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff796ffd640 1 -- 172.21.15.66:0/2836407016 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7a00055e0 con 0x5634e5a52ea0 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 10 client.16965 fetch_fsmap learned FSMap version 25 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634e5ad7000 con 0x5634e5a52ea0 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff796ffd640 1 -- 172.21.15.66:0/2836407016 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a0047070 con 0x5634e5a52ea0 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff796ffd640 1 -- 172.21.15.66:0/2836407016 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7a004d8c0 con 0x5634e5a52ea0 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 10 client.16965 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 20 client.16965 populate_metadata read hostname 'smithi066' 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5ad7690 0x5634e5ad9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 4 client.16965 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5634e59bea70 con 0x5634e5ad7690 2024-04-21T09:12:36.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.550+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5ad7690 0x5634e5ad9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7b77fe640 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5ad7690 0x5634e5ad9ab0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a091053c1f9be93c server_cookie=5dc2d4ddcd4088f2 in_seq=0 out_seq=0 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff796ffd640 10 client.16965 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff796ffd640 1 -- 172.21.15.66:0/2836407016 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5634e59bea70 con 0x5634e5ad7690 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff796ffd640 10 client.16965 handle_command_reply: tid=0 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 client.16965 shutdown 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 msgr2=0x5634e5a60760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a60760 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0042040 tx=0x7ff7a0004100 comp rx=0 tx=0).stop 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 shutdown_connections 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5634e5ad7690 0x5634e5ad9ab0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5634e5a5ba80 0x5634e5ace520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634e5a52ea0 0x5634e5a60760 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff796ffd640 0 client.16965 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 --2- 172.21.15.66:0/2836407016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e5a52580 0x5634e5a60220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 >> 172.21.15.66:0/2836407016 conn(0x5634e5a4ccb0 msgr2=0x5634e5a58aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 shutdown_connections 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 1 -- 172.21.15.66:0/2836407016 wait complete. 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.551+0000 7ff7c0c02740 20 client.16965 trim_cache size 0 max 16384 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout: "17ae514c-b326-452a-a634-e45e160e0f69": { 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "17ae514c-b326-452a-a634-e45e160e0f69", 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:12:36.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:12:36.560 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:12:36.560 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:12:36.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7a40a3d20 msgr2=0x7ff7a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7a40a3d20 0x7ff7a40a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009230 tx=0x7ff7a0004040 comp rx=0 tx=0).stop 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b819d000 msgr2=0x7ff7b819d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b819d000 0x7ff7b819d4b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009fd0 tx=0x7ff7b000b260 comp rx=0 tx=0).stop 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 shutdown_connections 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7a40a3d20 0x7ff7a40a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b819d000 0x7ff7b819d4b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7b8106150 0x7ff7b8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 --2- 172.21.15.66:0/3992969802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7b8101080 0x7ff7b8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 >> 172.21.15.66:0/3992969802 conn(0x7ff7b8078000 msgr2=0x7ff7b8105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 shutdown_connections 2024-04-21T09:12:36.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:12:36.552+0000 7ff7bdf32640 1 -- 172.21.15.66:0/3992969802 wait complete. 2024-04-21T09:12:36.563 INFO:tasks.cephfs.filesystem:scrub status for tag:17ae514c-b326-452a-a634-e45e160e0f69 - {'path': '/', 'tag': '17ae514c-b326-452a-a634-e45e160e0f69', 'options': 'recursive,force'} 2024-04-21T09:12:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:37 smithi186 ceph-mon[31505]: pgmap v135: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:37 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:12:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:37 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:37 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:12:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:37 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:37.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:37 smithi083 ceph-mon[31413]: pgmap v135: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:37.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:37 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:12:37.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:37 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:37.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:37 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:12:37.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:37 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:37 smithi066 ceph-mon[27953]: pgmap v135: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:37 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:12:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:37 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:37 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:12:37.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:37 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:12:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:39 smithi186 ceph-mon[31505]: pgmap v136: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:39 smithi083 ceph-mon[31413]: pgmap v136: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:39.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:39 smithi066 ceph-mon[27953]: pgmap v136: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:41 smithi186 ceph-mon[31505]: pgmap v137: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:41 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:12:41.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:41 smithi083 ceph-mon[31413]: pgmap v137: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:41.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:41 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:12:41.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:41 smithi066 ceph-mon[27953]: pgmap v137: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:41.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:41 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:12:43.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:43 smithi186 ceph-mon[31505]: pgmap v138: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:43.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:43 smithi083 ceph-mon[31413]: pgmap v138: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:43.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:43 smithi066 ceph-mon[27953]: pgmap v138: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:45.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:45 smithi186 ceph-mon[31505]: pgmap v139: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:45.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:45 smithi083 ceph-mon[31413]: pgmap v139: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:45.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:45 smithi066 ceph-mon[27953]: pgmap v139: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:47.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:47 smithi186 ceph-mon[31505]: pgmap v140: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:47.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:47 smithi083 ceph-mon[31413]: pgmap v140: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:47.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:47 smithi066 ceph-mon[27953]: pgmap v140: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:12:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:49 smithi186 ceph-mon[31505]: pgmap v141: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:49.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:49 smithi083 ceph-mon[31413]: pgmap v141: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:49.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:49 smithi066 ceph-mon[27953]: pgmap v141: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:12:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:51 smithi186 ceph-mon[31505]: pgmap v142: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:51.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:51 smithi083 ceph-mon[31413]: pgmap v142: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:51.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:51 smithi066 ceph-mon[27953]: pgmap v142: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:53.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:53 smithi186 ceph-mon[31505]: pgmap v143: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:53.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:53 smithi083 ceph-mon[31413]: pgmap v143: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:53.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:53 smithi066 ceph-mon[27953]: pgmap v143: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:55.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:55 smithi186 ceph-mon[31505]: pgmap v144: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:55.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:55 smithi083 ceph-mon[31413]: pgmap v144: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:55.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:55 smithi066 ceph-mon[27953]: pgmap v144: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:57.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:57 smithi186 ceph-mon[31505]: pgmap v145: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:57.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:57 smithi083 ceph-mon[31413]: pgmap v145: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:57.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:57 smithi066 ceph-mon[27953]: pgmap v145: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:12:59.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:12:59 smithi186 ceph-mon[31505]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:59.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:12:59 smithi083 ceph-mon[31413]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:12:59.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:12:59 smithi066 ceph-mon[27953]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:01 smithi186 ceph-mon[31505]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:01.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:01 smithi083 ceph-mon[31413]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:01.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:01 smithi066 ceph-mon[27953]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:03.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:03 smithi186 ceph-mon[31505]: pgmap v148: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:03 smithi083 ceph-mon[31413]: pgmap v148: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:03.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:03 smithi066 ceph-mon[27953]: pgmap v148: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:05 smithi186 ceph-mon[31505]: pgmap v149: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:13:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:05 smithi083 ceph-mon[31413]: pgmap v149: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:13:05.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:05 smithi066 ceph-mon[27953]: pgmap v149: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:13:06.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:13:06.564 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 -- 172.21.15.66:0/1247053685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 msgr2=0x7f91c41035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/1247053685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c41035a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009900 tx=0x7f91b402bd60 comp rx=0 tx=0).stop 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 -- 172.21.15.66:0/1247053685 shutdown_connections 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/1247053685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91c410c500 0x7f91c410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/1247053685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91c4103ae0 0x7f91c410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/1247053685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c41035a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 -- 172.21.15.66:0/1247053685 >> 172.21.15.66:0/1247053685 conn(0x7f91c4078000 msgr2=0x7f91c4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:06.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 -- 172.21.15.66:0/1247053685 shutdown_connections 2024-04-21T09:13:06.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 -- 172.21.15.66:0/1247053685 wait complete. 2024-04-21T09:13:06.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 Processor -- start 2024-04-21T09:13:06.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.682+0000 7f91ca8b2640 1 -- start start 2024-04-21T09:13:06.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91ca8b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91ca8b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91c4103ae0 0x7f91c4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91ca8b2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91c410c500 0x7f91c419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91ca8b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91c4112170 con 0x7f91c41031c0 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91ca8b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91c4111ff0 con 0x7f91c4103ae0 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91ca8b2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41122f0 con 0x7f91c410c500 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41768/0 (socket says 172.21.15.66:41768) 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 -- 172.21.15.66:0/4150537925 learned_addr learned my addr 172.21.15.66:0/4150537925 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91c410c500 0x7f91c419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c37fe640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91c4103ae0 0x7f91c4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 -- 172.21.15.66:0/4150537925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91c410c500 msgr2=0x7f91c419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91c410c500 0x7f91c419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 -- 172.21.15.66:0/4150537925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91c4103ae0 msgr2=0x7f91c4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91c4103ae0 0x7f91c4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c3fff640 1 -- 172.21.15.66:0/4150537925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c419d740 con 0x7f91c41031c0 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.683+0000 7f91c37fe640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91c4103ae0 0x7f91c4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.684+0000 7f91c3fff640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c4198750 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f91b4005940 tx=0x7f91b402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.684+0000 7f91c17fa640 1 -- 172.21.15.66:0/4150537925 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4043020 con 0x7f91c41031c0 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.684+0000 7f91c17fa640 1 -- 172.21.15.66:0/4150537925 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b402dd20 con 0x7f91c41031c0 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.684+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c419d9d0 con 0x7f91c41031c0 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.684+0000 7f91c17fa640 1 -- 172.21.15.66:0/4150537925 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4003b80 con 0x7f91c41031c0 2024-04-21T09:13:06.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.684+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c41a52c0 con 0x7f91c41031c0 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.686+0000 7f91c17fa640 1 -- 172.21.15.66:0/4150537925 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f91b4046020 con 0x7f91c41031c0 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.686+0000 7f91cd577740 1 Processor -- start 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91cd577740 1 -- start start 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91cd577740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456f5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91cd577740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb456fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c17fa640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f91ac0a3da0 0x7f91ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91cd577740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb45700e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91cd577740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb4565a1e0 con 0x55eb456f5ea0 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91cd577740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb455f7390 con 0x55eb456f5580 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91cd577740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55eb456561a0 con 0x55eb456fea80 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb456fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb456fe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41780/0 (socket says 172.21.15.66:41780) 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c37fe640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f91ac0a3da0 0x7f91ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 -- 172.21.15.66:0/256708006 learned_addr learned my addr 172.21.15.66:0/256708006 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb45700e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c17fa640 1 -- 172.21.15.66:0/4150537925 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f91b40f2bf0 con 0x7f91c41031c0 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 -- 172.21.15.66:0/256708006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 msgr2=0x55eb45700e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb45700e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 -- 172.21.15.66:0/256708006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 msgr2=0x55eb456f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c3fff640 1 -- 172.21.15.66:0/256708006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb454367d0 con 0x55eb456f5ea0 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.687+0000 7f91c37fe640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f91ac0a3da0 0x7f91ac0a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f91b80058c0 tx=0x7f91b8005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.688+0000 7f91c3fff640 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb456fe540 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f91b40f0020 tx=0x7f91b40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31196f92b572ed82 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.688+0000 7f91aa7fc640 1 -- 172.21.15.66:0/256708006 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40be290 con 0x55eb456f5ea0 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.688+0000 7f91aa7fc640 1 -- 172.21.15.66:0/256708006 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b40bf030 con 0x55eb456f5ea0 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.688+0000 7f91aa7fc640 1 -- 172.21.15.66:0/256708006 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40874f0 con 0x55eb456f5ea0 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.688+0000 7f91cd577740 1 -- 172.21.15.66:0/256708006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 msgr2=0x55eb456fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.688+0000 7f91cd577740 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb456fe540 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f91b40f0020 tx=0x7f91b40035f0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 -- 172.21.15.66:0/256708006 shutdown_connections 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb45700e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb456fe540 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 --2- 172.21.15.66:0/256708006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 -- 172.21.15.66:0/256708006 >> 172.21.15.66:0/256708006 conn(0x55eb456efcb0 msgr2=0x55eb456f0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 -- 172.21.15.66:0/256708006 shutdown_connections 2024-04-21T09:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 -- 172.21.15.66:0/256708006 wait complete. 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 -- start start 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.689+0000 7f91cd577740 1 Processor -- start 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5580 0x55eb456fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456fea80 0x55eb456fc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb45701630 con 0x55eb456f5580 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb456132f0 con 0x55eb456fea80 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55eb457017b0 con 0x55eb456f5ea0 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5580 0x55eb456fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 10 client.0 _ll_register_callbacks cb 0x55eb45620580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5580 0x55eb456fba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41784/0 (socket says 172.21.15.66:41784) 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 -- 172.21.15.66:0/155614995 learned_addr learned my addr 172.21.15.66:0/155614995 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c37fe640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 -- 172.21.15.66:0/155614995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 msgr2=0x55eb456fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456fea80 0x55eb456fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 -- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456fea80 msgr2=0x55eb456fc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456fea80 0x55eb456fc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 -- 172.21.15.66:0/155614995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4536c3f0 con 0x55eb456f5580 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c37fe640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456fea80 0x55eb456fc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91c3fff640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5580 0x55eb456fba80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f91b40bad90 tx=0x7f91b40c76a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96f56205344f66b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91a37fe640 1 -- 172.21.15.66:0/155614995 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40be290 con 0x55eb456f5580 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91a37fe640 1 -- 172.21.15.66:0/155614995 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f91b40f0250 con 0x55eb456f5580 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.690+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b40bf030 con 0x55eb456f5580 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91a37fe640 1 -- 172.21.15.66:0/155614995 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f91b40f0450 con 0x55eb456f5580 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 10 client.16977 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:06.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb4532c0f0 con 0x55eb456f5580 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91a37fe640 1 -- 172.21.15.66:0/155614995 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b40bf030 con 0x55eb456f5580 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91a37fe640 1 -- 172.21.15.66:0/155614995 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f91b40bdd80 con 0x55eb456f5580 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 10 client.16977 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 20 client.16977 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb4570e5f0 0x55eb457109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 4 client.16977 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb45667c00 con 0x55eb4570e5f0 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.691+0000 7f91c37fe640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb4570e5f0 0x55eb457109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.692+0000 7f91c37fe640 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb4570e5f0 0x55eb457109e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b2ecee92f7d99c server_cookie=d4259cab120e3487 in_seq=0 out_seq=0 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.692+0000 7f91a37fe640 10 client.16977 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.692+0000 7f91a37fe640 1 -- 172.21.15.66:0/155614995 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb45667c00 con 0x55eb4570e5f0 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.692+0000 7f91a37fe640 10 client.16977 handle_command_reply: tid=0 2024-04-21T09:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.692+0000 7f91cd577740 1 client.16977 shutdown 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5580 msgr2=0x55eb456fba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5580 0x55eb456fba80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f91b40bad90 tx=0x7f91b40c76a0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 shutdown_connections 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb4570e5f0 0x55eb457109e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456fea80 0x55eb456fc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91a37fe640 0 client.16977 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 --2- 172.21.15.66:0/155614995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5580 0x55eb456fba80 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 >> 172.21.15.66:0/155614995 conn(0x55eb456efcb0 msgr2=0x55eb456f1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 shutdown_connections 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 1 -- 172.21.15.66:0/155614995 wait complete. 2024-04-21T09:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.693+0000 7f91cd577740 20 client.16977 trim_cache size 0 max 16384 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 Processor -- start 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 -- start start 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456fea80 0x55eb4570a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41b1c20 con 0x55eb456fea80 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41af4f0 con 0x55eb456f5580 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91cd577740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41ad320 con 0x55eb456f5ea0 2024-04-21T09:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456fea80 0x55eb4570a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456fea80 0x55eb4570a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41794/0 (socket says 172.21.15.66:41794) 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 -- 172.21.15.66:0/3607231718 learned_addr learned my addr 172.21.15.66:0/3607231718 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c37fe640 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c3fff640 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 -- 172.21.15.66:0/3607231718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 msgr2=0x55eb456fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 -- 172.21.15.66:0/3607231718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 msgr2=0x55eb456fe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456fe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 -- 172.21.15.66:0/3607231718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb456efae0 con 0x55eb456fea80 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.711+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456fea80 0x55eb4570a400 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f91bc012530 tx=0x7f91bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282ae4430f34f0de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91a9ffb640 1 -- 172.21.15.66:0/3607231718 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc012e00 con 0x55eb456fea80 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91a9ffb640 1 -- 172.21.15.66:0/3607231718 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91bc003e20 con 0x55eb456fea80 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91a9ffb640 1 -- 172.21.15.66:0/3607231718 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc004380 con 0x55eb456fea80 2024-04-21T09:13:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 -- 172.21.15.66:0/3607231718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456fea80 msgr2=0x55eb4570a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456fea80 0x55eb4570a400 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f91bc012530 tx=0x7f91bc004a40 comp rx=0 tx=0).stop 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 -- 172.21.15.66:0/3607231718 shutdown_connections 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456fea80 0x55eb4570a400 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456f5ea0 0x55eb456fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 --2- 172.21.15.66:0/3607231718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb456fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 -- 172.21.15.66:0/3607231718 >> 172.21.15.66:0/3607231718 conn(0x55eb456efcb0 msgr2=0x55eb456f1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 -- 172.21.15.66:0/3607231718 shutdown_connections 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.712+0000 7f91cd577740 1 -- 172.21.15.66:0/3607231718 wait complete. 2024-04-21T09:13:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 -- start start 2024-04-21T09:13:06.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 Processor -- start 2024-04-21T09:13:06.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb45702e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb45703360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb4576c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41ad9b0 con 0x55eb456f5ea0 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41aee20 con 0x55eb456f5580 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41ae080 con 0x55eb456fea80 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91cd577740 10 client.0 _ll_register_callbacks cb 0x7f91bc009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb45703360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb45703360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41804/0 (socket says 172.21.15.66:41804) 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91c37fe640 1 -- 172.21.15.66:0/3149694222 learned_addr learned my addr 172.21.15.66:0/3149694222 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91c3fff640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb45702e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.713+0000 7f91c8e4f640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb4576c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91c37fe640 1 -- 172.21.15.66:0/3149694222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 msgr2=0x55eb4576c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91c37fe640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb4576c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91c37fe640 1 -- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 msgr2=0x55eb45702e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91c37fe640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb45702e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91c37fe640 1 -- 172.21.15.66:0/3149694222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb45700e70 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91c3fff640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb45702e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91c37fe640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb45703360 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x55eb456fd050 tx=0x7f91b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5858149c9e9ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91a37fe640 1 -- 172.21.15.66:0/3149694222 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8004220 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91a37fe640 1 -- 172.21.15.66:0/3149694222 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f91b80043c0 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91bc003e20 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91a37fe640 1 -- 172.21.15.66:0/3149694222 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f91b8035050 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91cd577740 10 client.16983 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb457753d0 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.714+0000 7f91a37fe640 1 -- 172.21.15.66:0/3149694222 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91bc003e20 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91a37fe640 1 -- 172.21.15.66:0/3149694222 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f91b80114f0 con 0x55eb456f5ea0 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91cd577740 10 client.16983 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91cd577740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91cd577740 20 client.16983 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91cd577740 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb45775b80 0x55eb45777fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91cd577740 4 client.16983 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb45661a70 con 0x55eb45775b80 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91c3fff640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb45775b80 0x55eb45777fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91a37fe640 10 client.16983 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91c3fff640 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb45775b80 0x55eb45777fa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65b3b2d45a9d9d6 server_cookie=7b44b2f1c96f56a4 in_seq=0 out_seq=0 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91a37fe640 1 -- 172.21.15.66:0/3149694222 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eb45661a70 con 0x55eb45775b80 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91a37fe640 10 client.16983 handle_command_reply: tid=0 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.715+0000 7f91cd577740 1 client.16983 shutdown 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 msgr2=0x55eb45703360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb45703360 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x55eb456fd050 tx=0x7f91b8004730 comp rx=0 tx=0).stop 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 shutdown_connections 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55eb45775b80 0x55eb45777fa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb456fea80 0x55eb4576c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb456f5ea0 0x55eb45703360 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 --2- 172.21.15.66:0/3149694222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb456f5580 0x55eb45702e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91a37fe640 0 client.16983 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 >> 172.21.15.66:0/3149694222 conn(0x55eb456efcb0 msgr2=0x55eb456f1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 shutdown_connections 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 1 -- 172.21.15.66:0/3149694222 wait complete. 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91cd577740 20 client.16983 trim_cache size 0 max 16384 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:13:06.722 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:13:06.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f91ac0a3da0 msgr2=0x7f91ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f91ac0a3da0 0x7f91ac0a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f91b80058c0 tx=0x7f91b8005850 comp rx=0 tx=0).stop 2024-04-21T09:13:06.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 msgr2=0x7f91c4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c4198750 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f91b4005940 tx=0x7f91b402d920 comp rx=0 tx=0).stop 2024-04-21T09:13:06.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 shutdown_connections 2024-04-21T09:13:06.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f91ac0a3da0 0x7f91ac0a6260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91c410c500 0x7f91c419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91c4103ae0 0x7f91c4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 --2- 172.21.15.66:0/4150537925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c41031c0 0x7f91c4198750 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.716+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 >> 172.21.15.66:0/4150537925 conn(0x7f91c4078000 msgr2=0x7f91c40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:06.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.717+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 shutdown_connections 2024-04-21T09:13:06.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.717+0000 7f91ca8b2640 1 -- 172.21.15.66:0/4150537925 wait complete. 2024-04-21T09:13:06.727 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:13:06.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:13:06.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.830+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/4273425308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a741004c0 msgr2=0x7f5a74100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.830+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/4273425308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a741004c0 0x7f5a74100940 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009930 tx=0x7f5a6402bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:06.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.830+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/4273425308 shutdown_connections 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.830+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/4273425308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a74100e80 0x7f5a7410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.830+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/4273425308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a741004c0 0x7f5a74100940 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.830+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/4273425308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a74106f50 0x7f5a74107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.830+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/4273425308 >> 172.21.15.66:0/4273425308 conn(0x7f5a74077450 msgr2=0x7f5a740fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/4273425308 shutdown_connections 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/4273425308 wait complete. 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 Processor -- start 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 -- start start 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a741004c0 0x7f5a741016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74100e80 0x7f5a74101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a74106f50 0x7f5a74102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74112280 con 0x7f5a74100e80 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74112100 con 0x7f5a74106f50 2024-04-21T09:13:06.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.831+0000 7f5a7b5a4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74112400 con 0x7f5a741004c0 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74100e80 0x7f5a74101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74100e80 0x7f5a74101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41812/0 (socket says 172.21.15.66:41812) 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 -- 172.21.15.66:0/3089180680 learned_addr learned my addr 172.21.15.66:0/3089180680 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a79b41640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a74106f50 0x7f5a74102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 -- 172.21.15.66:0/3089180680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a741004c0 msgr2=0x7f5a741016a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a741004c0 0x7f5a741016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 -- 172.21.15.66:0/3089180680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a74106f50 msgr2=0x7f5a74102120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a74106f50 0x7f5a74102120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 -- 172.21.15.66:0/3089180680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741a6be0 con 0x7f5a74100e80 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a79b41640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a74106f50 0x7f5a74102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a78b3f640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74100e80 0x7f5a74101be0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5a64005940 tx=0x7f5a640040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a6a7fc640 1 -- 172.21.15.66:0/3089180680 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64043020 con 0x7f5a74100e80 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a6a7fc640 1 -- 172.21.15.66:0/3089180680 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a640044c0 con 0x7f5a74100e80 2024-04-21T09:13:06.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741a6e70 con 0x7f5a74100e80 2024-04-21T09:13:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a6a7fc640 1 -- 172.21.15.66:0/3089180680 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64003bf0 con 0x7f5a74100e80 2024-04-21T09:13:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.832+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741a7330 con 0x7f5a74100e80 2024-04-21T09:13:06.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.833+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a74069230 con 0x7f5a74100e80 2024-04-21T09:13:06.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.834+0000 7f5a6a7fc640 1 -- 172.21.15.66:0/3089180680 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5a64046020 con 0x7f5a74100e80 2024-04-21T09:13:06.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.835+0000 7f5a6a7fc640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5a600a3d50 0x7f5a600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:06.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.835+0000 7f5a79340640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5a600a3d50 0x7f5a600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:06.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.835+0000 7f5a6a7fc640 1 -- 172.21.15.66:0/3089180680 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f5a640f2b80 con 0x7f5a74100e80 2024-04-21T09:13:06.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.835+0000 7f5a79340640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5a600a3d50 0x7f5a600a6210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000a930 tx=0x7f5a70005c00 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:06.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:06.839+0000 7f5a6a7fc640 1 -- 172.21.15.66:0/3089180680 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a6404d070 con 0x7f5a74100e80 2024-04-21T09:13:07.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.067+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a74102b80 con 0x7f5a74100e80 2024-04-21T09:13:07.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.068+0000 7f5a6a7fc640 1 -- 172.21.15.66:0/3089180680 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f5a640befc0 con 0x7f5a74100e80 2024-04-21T09:13:07.067 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:13:07.067 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:13:07.068 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:13:07.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.069+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5a600a3d50 msgr2=0x7f5a600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5a600a3d50 0x7f5a600a6210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000a930 tx=0x7f5a70005c00 comp rx=0 tx=0).stop 2024-04-21T09:13:07.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74100e80 msgr2=0x7f5a74101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74100e80 0x7f5a74101be0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5a64005940 tx=0x7f5a640040c0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 shutdown_connections 2024-04-21T09:13:07.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5a600a3d50 0x7f5a600a6210 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a74106f50 0x7f5a74102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74100e80 0x7f5a74101be0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 --2- 172.21.15.66:0/3089180680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a741004c0 0x7f5a741016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 >> 172.21.15.66:0/3089180680 conn(0x7f5a74077450 msgr2=0x7f5a74104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 shutdown_connections 2024-04-21T09:13:07.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.070+0000 7f5a7b5a4640 1 -- 172.21.15.66:0/3089180680 wait complete. 2024-04-21T09:13:07.080 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 -- 172.21.15.66:0/449666707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7681015c0 msgr2=0x7fe768101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 --2- 172.21.15.66:0/449666707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7681015c0 0x7fe768101a20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009900 tx=0x7fe75c02bc80 comp rx=0 tx=0).stop 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 -- 172.21.15.66:0/449666707 shutdown_connections 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 --2- 172.21.15.66:0/449666707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768101f60 0x7fe76810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 --2- 172.21.15.66:0/449666707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7681015c0 0x7fe768101a20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 --2- 172.21.15.66:0/449666707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe768105e20 0x7fe768106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 -- 172.21.15.66:0/449666707 >> 172.21.15.66:0/449666707 conn(0x7fe768077410 msgr2=0x7fe7680fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 -- 172.21.15.66:0/449666707 shutdown_connections 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.185+0000 7fe76e13b640 1 -- 172.21.15.66:0/449666707 wait complete. 2024-04-21T09:13:07.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 Processor -- start 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 -- start start 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681015c0 0x7fe76819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe768101f60 0x7fe76819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe768105e20 0x7fe7681a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe768111190 con 0x7fe768101f60 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe768111010 con 0x7fe7681015c0 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe76e13b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe768111310 con 0x7fe768105e20 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe768101f60 0x7fe76819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe768101f60 0x7fe76819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41838/0 (socket says 172.21.15.66:41838) 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 -- 172.21.15.66:0/3675323906 learned_addr learned my addr 172.21.15.66:0/3675323906 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe7677fe640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681015c0 0x7fe76819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe767fff640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe768105e20 0x7fe7681a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 -- 172.21.15.66:0/3675323906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe768105e20 msgr2=0x7fe7681a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe768105e20 0x7fe7681a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 -- 172.21.15.66:0/3675323906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681015c0 msgr2=0x7fe76819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681015c0 0x7fe76819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.186+0000 7fe766ffd640 1 -- 172.21.15.66:0/3675323906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7681a3e10 con 0x7fe768101f60 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.187+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe768101f60 0x7fe76819f590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe75c006010 tx=0x7fe75c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.187+0000 7fe764ff9640 1 -- 172.21.15.66:0/3675323906 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c043020 con 0x7fe768101f60 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.187+0000 7fe764ff9640 1 -- 172.21.15.66:0/3675323906 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c004350 con 0x7fe768101f60 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.187+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7681a40a0 con 0x7fe768101f60 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.187+0000 7fe764ff9640 1 -- 172.21.15.66:0/3675323906 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c003a10 con 0x7fe768101f60 2024-04-21T09:13:07.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.187+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681ab990 con 0x7fe768101f60 2024-04-21T09:13:07.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.189+0000 7fe764ff9640 1 -- 172.21.15.66:0/3675323906 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe75c046020 con 0x7fe768101f60 2024-04-21T09:13:07.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.189+0000 7fe770e05740 1 Processor -- start 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.189+0000 7fe770e05740 1 -- start start 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.189+0000 7fe770e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d6093858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe770e05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d60938e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe770e05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d60938ea00 0x55d609390e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe770e05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d6092ea1f0 con 0x55d60938ea00 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe770e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6092874b0 con 0x55d6093854d0 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe764ff9640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe7500a3d20 0x7fe7500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe770e05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d6092e61b0 con 0x55d609385e80 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d60938ea00 0x55d609390e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d60938ea00 0x55d609390e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41848/0 (socket says 172.21.15.66:41848) 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 -- 172.21.15.66:0/3712166306 learned_addr learned my addr 172.21.15.66:0/3712166306 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe767fff640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe7500a3d20 0x7fe7500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe7677fe640 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d60938e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe767fff640 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d6093858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe764ff9640 1 -- 172.21.15.66:0/3675323906 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe75c0f25a0 con 0x7fe768101f60 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 -- 172.21.15.66:0/3712166306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 msgr2=0x55d60938e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d60938e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 -- 172.21.15.66:0/3712166306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 msgr2=0x55d6093858b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d6093858b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 -- 172.21.15.66:0/3712166306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6092791b0 con 0x55d60938ea00 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe767fff640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe7500a3d20 0x7fe7500a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe760005fd0 tx=0x7fe760011bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d60938ea00 0x55d609390e50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe75c04c000 tx=0x7fe75c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6b832e673889f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe74dffb640 1 -- 172.21.15.66:0/3712166306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0be9e0 con 0x55d60938ea00 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe74dffb640 1 -- 172.21.15.66:0/3712166306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c0b8e50 con 0x55d60938ea00 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.190+0000 7fe74dffb640 1 -- 172.21.15.66:0/3712166306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0ba950 con 0x55d60938ea00 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 -- 172.21.15.66:0/3712166306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d60938ea00 msgr2=0x55d609390e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d60938ea00 0x55d609390e50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe75c04c000 tx=0x7fe75c0056e0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 -- 172.21.15.66:0/3712166306 shutdown_connections 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d60938ea00 0x55d609390e50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d60938e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 --2- 172.21.15.66:0/3712166306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d6093858b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 -- 172.21.15.66:0/3712166306 >> 172.21.15.66:0/3712166306 conn(0x55d60937fba0 msgr2=0x55d60937ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 -- 172.21.15.66:0/3712166306 shutdown_connections 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 -- 172.21.15.66:0/3712166306 wait complete. 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.191+0000 7fe770e05740 1 -- start start 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 1 Processor -- start 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d60938e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d609385e80 0x55d60938b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 0x55d60938bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d6092ea1f0 con 0x55d609385e80 2024-04-21T09:13:07.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6092a3500 con 0x55d6093854d0 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d609391730 con 0x55d60938ea00 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe770e05740 10 client.0 _ll_register_callbacks cb 0x55d609271b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d609385e80 0x55d60938b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d609385e80 0x55d60938b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41860/0 (socket says 172.21.15.66:41860) 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 -- 172.21.15.66:0/3525811092 learned_addr learned my addr 172.21.15.66:0/3525811092 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe7677fe640 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d60938e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe767fff640 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 0x55d60938bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 -- 172.21.15.66:0/3525811092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 msgr2=0x55d60938bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 0x55d60938bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 -- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 msgr2=0x55d60938e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d60938e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.192+0000 7fe766ffd640 1 -- 172.21.15.66:0/3525811092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d608ffc3f0 con 0x55d609385e80 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe766ffd640 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d609385e80 0x55d60938b600 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0be270 tx=0x7fe75c0be3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcd55a3f02e7019 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe746ffd640 1 -- 172.21.15.66:0/3525811092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0db070 con 0x55d609385e80 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe746ffd640 1 -- 172.21.15.66:0/3525811092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe75c0f21e0 con 0x55d609385e80 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c0b8e50 con 0x55d609385e80 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe746ffd640 1 -- 172.21.15.66:0/3525811092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe75c0f2040 con 0x55d609385e80 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 10 client.17001 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d6090c67d0 con 0x55d609385e80 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe746ffd640 1 -- 172.21.15.66:0/3525811092 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c0b8e50 con 0x55d609385e80 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe746ffd640 1 -- 172.21.15.66:0/3525811092 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe75c0c6ac0 con 0x55d609385e80 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 10 client.17001 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 10 client.17001 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 20 client.17001 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d60939e710 0x55d6093a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 4 client.17001 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:07.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.193+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d609289930 con 0x55d60939e710 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.194+0000 7fe7677fe640 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d60939e710 0x55d6093a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.194+0000 7fe7677fe640 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d60939e710 0x55d6093a0b20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e49c23f17b8ab4f1 server_cookie=c8e57c1600d87eeb in_seq=0 out_seq=0 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.194+0000 7fe746ffd640 10 client.17001 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe746ffd640 1 -- 172.21.15.66:0/3525811092 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d609289930 con 0x55d60939e710 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe746ffd640 10 client.17001 handle_command_reply: tid=0 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 client.17001 shutdown 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d609385e80 msgr2=0x55d60938b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d609385e80 0x55d60938b600 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0be270 tx=0x7fe75c0be3b0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 shutdown_connections 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d60939e710 0x55d6093a0b20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 0x55d60938bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d609385e80 0x55d60938b600 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe746ffd640 0 client.17001 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:07.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 --2- 172.21.15.66:0/3525811092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6093854d0 0x55d60938e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 >> 172.21.15.66:0/3525811092 conn(0x55d60937fba0 msgr2=0x55d6093818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 shutdown_connections 2024-04-21T09:13:07.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 1 -- 172.21.15.66:0/3525811092 wait complete. 2024-04-21T09:13:07.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.195+0000 7fe770e05740 20 client.17001 trim_cache size 0 max 16384 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.208+0000 7fe770e05740 1 Processor -- start 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe770e05740 1 -- start start 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe770e05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d6093869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe770e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d609385e80 0x55d60938dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe770e05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 0x55d60938e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe770e05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b80d0 con 0x55d6093854d0 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe770e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b59a0 con 0x55d609385e80 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe770e05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b37d0 con 0x55d60938ea00 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d6093869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d6093869c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41874/0 (socket says 172.21.15.66:41874) 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 -- 172.21.15.66:0/2415144649 learned_addr learned my addr 172.21.15.66:0/2415144649 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 -- 172.21.15.66:0/2415144649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 msgr2=0x55d60938e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 --2- 172.21.15.66:0/2415144649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 0x55d60938e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 -- 172.21.15.66:0/2415144649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d609385e80 msgr2=0x55d60938dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 --2- 172.21.15.66:0/2415144649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d609385e80 0x55d60938dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 -- 172.21.15.66:0/2415144649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d60939afa0 con 0x55d6093854d0 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe7677fe640 1 --2- 172.21.15.66:0/2415144649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d6093869c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x55d6093a1360 tx=0x7fe758005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1154576dab1ea194 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe74d7fa640 1 -- 172.21.15.66:0/2415144649 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580200f0 con 0x55d6093854d0 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe74d7fa640 1 -- 172.21.15.66:0/2415144649 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe758020740 con 0x55d6093854d0 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.209+0000 7fe74d7fa640 1 -- 172.21.15.66:0/2415144649 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758010b50 con 0x55d6093854d0 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 -- 172.21.15.66:0/2415144649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 msgr2=0x55d6093869c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 --2- 172.21.15.66:0/2415144649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d6093869c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x55d6093a1360 tx=0x7fe758005f90 comp rx=0 tx=0).stop 2024-04-21T09:13:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 -- 172.21.15.66:0/2415144649 shutdown_connections 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 --2- 172.21.15.66:0/2415144649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d60938ea00 0x55d60938e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 --2- 172.21.15.66:0/2415144649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d609385e80 0x55d60938dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 --2- 172.21.15.66:0/2415144649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d6093869c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 -- 172.21.15.66:0/2415144649 >> 172.21.15.66:0/2415144649 conn(0x55d60937fba0 msgr2=0x55d6093862b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 -- 172.21.15.66:0/2415144649 shutdown_connections 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 -- 172.21.15.66:0/2415144649 wait complete. 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 -- start start 2024-04-21T09:13:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.210+0000 7fe770e05740 1 Processor -- start 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d609392520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d609392a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d60938ea00 0x55d609392fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b3e60 con 0x55d6093854d0 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b52d0 con 0x55d60938ea00 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b4530 con 0x55d609385e80 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 10 client.0 _ll_register_callbacks cb 0x7fe75c0be2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d609392520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d609392520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41890/0 (socket says 172.21.15.66:41890) 2024-04-21T09:13:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 -- 172.21.15.66:0/2360518787 learned_addr learned my addr 172.21.15.66:0/2360518787 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe766ffd640 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d609392a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 -- 172.21.15.66:0/2360518787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 msgr2=0x55d609392a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d609392a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 -- 172.21.15.66:0/2360518787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d60938ea00 msgr2=0x55d609392fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d60938ea00 0x55d609392fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 -- 172.21.15.66:0/2360518787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d609403d60 con 0x55d6093854d0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe7677fe640 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d609392520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe758003d30 tx=0x7fe758003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b75a9f90619092 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe746ffd640 1 -- 172.21.15.66:0/2360518787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758030020 con 0x55d6093854d0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe746ffd640 1 -- 172.21.15.66:0/2360518787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe7580200f0 con 0x55d6093854d0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.211+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe758020740 con 0x55d6093854d0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe746ffd640 1 -- 172.21.15.66:0/2360518787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe75803b050 con 0x55d6093854d0 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 10 client.17007 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d609405040 con 0x55d6093854d0 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe746ffd640 1 -- 172.21.15.66:0/2360518787 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe758020740 con 0x55d6093854d0 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe746ffd640 1 -- 172.21.15.66:0/2360518787 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe75802c570 con 0x55d6093854d0 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 10 client.17007 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 10 client.17007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 20 client.17007 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d609405810 0x55d609407c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 4 client.17007 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6092f7c10 con 0x55d609405810 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.212+0000 7fe766ffd640 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d609405810 0x55d609407c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe746ffd640 10 client.17007 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe766ffd640 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d609405810 0x55d609407c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ebfa4e4b4c79cdc server_cookie=e901bb51bd5e9b33 in_seq=0 out_seq=0 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe746ffd640 1 -- 172.21.15.66:0/2360518787 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6092f7c10 con 0x55d609405810 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe746ffd640 10 client.17007 handle_command_reply: tid=0 2024-04-21T09:13:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 client.17007 shutdown 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 msgr2=0x55d609392520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d609392520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe758003d30 tx=0x7fe758003d60 comp rx=0 tx=0).stop 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 shutdown_connections 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55d609405810 0x55d609407c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d60938ea00 0x55d609392fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe746ffd640 0 client.17007 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d609385e80 0x55d609392a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 --2- 172.21.15.66:0/2360518787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d6093854d0 0x55d609392520 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 >> 172.21.15.66:0/2360518787 conn(0x55d60937fba0 msgr2=0x55d609386b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 shutdown_connections 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 1 -- 172.21.15.66:0/2360518787 wait complete. 2024-04-21T09:13:07.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.213+0000 7fe770e05740 20 client.17007 trim_cache size 0 max 16384 2024-04-21T09:13:07.220 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe7500a3d20 msgr2=0x7fe7500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe7500a3d20 0x7fe7500a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe760005fd0 tx=0x7fe760011bc0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe768101f60 msgr2=0x7fe76819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe768101f60 0x7fe76819f590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe75c006010 tx=0x7fe75c005d90 comp rx=0 tx=0).stop 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 shutdown_connections 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe7500a3d20 0x7fe7500a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe768105e20 0x7fe7681a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe768101f60 0x7fe76819f590 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 --2- 172.21.15.66:0/3675323906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681015c0 0x7fe76819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 >> 172.21.15.66:0/3675323906 conn(0x7fe768077410 msgr2=0x7fe768100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 shutdown_connections 2024-04-21T09:13:07.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.214+0000 7fe76e13b640 1 -- 172.21.15.66:0/3675323906 wait complete. 2024-04-21T09:13:07.224 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:13:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 -- 172.21.15.66:0/1927957694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 msgr2=0x7f6b9410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 --2- 172.21.15.66:0/1927957694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410dd40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009ca0 tx=0x7f6b8c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 -- 172.21.15.66:0/1927957694 shutdown_connections 2024-04-21T09:13:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 --2- 172.21.15.66:0/1927957694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410dd40 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 --2- 172.21.15.66:0/1927957694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b94101a50 0x7f6b94105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 --2- 172.21.15.66:0/1927957694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b941010a0 0x7f6b94101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 -- 172.21.15.66:0/1927957694 >> 172.21.15.66:0/1927957694 conn(0x7f6b94077eb0 msgr2=0x7f6b940782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 -- 172.21.15.66:0/1927957694 shutdown_connections 2024-04-21T09:13:07.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.328+0000 7f6b9900f640 1 -- 172.21.15.66:0/1927957694 wait complete. 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 Processor -- start 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 -- start start 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b941010a0 0x7f6b9410e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b9410f150 0x7f6b941a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941121f0 con 0x7f6b94106170 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94112070 con 0x7f6b9410f150 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.329+0000 7f6b9900f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94112370 con 0x7f6b941010a0 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41912/0 (socket says 172.21.15.66:41912) 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 -- 172.21.15.66:0/3860441081 learned_addr learned my addr 172.21.15.66:0/3860441081 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9359e640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b9410f150 0x7f6b941a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b941010a0 0x7f6b9410e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 -- 172.21.15.66:0/3860441081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b941010a0 msgr2=0x7f6b9410e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b941010a0 0x7f6b9410e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 -- 172.21.15.66:0/3860441081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b9410f150 msgr2=0x7f6b941a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b9410f150 0x7f6b941a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 -- 172.21.15.66:0/3860441081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b941a6f60 con 0x7f6b94106170 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b941010a0 0x7f6b9410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9359e640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b9410f150 0x7f6b941a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9259c640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410ec10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d740 tx=0x7f6b8800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b7bfff640 1 -- 172.21.15.66:0/3860441081 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88017070 con 0x7f6b94106170 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b941a7250 con 0x7f6b94106170 2024-04-21T09:13:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b7bfff640 1 -- 172.21.15.66:0/3860441081 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8800de50 con 0x7f6b94106170 2024-04-21T09:13:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b7bfff640 1 -- 172.21.15.66:0/3860441081 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88004ce0 con 0x7f6b94106170 2024-04-21T09:13:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.330+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941a77a0 con 0x7f6b94106170 2024-04-21T09:13:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b7bfff640 1 -- 172.21.15.66:0/3860441081 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6b88004e80 con 0x7f6b94106170 2024-04-21T09:13:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 Processor -- start 2024-04-21T09:13:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 -- start start 2024-04-21T09:13:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f92258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f9230e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5602f918a1f0 con 0x5602f9225e80 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602f91274b0 con 0x5602f92254d0 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5602f91861b0 con 0x5602f922ea00 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b7bfff640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6b7c0a3d20 0x7f6b7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b92d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b92d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41918/0 (socket says 172.21.15.66:41918) 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b92d9d640 1 -- 172.21.15.66:0/3228764989 learned_addr learned my addr 172.21.15.66:0/3228764989 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9359e640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6b7c0a3d20 0x7f6b7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9259c640 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f9230e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.333+0000 7f6b9359e640 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f92258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b7bfff640 1 -- 172.21.15.66:0/3860441081 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6b880d0550 con 0x7f6b94106170 2024-04-21T09:13:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b92d9d640 1 -- 172.21.15.66:0/3228764989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 msgr2=0x5602f9230e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f9230e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b92d9d640 1 -- 172.21.15.66:0/3228764989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 msgr2=0x5602f92258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f92258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b92d9d640 1 -- 172.21.15.66:0/3228764989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f91191b0 con 0x5602f9225e80 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b9359e640 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f92258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b9259c640 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f9230e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922e430 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009c30 tx=0x7f6b84010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d375217e101bb36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b9359e640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6b7c0a3d20 0x7f6b7c0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c006010 tx=0x7f6b8c02e040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b797fa640 1 -- 172.21.15.66:0/3228764989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84004030 con 0x5602f9225e80 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b797fa640 1 -- 172.21.15.66:0/3228764989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b840041d0 con 0x5602f9225e80 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.334+0000 7f6b797fa640 1 -- 172.21.15.66:0/3228764989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840126b0 con 0x5602f9225e80 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/3228764989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 msgr2=0x5602f922e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922e430 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009c30 tx=0x7f6b84010740 comp rx=0 tx=0).stop 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/3228764989 shutdown_connections 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f9230e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922e430 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/3228764989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f92258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/3228764989 >> 172.21.15.66:0/3228764989 conn(0x5602f921fba0 msgr2=0x5602f921ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/3228764989 shutdown_connections 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/3228764989 wait complete. 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 -- start start 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 Processor -- start 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f92254d0 0x5602f922e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.335+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f922ea00 0x5602f922bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5602f9231970 con 0x5602f9225e80 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602f9143500 con 0x5602f922ea00 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5602f9231af0 con 0x5602f92254d0 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9bbfd740 10 client.0 _ll_register_callbacks cb 0x5602f9111b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41926/0 (socket says 172.21.15.66:41926) 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 -- 172.21.15.66:0/1633258679 learned_addr learned my addr 172.21.15.66:0/1633258679 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9359e640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f922ea00 0x5602f922bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f92254d0 0x5602f922e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 -- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f92254d0 msgr2=0x5602f922e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f92254d0 0x5602f922e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 -- 172.21.15.66:0/1633258679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f922ea00 msgr2=0x5602f922bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f922ea00 0x5602f922bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 -- 172.21.15.66:0/1633258679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f8e9c3f0 con 0x5602f9225e80 2024-04-21T09:13:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9359e640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f922ea00 0x5602f922bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9259c640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922b5f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6b880985e0 tx=0x7f6b88018ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d234274b733d6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b76ff5640 1 -- 172.21.15.66:0/1633258679 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88066c90 con 0x5602f9225e80 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b76ff5640 1 -- 172.21.15.66:0/1633258679 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6b88022ce0 con 0x5602f9225e80 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b840041d0 con 0x5602f9225e80 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b76ff5640 1 -- 172.21.15.66:0/1633258679 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b8809c180 con 0x5602f9225e80 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9bbfd740 10 client.17019 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.336+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5602f8f667d0 con 0x5602f9225e80 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b76ff5640 1 -- 172.21.15.66:0/1633258679 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b840041d0 con 0x5602f9225e80 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b76ff5640 1 -- 172.21.15.66:0/1633258679 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b880cdbb0 con 0x5602f9225e80 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b9bbfd740 10 client.17019 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b9bbfd740 10 client.17019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b9bbfd740 20 client.17019 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f923eb60 0x5602f9240f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b9bbfd740 4 client.17019 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602f9129930 con 0x5602f923eb60 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.337+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f923eb60 0x5602f9240f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b76ff5640 10 client.17019 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f923eb60 0x5602f9240f90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2861c963b6911e9 server_cookie=4b80d199be7f76ea in_seq=0 out_seq=0 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b76ff5640 1 -- 172.21.15.66:0/1633258679 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602f9129930 con 0x5602f923eb60 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b76ff5640 10 client.17019 handle_command_reply: tid=0 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 client.17019 shutdown 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 msgr2=0x5602f922b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922b5f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6b880985e0 tx=0x7f6b88018ab0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 shutdown_connections 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f922ea00 0x5602f922bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f923eb60 0x5602f9240f90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922b5f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1633258679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f92254d0 0x5602f922e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b76ff5640 0 client.17019 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:07.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.338+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 >> 172.21.15.66:0/1633258679 conn(0x5602f921fba0 msgr2=0x5602f92218f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.339+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 shutdown_connections 2024-04-21T09:13:07.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.339+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1633258679 wait complete. 2024-04-21T09:13:07.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.339+0000 7f6b9bbfd740 20 client.17019 trim_cache size 0 max 16384 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 Processor -- start 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 -- start start 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f923a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f922e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941b3e00 con 0x5602f9225e80 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941b16d0 con 0x5602f92254d0 2024-04-21T09:13:07.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941af500 con 0x5602f922ea00 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41936/0 (socket says 172.21.15.66:41936) 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 -- 172.21.15.66:0/1419289597 learned_addr learned my addr 172.21.15.66:0/1419289597 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 -- 172.21.15.66:0/1419289597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 msgr2=0x5602f922e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9359e640 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f922e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f923a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f922e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 -- 172.21.15.66:0/1419289597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 msgr2=0x5602f923a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f923a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9259c640 1 -- 172.21.15.66:0/1419289597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f9231210 con 0x5602f9225e80 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.352+0000 7f6b9359e640 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f922e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9259c640 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922dd20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6b88098b50 tx=0x7f6b8809c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4b64da366a191c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b78ff9640 1 -- 172.21.15.66:0/1419289597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880a0530 con 0x5602f9225e80 2024-04-21T09:13:07.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b78ff9640 1 -- 172.21.15.66:0/1419289597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b88095020 con 0x5602f9225e80 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b78ff9640 1 -- 172.21.15.66:0/1419289597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880987f0 con 0x5602f9225e80 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1419289597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 msgr2=0x5602f922dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922dd20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6b88098b50 tx=0x7f6b8809c060 comp rx=0 tx=0).stop 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1419289597 shutdown_connections 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f922ea00 0x5602f922e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f9225e80 0x5602f922dd20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/1419289597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f923a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1419289597 >> 172.21.15.66:0/1419289597 conn(0x5602f921fba0 msgr2=0x5602f921ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1419289597 shutdown_connections 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.353+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/1419289597 wait complete. 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 -- start start 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 Processor -- start 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f9237030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f9225e80 0x5602f9227f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f922ea00 0x5602f9228470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941afb90 con 0x5602f922ea00 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941b1000 con 0x5602f92254d0 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941b0260 con 0x5602f9225e80 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9bbfd740 10 client.0 _ll_register_callbacks cb 0x7f6b8809bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f922ea00 0x5602f9228470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f922ea00 0x5602f9228470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41940/0 (socket says 172.21.15.66:41940) 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 -- 172.21.15.66:0/4166132484 learned_addr learned my addr 172.21.15.66:0/4166132484 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f9237030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9259c640 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f9225e80 0x5602f9227f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 -- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f9225e80 msgr2=0x5602f9227f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f9225e80 0x5602f9227f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 -- 172.21.15.66:0/4166132484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 msgr2=0x5602f9237030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f9237030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.354+0000 7f6b9359e640 1 -- 172.21.15.66:0/4166132484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f9228bb0 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9359e640 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f922ea00 0x5602f9228470 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c037040 tx=0x7f6b8c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6c380b77f76ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b76ff5640 1 -- 172.21.15.66:0/4166132484 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c053020 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b76ff5640 1 -- 172.21.15.66:0/4166132484 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6b8c004040 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b88095020 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b76ff5640 1 -- 172.21.15.66:0/4166132484 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c049040 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 10 client.17025 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5602f92a4f70 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b76ff5640 1 -- 172.21.15.66:0/4166132484 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b88095020 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b76ff5640 1 -- 172.21.15.66:0/4166132484 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b8c04f8c0 con 0x5602f922ea00 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 10 client.17025 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 10 client.17025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:13:07.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 20 client.17025 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f92a5710 0x5602f92a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 4 client.17025 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.355+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602f9197c10 con 0x5602f92a5710 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f92a5710 0x5602f92a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b92d9d640 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f92a5710 0x5602f92a7b30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c41f0de29523ced server_cookie=e98bed4a9ce378bb in_seq=0 out_seq=0 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b76ff5640 10 client.17025 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b76ff5640 1 -- 172.21.15.66:0/4166132484 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602f9197c10 con 0x5602f92a5710 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b76ff5640 10 client.17025 handle_command_reply: tid=0 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 client.17025 shutdown 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f922ea00 msgr2=0x5602f9228470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f922ea00 0x5602f9228470 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c037040 tx=0x7f6b8c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 shutdown_connections 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5602f92a5710 0x5602f92a7b30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5602f922ea00 0x5602f9228470 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b76ff5640 0 client.17025 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602f9225e80 0x5602f9227f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 --2- 172.21.15.66:0/4166132484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602f92254d0 0x5602f9237030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 >> 172.21.15.66:0/4166132484 conn(0x5602f921fba0 msgr2=0x5602f92262b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 shutdown_connections 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 1 -- 172.21.15.66:0/4166132484 wait complete. 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.356+0000 7f6b9bbfd740 20 client.17025 trim_cache size 0 max 16384 2024-04-21T09:13:07.363 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:13:07.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6b7c0a3d20 msgr2=0x7f6b7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6b7c0a3d20 0x7f6b7c0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c006010 tx=0x7f6b8c02e040 comp rx=0 tx=0).stop 2024-04-21T09:13:07.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 msgr2=0x7f6b9410ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410ec10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d740 tx=0x7f6b8800dc00 comp rx=0 tx=0).stop 2024-04-21T09:13:07.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 shutdown_connections 2024-04-21T09:13:07.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6b7c0a3d20 0x7f6b7c0a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b9410f150 0x7f6b941a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b94106170 0x7f6b9410ec10 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 --2- 172.21.15.66:0/3860441081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b941010a0 0x7f6b9410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 >> 172.21.15.66:0/3860441081 conn(0x7f6b94077eb0 msgr2=0x7f6b94100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 shutdown_connections 2024-04-21T09:13:07.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.357+0000 7f6b9900f640 1 -- 172.21.15.66:0/3860441081 wait complete. 2024-04-21T09:13:07.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:13:07.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2027438292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f006b1f0 msgr2=0x7f74f0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2027438292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f006b1f0 0x7f74f0106b90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009900 tx=0x7f74d802bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2027438292 shutdown_connections 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2027438292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74f0107270 0x7f74f010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2027438292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f006b1f0 0x7f74f0106b90 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2027438292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f006a840 0x7f74f006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2027438292 >> 172.21.15.66:0/2027438292 conn(0x7f74f0077eb0 msgr2=0x7f74f00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2027438292 shutdown_connections 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.472+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2027438292 wait complete. 2024-04-21T09:13:07.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 Processor -- start 2024-04-21T09:13:07.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 -- start start 2024-04-21T09:13:07.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f006a840 0x7f74f01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74f0107270 0x7f74f0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f019cfb0 0x7f74f019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0112190 con 0x7f74f019cfb0 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0112010 con 0x7f74f0107270 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.473+0000 7f74f4e6a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0112310 con 0x7f74f006a840 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f019cfb0 0x7f74f019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f019cfb0 0x7f74f019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41958/0 (socket says 172.21.15.66:41958) 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 -- 172.21.15.66:0/2890507011 learned_addr learned my addr 172.21.15.66:0/2890507011 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74ee59c640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f006a840 0x7f74f01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74f0107270 0x7f74f0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 -- 172.21.15.66:0/2890507011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f006a840 msgr2=0x7f74f01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f006a840 0x7f74f01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 -- 172.21.15.66:0/2890507011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74f0107270 msgr2=0x7f74f0198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74f0107270 0x7f74f0198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 -- 172.21.15.66:0/2890507011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f019daa0 con 0x7f74f019cfb0 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74ee59c640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f006a840 0x7f74f01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:07.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74eed9d640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f019cfb0 0x7f74f019d460 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f74e800eb10 tx=0x7f74e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74e7fff640 1 -- 172.21.15.66:0/2890507011 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8004b20 con 0x7f74f019cfb0 2024-04-21T09:13:07.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74e7fff640 1 -- 172.21.15.66:0/2890507011 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8004cc0 con 0x7f74f019cfb0 2024-04-21T09:13:07.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74e7fff640 1 -- 172.21.15.66:0/2890507011 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8010670 con 0x7f74f019cfb0 2024-04-21T09:13:07.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f019dd90 con 0x7f74f019cfb0 2024-04-21T09:13:07.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.474+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a5500 con 0x7f74f019cfb0 2024-04-21T09:13:07.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.476+0000 7f74e7fff640 1 -- 172.21.15.66:0/2890507011 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f74e8005020 con 0x7f74f019cfb0 2024-04-21T09:13:07.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 Processor -- start 2024-04-21T09:13:07.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 -- start start 2024-04-21T09:13:07.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83f7e40 0x55a9c83f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9c83531f0 con 0x55a9c83ee330 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9c82f04b0 con 0x55a9c83eece0 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a9c834f1b0 con 0x55a9c83f7e40 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.477+0000 7f74e7fff640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74dc0a3d20 0x7f74dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41962/0 (socket says 172.21.15.66:41962) 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e5d9b640 1 -- 172.21.15.66:0/1624296700 learned_addr learned my addr 172.21.15.66:0/1624296700 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74ee59c640 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74eed9d640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74dc0a3d20 0x7f74dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e7fff640 1 -- 172.21.15.66:0/2890507011 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f74e80d11e0 con 0x7f74f019cfb0 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83f7e40 0x55a9c83f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74ee59c640 1 -- 172.21.15.66:0/1624296700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83f7e40 msgr2=0x55a9c83f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74ee59c640 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83f7e40 0x55a9c83f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74ee59c640 1 -- 172.21.15.66:0/1624296700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 msgr2=0x55a9c83ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74ee59c640 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74ee59c640 1 -- 172.21.15.66:0/1624296700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c82e21b0 con 0x55a9c83eece0 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74eed9d640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74dc0a3d20 0x7f74dc0a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74e80bbfd0 tx=0x7f74e80bd040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74ee59c640 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83f7820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74e000bc80 tx=0x7f74e000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ec349ed734e806 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e4d99640 1 -- 172.21.15.66:0/1624296700 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0015070 con 0x55a9c83eece0 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e4d99640 1 -- 172.21.15.66:0/1624296700 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e001a910 con 0x55a9c83eece0 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.478+0000 7f74e4d99640 1 -- 172.21.15.66:0/1624296700 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e00118e0 con 0x55a9c83eece0 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1624296700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 msgr2=0x55a9c83f7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83f7820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74e000bc80 tx=0x7f74e000f730 comp rx=0 tx=0).stop 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1624296700 shutdown_connections 2024-04-21T09:13:07.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83f7e40 0x55a9c83f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83f7820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1624296700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1624296700 >> 172.21.15.66:0/1624296700 conn(0x55a9c83e8ba0 msgr2=0x55a9c83e8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1624296700 shutdown_connections 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.479+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1624296700 wait complete. 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 -- start start 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 Processor -- start 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 0x55a9c83fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9c83fedc0 con 0x55a9c83ee330 2024-04-21T09:13:07.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9c830c500 con 0x55a9c83f7e40 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a9c83fef40 con 0x55a9c83eece0 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74f7a5d740 10 client.0 _ll_register_callbacks cb 0x55a9c82dab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fe230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41964/0 (socket says 172.21.15.66:41964) 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 -- 172.21.15.66:0/2216569674 learned_addr learned my addr 172.21.15.66:0/2216569674 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 0x55a9c83fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 -- 172.21.15.66:0/2216569674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 msgr2=0x55a9c83fb5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fb5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 -- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 msgr2=0x55a9c83fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 0x55a9c83fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 -- 172.21.15.66:0/2216569674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9c80653f0 con 0x55a9c83ee330 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fe230 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009230 tx=0x7f74d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e4697864029d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.480+0000 7f74d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:07.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74d5ffb640 1 -- 172.21.15.66:0/2216569674 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d802cca0 con 0x55a9c83ee330 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e001a910 con 0x55a9c83ee330 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74d5ffb640 1 -- 172.21.15.66:0/2216569674 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f74d8005240 con 0x55a9c83ee330 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74d5ffb640 1 -- 172.21.15.66:0/2216569674 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74d8054050 con 0x55a9c83ee330 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 10 client.17037 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9c812f7d0 con 0x55a9c83ee330 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74d5ffb640 1 -- 172.21.15.66:0/2216569674 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e001a910 con 0x55a9c83ee330 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74d5ffb640 1 -- 172.21.15.66:0/2216569674 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74d8003680 con 0x55a9c83ee330 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 10 client.17037 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 20 client.17037 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c840bfb0 0x55a9c840e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 4 client.17037 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:07.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9c82f2930 con 0x55a9c840bfb0 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.481+0000 7f74ee59c640 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c840bfb0 0x55a9c840e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.482+0000 7f74d5ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.482+0000 7f74ee59c640 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c840bfb0 0x55a9c840e440 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e493ea33c47ff8 server_cookie=5f63e022ae921b90 in_seq=0 out_seq=0 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74d5ffb640 1 -- 172.21.15.66:0/2216569674 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9c82f2930 con 0x55a9c840bfb0 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74d5ffb640 10 client.17037 handle_command_reply: tid=0 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 client.17037 shutdown 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 msgr2=0x55a9c83fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fe230 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009230 tx=0x7f74d8005fd0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 shutdown_connections 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c840bfb0 0x55a9c840e440 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 0x55a9c83fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/2216569674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fe230 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74d5ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:07.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 >> 172.21.15.66:0/2216569674 conn(0x55a9c83e8ba0 msgr2=0x55a9c83ea8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 shutdown_connections 2024-04-21T09:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 1 -- 172.21.15.66:0/2216569674 wait complete. 2024-04-21T09:13:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.483+0000 7f74f7a5d740 20 client.17037 trim_cache size 0 max 16384 2024-04-21T09:13:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.496+0000 7f74f7a5d740 1 Processor -- start 2024-04-21T09:13:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.496+0000 7f74f7a5d740 1 -- start start 2024-04-21T09:13:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83ee330 0x55a9c8407cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83f7e40 0x55a9c83fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01af910 con 0x55a9c83f7e40 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01ad1e0 con 0x55a9c83eece0 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01ab010 con 0x55a9c83ee330 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83f7e40 0x55a9c83fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83f7e40 0x55a9c83fe230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41978/0 (socket says 172.21.15.66:41978) 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 -- 172.21.15.66:0/1907250940 learned_addr learned my addr 172.21.15.66:0/1907250940 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 -- 172.21.15.66:0/1907250940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83ee330 msgr2=0x55a9c8407cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83ee330 0x55a9c8407cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74ee59c640 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 -- 172.21.15.66:0/1907250940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 msgr2=0x55a9c83fdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83fdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 -- 172.21.15.66:0/1907250940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c83f9b10 con 0x55a9c83f7e40 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74ee59c640 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83f7e40 0x55a9c83fe230 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005de0 tx=0x7f74d8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3743701d7c1b870 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74d7fff640 1 -- 172.21.15.66:0/1907250940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d802cca0 con 0x55a9c83f7e40 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74d7fff640 1 -- 172.21.15.66:0/1907250940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d8051070 con 0x55a9c83f7e40 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.497+0000 7f74d7fff640 1 -- 172.21.15.66:0/1907250940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d80046c0 con 0x55a9c83f7e40 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1907250940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83f7e40 msgr2=0x55a9c83fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83f7e40 0x55a9c83fe230 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005de0 tx=0x7f74d8005e10 comp rx=0 tx=0).stop 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1907250940 shutdown_connections 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83f7e40 0x55a9c83fe230 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83eece0 0x55a9c83fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/1907250940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83ee330 0x55a9c8407cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1907250940 >> 172.21.15.66:0/1907250940 conn(0x55a9c83e8ba0 msgr2=0x55a9c83ea880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1907250940 shutdown_connections 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 -- 172.21.15.66:0/1907250940 wait complete. 2024-04-21T09:13:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 -- start start 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.498+0000 7f74f7a5d740 1 Processor -- start 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 0x55a9c83fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01ab6a0 con 0x55a9c83ee330 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01acb10 con 0x55a9c83f7e40 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01abd70 con 0x55a9c83eece0 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 10 client.0 _ll_register_callbacks cb 0x7f74d8004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41988/0 (socket says 172.21.15.66:41988) 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 -- 172.21.15.66:0/3632254172 learned_addr learned my addr 172.21.15.66:0/3632254172 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74ee59c640 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 -- 172.21.15.66:0/3632254172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 msgr2=0x55a9c83fd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 -- 172.21.15.66:0/3632254172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 msgr2=0x55a9c83fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 0x55a9c83fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 -- 172.21.15.66:0/3632254172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9c8471850 con 0x55a9c83ee330 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74ee59c640 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74e5d9b640 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fd480 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f74d8006010 tx=0x7f74d80095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73912d0cb237e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74d5ffb640 1 -- 172.21.15.66:0/3632254172 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d80035f0 con 0x55a9c83ee330 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74d5ffb640 1 -- 172.21.15.66:0/3632254172 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f74d8003790 con 0x55a9c83ee330 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d8051070 con 0x55a9c83ee330 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.499+0000 7f74d5ffb640 1 -- 172.21.15.66:0/3632254172 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74d8041470 con 0x55a9c83ee330 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 10 client.17043 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9c8472ac0 con 0x55a9c83ee330 2024-04-21T09:13:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74d5ffb640 1 -- 172.21.15.66:0/3632254172 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d8051070 con 0x55a9c83ee330 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74d5ffb640 1 -- 172.21.15.66:0/3632254172 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74d80344d0 con 0x55a9c83ee330 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 10 client.17043 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 20 client.17043 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c8472f90 0x55a9c8475380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 4 client.17043 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9c8360c10 con 0x55a9c8472f90 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74ee59c640 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c8472f90 0x55a9c8475380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74ee59c640 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c8472f90 0x55a9c8475380 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3a9e8206120eb1 server_cookie=d8c8a50cd678cb9d in_seq=0 out_seq=0 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.500+0000 7f74d5ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74d5ffb640 1 -- 172.21.15.66:0/3632254172 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9c8360c10 con 0x55a9c8472f90 2024-04-21T09:13:07.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74d5ffb640 10 client.17043 handle_command_reply: tid=0 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 client.17043 shutdown 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 msgr2=0x55a9c83fd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fd480 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f74d8006010 tx=0x7f74d80095b0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 shutdown_connections 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a9c8472f90 0x55a9c8475380 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9c83f7e40 0x55a9c83fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74d5ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9c83eece0 0x55a9c83fd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 --2- 172.21.15.66:0/3632254172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9c83ee330 0x55a9c83fd480 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 >> 172.21.15.66:0/3632254172 conn(0x55a9c83e8ba0 msgr2=0x55a9c83e9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 shutdown_connections 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 1 -- 172.21.15.66:0/3632254172 wait complete. 2024-04-21T09:13:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f7a5d740 20 client.17043 trim_cache size 0 max 16384 2024-04-21T09:13:07.508 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:13:07.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74dc0a3d20 msgr2=0x7f74dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74dc0a3d20 0x7f74dc0a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74e80bbfd0 tx=0x7f74e80bd040 comp rx=0 tx=0).stop 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f019cfb0 msgr2=0x7f74f019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.501+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f019cfb0 0x7f74f019d460 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f74e800eb10 tx=0x7f74e800c2d0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 shutdown_connections 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f74dc0a3d20 0x7f74dc0a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74f019cfb0 0x7f74f019d460 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74f0107270 0x7f74f0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 --2- 172.21.15.66:0/2890507011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f006a840 0x7f74f01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 >> 172.21.15.66:0/2890507011 conn(0x7f74f0077eb0 msgr2=0x7f74f0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 shutdown_connections 2024-04-21T09:13:07.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:07.502+0000 7f74f4e6a640 1 -- 172.21.15.66:0/2890507011 wait complete. 2024-04-21T09:13:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:07 smithi066 ceph-mon[27953]: pgmap v150: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-21T09:13:07.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:07 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3089180680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:13:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:07 smithi186 ceph-mon[31505]: pgmap v150: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-21T09:13:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:07 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3089180680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:13:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:07 smithi083 ceph-mon[31413]: pgmap v150: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-21T09:13:07.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:07 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3089180680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:13:08.513 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:13:08.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:13:08.513 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:13:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/2696908431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 msgr2=0x7f7f38100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/2696908431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38100a40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009900 tx=0x7f7f2802bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/2696908431 shutdown_connections 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/2696908431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f38100f80 0x7f7f3810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/2696908431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38100a40 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/2696908431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f38106680 0x7f7f38106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/2696908431 >> 172.21.15.66:0/2696908431 conn(0x7f7f38077410 msgr2=0x7f7f380fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/2696908431 shutdown_connections 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/2696908431 wait complete. 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 Processor -- start 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.634+0000 7f7f3f7f4640 1 -- start start 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3f7f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3f7f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f38100f80 0x7f7f38196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3f7f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f38106680 0x7f7f3819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3f7f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810ffb0 con 0x7f7f381005c0 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3f7f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810fe30 con 0x7f7f38100f80 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3f7f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7f38110130 con 0x7f7f38106680 2024-04-21T09:13:08.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42012/0 (socket says 172.21.15.66:42012) 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 -- 172.21.15.66:0/1293164630 learned_addr learned my addr 172.21.15.66:0/1293164630 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f38100f80 0x7f7f38196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 -- 172.21.15.66:0/1293164630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f38106680 msgr2=0x7f7f3819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f38106680 0x7f7f3819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 -- 172.21.15.66:0/1293164630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f38100f80 msgr2=0x7f7f38196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f38100f80 0x7f7f38196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 -- 172.21.15.66:0/1293164630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3819b560 con 0x7f7f381005c0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.635+0000 7f7f3d590640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38196570 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400edd0 tx=0x7f7f3400a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.636+0000 7f7f2e7fc640 1 -- 172.21.15.66:0/1293164630 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34018620 con 0x7f7f381005c0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.636+0000 7f7f2e7fc640 1 -- 172.21.15.66:0/1293164630 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3400ce50 con 0x7f7f381005c0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.636+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3819b850 con 0x7f7f381005c0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.636+0000 7f7f2e7fc640 1 -- 172.21.15.66:0/1293164630 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400ba80 con 0x7f7f381005c0 2024-04-21T09:13:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.636+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f381a30e0 con 0x7f7f381005c0 2024-04-21T09:13:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.638+0000 7f7f2e7fc640 1 -- 172.21.15.66:0/1293164630 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7f3400bc20 con 0x7f7f381005c0 2024-04-21T09:13:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.638+0000 7f7f423de740 1 Processor -- start 2024-04-21T09:13:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.638+0000 7f7f423de740 1 -- start start 2024-04-21T09:13:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f423de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e724cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f423de740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e72d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f423de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e730260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f423de740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55883e688f00 con 0x55883e725290 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f423de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55883e626590 con 0x55883e72de10 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f423de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55883e684ec0 con 0x55883e7248e0 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3d590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e72d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f2e7fc640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7f240a3d20 0x7f7f240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e724cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3d590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e72d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42024/0 (socket says 172.21.15.66:42024) 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3d590640 1 -- 172.21.15.66:0/525825310 learned_addr learned my addr 172.21.15.66:0/525825310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e730260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7f240a3d20 0x7f7f240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 -- 172.21.15.66:0/525825310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 msgr2=0x55883e730260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e730260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f2e7fc640 1 -- 172.21.15.66:0/1293164630 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7f340d1a10 con 0x7f7f381005c0 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 -- 172.21.15.66:0/525825310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 msgr2=0x55883e72d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e72d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 -- 172.21.15.66:0/525825310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883e4637e0 con 0x55883e7248e0 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.639+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e724cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f30015840 tx=0x7f7f30015d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4032de00164f7473 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7f240a3d20 0x7f7f240a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c000 tx=0x7f7f3001b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f237fe640 1 -- 172.21.15.66:0/525825310 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30003b70 con 0x55883e7248e0 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f237fe640 1 -- 172.21.15.66:0/525825310 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f30003d10 con 0x55883e7248e0 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f237fe640 1 -- 172.21.15.66:0/525825310 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004610 con 0x55883e7248e0 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f423de740 1 -- 172.21.15.66:0/525825310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 msgr2=0x55883e724cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f423de740 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e724cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f30015840 tx=0x7f7f30015d00 comp rx=0 tx=0).stop 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f423de740 1 -- 172.21.15.66:0/525825310 shutdown_connections 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f423de740 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e730260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f423de740 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e72d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f423de740 1 --2- 172.21.15.66:0/525825310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e724cc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.640+0000 7f7f423de740 1 -- 172.21.15.66:0/525825310 >> 172.21.15.66:0/525825310 conn(0x55883e71efb0 msgr2=0x55883e71f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 -- 172.21.15.66:0/525825310 shutdown_connections 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 -- 172.21.15.66:0/525825310 wait complete. 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 -- start start 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 Processor -- start 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e725290 0x55883e72b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e72de10 0x55883e72b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55883e730cc0 con 0x55883e72de10 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55883e642580 con 0x55883e725290 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55883e730e40 con 0x55883e7248e0 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.641+0000 7f7f423de740 10 client.0 _ll_register_callbacks cb 0x55883e648650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e72de10 0x55883e72b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e72de10 0x55883e72b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42038/0 (socket says 172.21.15.66:42038) 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 -- 172.21.15.66:0/3960224842 learned_addr learned my addr 172.21.15.66:0/3960224842 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3d590640 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e725290 0x55883e72b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 -- 172.21.15.66:0/3960224842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 msgr2=0x55883e72adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 -- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e725290 msgr2=0x55883e72b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e725290 0x55883e72b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 -- 172.21.15.66:0/3960224842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883e4c3f70 con 0x55883e72de10 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e72de10 0x55883e72b870 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7f30005fd0 tx=0x7f7f30003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3cb8a7920dd3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3960224842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004910 con 0x55883e72de10 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3960224842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7f30004310 con 0x55883e72de10 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f30003d10 con 0x55883e72de10 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.642+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3960224842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f30039050 con 0x55883e72de10 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 10 client.17055 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55883e35b170 con 0x55883e72de10 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3960224842 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f30003d10 con 0x55883e72de10 2024-04-21T09:13:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3960224842 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f30026440 con 0x55883e72de10 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 10 client.17055 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 20 client.17055 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e73e2a0 0x55883e740690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 4 client.17055 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55883e696920 con 0x55883e73e2a0 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.643+0000 7f7f3d590640 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e73e2a0 0x55883e740690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f20ff9640 10 client.17055 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f3d590640 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e73e2a0 0x55883e740690 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114452b97415297c server_cookie=34a793a87900a184 in_seq=0 out_seq=0 2024-04-21T09:13:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3960224842 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55883e696920 con 0x55883e73e2a0 2024-04-21T09:13:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f20ff9640 10 client.17055 handle_command_reply: tid=0 2024-04-21T09:13:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f423de740 1 client.17055 shutdown 2024-04-21T09:13:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e72de10 msgr2=0x55883e72b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f423de740 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e72de10 0x55883e72b870 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7f30005fd0 tx=0x7f7f30003ba0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 shutdown_connections 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f423de740 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e73e2a0 0x55883e740690 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.644+0000 7f7f423de740 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e72de10 0x55883e72b870 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.645+0000 7f7f423de740 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e725290 0x55883e72b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.645+0000 7f7f20ff9640 0 client.17055 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.645+0000 7f7f423de740 1 --2- 172.21.15.66:0/3960224842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.645+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 >> 172.21.15.66:0/3960224842 conn(0x55883e71efb0 msgr2=0x55883e720d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.645+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 shutdown_connections 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.645+0000 7f7f423de740 1 -- 172.21.15.66:0/3960224842 wait complete. 2024-04-21T09:13:08.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.645+0000 7f7f423de740 20 client.17055 trim_cache size 0 max 16384 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 Processor -- start 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 -- start start 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e736280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e745e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381afa00 con 0x55883e725290 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381ad2d0 con 0x55883e72de10 2024-04-21T09:13:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f423de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381ab100 con 0x55883e7248e0 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e736280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e736280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42050/0 (socket says 172.21.15.66:42050) 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 -- 172.21.15.66:0/2942277496 learned_addr learned my addr 172.21.15.66:0/2942277496 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3d590640 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e745e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 -- 172.21.15.66:0/2942277496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 msgr2=0x55883e72d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 -- 172.21.15.66:0/2942277496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 msgr2=0x55883e745e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e745e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 -- 172.21.15.66:0/2942277496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883e740c20 con 0x55883e725290 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e745e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.663+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e736280 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009900 tx=0x7f7f2802c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22de57845b0e7f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f22ffd640 1 -- 172.21.15.66:0/2942277496 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2802ca60 con 0x55883e725290 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f22ffd640 1 -- 172.21.15.66:0/2942277496 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f28005040 con 0x55883e725290 2024-04-21T09:13:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f22ffd640 1 -- 172.21.15.66:0/2942277496 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28004920 con 0x55883e725290 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 -- 172.21.15.66:0/2942277496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 msgr2=0x55883e736280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e736280 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009900 tx=0x7f7f2802c830 comp rx=0 tx=0).stop 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 -- 172.21.15.66:0/2942277496 shutdown_connections 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e745e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e725290 0x55883e736280 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 --2- 172.21.15.66:0/2942277496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e7248e0 0x55883e72d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 -- 172.21.15.66:0/2942277496 >> 172.21.15.66:0/2942277496 conn(0x55883e71efb0 msgr2=0x55883e7202b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 -- 172.21.15.66:0/2942277496 shutdown_connections 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.664+0000 7f7f423de740 1 -- 172.21.15.66:0/2942277496 wait complete. 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 -- start start 2024-04-21T09:13:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 Processor -- start 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e7248e0 0x55883e79a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e725290 0x55883e79abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e79b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381ab790 con 0x55883e7248e0 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381acc00 con 0x55883e72de10 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381abe60 con 0x55883e725290 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f423de740 10 client.0 _ll_register_callbacks cb 0x7f7f30015f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e7248e0 0x55883e79a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e7248e0 0x55883e79a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42062/0 (socket says 172.21.15.66:42062) 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 -- 172.21.15.66:0/3857358170 learned_addr learned my addr 172.21.15.66:0/3857358170 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e79b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e725290 0x55883e79abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 -- 172.21.15.66:0/3857358170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e725290 msgr2=0x55883e79abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e725290 0x55883e79abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 -- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 msgr2=0x55883e79b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e79b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3d590640 1 -- 172.21.15.66:0/3857358170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883e7a58a0 con 0x55883e7248e0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.665+0000 7f7f3dd91640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e79b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e725290 0x55883e79abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f3d590640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e7248e0 0x55883e79a670 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7f340881b0 tx=0x7f7f34088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5a825516dcd499 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3857358170 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340a62b0 con 0x55883e7248e0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3857358170 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f7f3400ad50 con 0x55883e7248e0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f28005040 con 0x55883e7248e0 2024-04-21T09:13:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3857358170 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f3409d640 con 0x55883e7248e0 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 10 client.17061 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55883e7a6bb0 con 0x55883e7248e0 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3857358170 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f28005040 con 0x55883e7248e0 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3857358170 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f3400b040 con 0x55883e7248e0 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 10 client.17061 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 20 client.17061 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e7a7330 0x55883e7a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 4 client.17061 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.666+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55883e690790 con 0x55883e7a7330 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.667+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e7a7330 0x55883e7a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.667+0000 7f7f20ff9640 10 client.17061 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.667+0000 7f7f3cd8f640 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e7a7330 0x55883e7a9750 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb156508071f914 server_cookie=7f35ca18cbf900f5 in_seq=0 out_seq=0 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f20ff9640 1 -- 172.21.15.66:0/3857358170 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55883e690790 con 0x55883e7a7330 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f20ff9640 10 client.17061 handle_command_reply: tid=0 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 client.17061 shutdown 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e7248e0 msgr2=0x55883e79a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e7248e0 0x55883e79a670 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7f340881b0 tx=0x7f7f34088040 comp rx=0 tx=0).stop 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 shutdown_connections 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55883e7a7330 0x55883e7a9750 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55883e72de10 0x55883e79b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883e725290 0x55883e79abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f20ff9640 0 client.17061 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 --2- 172.21.15.66:0/3857358170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55883e7248e0 0x55883e79a670 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.668+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 >> 172.21.15.66:0/3857358170 conn(0x55883e71efb0 msgr2=0x55883e720b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 shutdown_connections 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f423de740 1 -- 172.21.15.66:0/3857358170 wait complete. 2024-04-21T09:13:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f423de740 20 client.17061 trim_cache size 0 max 16384 2024-04-21T09:13:08.674 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:13:08.674 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:13:08.674 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "eca9aeee-dad4-4cd5-b3d4-1ad84ebad14b", 2024-04-21T09:13:08.674 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:13:08.674 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7f240a3d20 msgr2=0x7f7f240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7f240a3d20 0x7f7f240a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c000 tx=0x7f7f3001b000 comp rx=0 tx=0).stop 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 msgr2=0x7f7f38196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38196570 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400edd0 tx=0x7f7f3400a720 comp rx=0 tx=0).stop 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 shutdown_connections 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7f240a3d20 0x7f7f240a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f38106680 0x7f7f3819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f38100f80 0x7f7f38196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 --2- 172.21.15.66:0/1293164630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f381005c0 0x7f7f38196570 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 >> 172.21.15.66:0/1293164630 conn(0x7f7f38077410 msgr2=0x7f7f380fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 shutdown_connections 2024-04-21T09:13:08.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.669+0000 7f7f3f7f4640 1 -- 172.21.15.66:0/1293164630 wait complete. 2024-04-21T09:13:08.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:13:08.680 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:13:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 -- 172.21.15.66:0/2898301658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc61006a5b0 msgr2=0x7fc61006aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 --2- 172.21.15.66:0/2898301658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc61006a5b0 0x7fc61006aa30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009930 tx=0x7fc60c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 -- 172.21.15.66:0/2898301658 shutdown_connections 2024-04-21T09:13:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 --2- 172.21.15.66:0/2898301658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc61006af70 0x7fc61010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 --2- 172.21.15.66:0/2898301658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc61006a5b0 0x7fc61006aa30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 --2- 172.21.15.66:0/2898301658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610108060 0x7fc610108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 -- 172.21.15.66:0/2898301658 >> 172.21.15.66:0/2898301658 conn(0x7fc610077c40 msgr2=0x7fc610078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 -- 172.21.15.66:0/2898301658 shutdown_connections 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.785+0000 7fc618375640 1 -- 172.21.15.66:0/2898301658 wait complete. 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 Processor -- start 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 -- start start 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc61006a5b0 0x7fc6101986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61006af70 0x7fc610198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc610108060 0x7fc61019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc610112200 con 0x7fc610108060 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc610112080 con 0x7fc61006af70 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc618375640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc610112380 con 0x7fc61006a5b0 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc616912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc610108060 0x7fc61019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc616912640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc610108060 0x7fc61019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42076/0 (socket says 172.21.15.66:42076) 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc616912640 1 -- 172.21.15.66:0/3340272916 learned_addr learned my addr 172.21.15.66:0/3340272916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc615910640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61006af70 0x7fc610198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.786+0000 7fc616111640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc61006a5b0 0x7fc6101986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc616912640 1 -- 172.21.15.66:0/3340272916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc61006a5b0 msgr2=0x7fc6101986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc616912640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc61006a5b0 0x7fc6101986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc616912640 1 -- 172.21.15.66:0/3340272916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61006af70 msgr2=0x7fc610198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc616912640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61006af70 0x7fc610198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc616912640 1 -- 172.21.15.66:0/3340272916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61019d6d0 con 0x7fc610108060 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc616111640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc61006a5b0 0x7fc6101986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc616912640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc610108060 0x7fc61019cfa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b970 tx=0x7fc60800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc6037fe640 1 -- 172.21.15.66:0/3340272916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608019070 con 0x7fc610108060 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc6037fe640 1 -- 172.21.15.66:0/3340272916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc608014440 con 0x7fc610108060 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc6037fe640 1 -- 172.21.15.66:0/3340272916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080075d0 con 0x7fc610108060 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61019d9c0 con 0x7fc610108060 2024-04-21T09:13:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.787+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6101a5250 con 0x7fc610108060 2024-04-21T09:13:08.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc6037fe640 1 -- 172.21.15.66:0/3340272916 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc60801c070 con 0x7fc610108060 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 Processor -- start 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 -- start start 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb801cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb8025950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb8028280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556fb7f7d1e0 con 0x556fb8025e90 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556fb7f1a390 con 0x556fb801d2b0 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc61abfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556fb7f791a0 con 0x556fb801c990 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.790+0000 7fc615910640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb8028280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb8028280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42086/0 (socket says 172.21.15.66:42086) 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 -- 172.21.15.66:0/1015681241 learned_addr learned my addr 172.21.15.66:0/1015681241 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc616111640 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb8025950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc6037fe640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc5fc0a3d20 0x7fc5fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb801cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc616912640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc5fc0a3d20 0x7fc5fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 -- 172.21.15.66:0/1015681241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 msgr2=0x556fb801cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb801cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 -- 172.21.15.66:0/1015681241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 msgr2=0x556fb8025950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb8025950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 -- 172.21.15.66:0/1015681241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb7d597d0 con 0x556fb8025e90 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc6037fe640 1 -- 172.21.15.66:0/3340272916 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc6080cffb0 con 0x7fc610108060 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc616111640 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb8025950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc615910640 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb8028280 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009930 tx=0x7fc60c009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a762ef46830cf6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc616912640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc5fc0a3d20 0x7fc5fc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc60809baa0 tx=0x7fc60800e040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc600bfe640 1 -- 172.21.15.66:0/1015681241 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c047020 con 0x556fb8025e90 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc600bfe640 1 -- 172.21.15.66:0/1015681241 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60c02cc90 con 0x556fb8025e90 2024-04-21T09:13:08.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.791+0000 7fc600bfe640 1 -- 172.21.15.66:0/1015681241 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0048c0 con 0x556fb8025e90 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 -- 172.21.15.66:0/1015681241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 msgr2=0x556fb8028280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb8028280 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009930 tx=0x7fc60c009600 comp rx=0 tx=0).stop 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 -- 172.21.15.66:0/1015681241 shutdown_connections 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb8028280 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb8025950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 --2- 172.21.15.66:0/1015681241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb801cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 -- 172.21.15.66:0/1015681241 >> 172.21.15.66:0/1015681241 conn(0x556fb80170c0 msgr2=0x556fb80174a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 -- 172.21.15.66:0/1015681241 shutdown_connections 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 -- 172.21.15.66:0/1015681241 wait complete. 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.792+0000 7fc61abfd740 1 -- start start 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 1 Processor -- start 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8022f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8023460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 0x556fb80239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556fb8028c20 con 0x556fb801d2b0 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556fb7f362f0 con 0x556fb801c990 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556fb8028da0 con 0x556fb8025e90 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc61abfd740 10 client.0 _ll_register_callbacks cb 0x556fb7f43580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8023460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8023460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42088/0 (socket says 172.21.15.66:42088) 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 -- 172.21.15.66:0/3286481079 learned_addr learned my addr 172.21.15.66:0/3286481079 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc615910640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8022f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc615910640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 0x556fb80239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 -- 172.21.15.66:0/3286481079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 msgr2=0x556fb80239a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 0x556fb80239a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 -- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 msgr2=0x556fb8022f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8022f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 -- 172.21.15.66:0/3286481079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb7c8f3f0 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc615910640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8022f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc616111640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8023460 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc604010820 tx=0x7fc604010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdae486583aaa8c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.793+0000 7fc5f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/3286481079 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60401d060 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc60c02cc90 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/3286481079 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc60400eda0 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/3286481079 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc604024050 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 10 client.17073 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556fb7c4f0f0 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/3286481079 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60c02cc90 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/3286481079 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc60400d530 con 0x556fb801d2b0 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 10 client.17073 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:08.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 20 client.17073 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb8035f30 0x556fb8038340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 4 client.17073 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fb7f8ac00 con 0x556fb8035f30 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.794+0000 7fc615910640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb8035f30 0x556fb8038340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.795+0000 7fc5f9ffb640 10 client.17073 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.795+0000 7fc615910640 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb8035f30 0x556fb8038340 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185ec9198c4e8885 server_cookie=e22a2d8cf583ca3c in_seq=0 out_seq=0 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.795+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/3286481079 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fb7f8ac00 con 0x556fb8035f30 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.795+0000 7fc5f9ffb640 10 client.17073 handle_command_reply: tid=0 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.795+0000 7fc61abfd740 1 client.17073 shutdown 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 msgr2=0x556fb8023460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8023460 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc604010820 tx=0x7fc604010ce0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 shutdown_connections 2024-04-21T09:13:08.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb8035f30 0x556fb8038340 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 0x556fb80239a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8023460 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc5f9ffb640 0 client.17073 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 --2- 172.21.15.66:0/3286481079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8022f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 >> 172.21.15.66:0/3286481079 conn(0x556fb80170c0 msgr2=0x556fb8018e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 shutdown_connections 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 1 -- 172.21.15.66:0/3286481079 wait complete. 2024-04-21T09:13:08.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.796+0000 7fc61abfd740 20 client.17073 trim_cache size 0 max 16384 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 Processor -- start 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 -- start start 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8031b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8011bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 0x556fb803dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101af950 con 0x556fb801d2b0 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101ad220 con 0x556fb801c990 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.813+0000 7fc61abfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101ab050 con 0x556fb8025e90 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8011bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8011bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42098/0 (socket says 172.21.15.66:42098) 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 -- 172.21.15.66:0/2737368431 learned_addr learned my addr 172.21.15.66:0/2737368431 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 -- 172.21.15.66:0/2737368431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 msgr2=0x556fb803dc50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 --2- 172.21.15.66:0/2737368431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 0x556fb803dc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 -- 172.21.15.66:0/2737368431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 msgr2=0x556fb8031b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 --2- 172.21.15.66:0/2737368431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8031b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 -- 172.21.15.66:0/2737368431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb8028280 con 0x556fb801d2b0 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc616111640 1 --2- 172.21.15.66:0/2737368431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8011bf0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc604000c00 tx=0x7fc604010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4947f0caff04cd72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc5fbfff640 1 -- 172.21.15.66:0/2737368431 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60402c020 con 0x556fb801d2b0 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc5fbfff640 1 -- 172.21.15.66:0/2737368431 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60400dd70 con 0x556fb801d2b0 2024-04-21T09:13:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.814+0000 7fc5fbfff640 1 -- 172.21.15.66:0/2737368431 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604005530 con 0x556fb801d2b0 2024-04-21T09:13:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- 172.21.15.66:0/2737368431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 msgr2=0x556fb8011bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 --2- 172.21.15.66:0/2737368431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8011bf0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc604000c00 tx=0x7fc604010430 comp rx=0 tx=0).stop 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- 172.21.15.66:0/2737368431 shutdown_connections 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 --2- 172.21.15.66:0/2737368431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb8025e90 0x556fb803dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 --2- 172.21.15.66:0/2737368431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb801d2b0 0x556fb8011bf0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 --2- 172.21.15.66:0/2737368431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801c990 0x556fb8031b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- 172.21.15.66:0/2737368431 >> 172.21.15.66:0/2737368431 conn(0x556fb80170c0 msgr2=0x556fb80114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- 172.21.15.66:0/2737368431 shutdown_connections 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- 172.21.15.66:0/2737368431 wait complete. 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- start start 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 Processor -- start 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb8029d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb802a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb802a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101ab6e0 con 0x556fb8025e90 2024-04-21T09:13:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101acb50 con 0x556fb801d2b0 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.815+0000 7fc61abfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101abdb0 con 0x556fb801c990 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc61abfd740 10 client.0 _ll_register_callbacks cb 0x7fc60400b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb802a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb802a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42106/0 (socket says 172.21.15.66:42106) 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 -- 172.21.15.66:0/11707262 learned_addr learned my addr 172.21.15.66:0/11707262 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb8029d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 -- 172.21.15.66:0/11707262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 msgr2=0x556fb8029d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc616111640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb802a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb8029d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 -- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 msgr2=0x556fb802a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb802a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 -- 172.21.15.66:0/11707262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb809b7b0 con 0x556fb8025e90 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc616111640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb802a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc615910640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb802a810 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009900 tx=0x7fc60c045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553f4a64075e0af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc5f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/11707262 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c059060 con 0x556fb8025e90 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/11707262 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc60c0037f0 con 0x556fb8025e90 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc60400dd70 con 0x556fb8025e90 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/11707262 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc60c047070 con 0x556fb8025e90 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc61abfd740 10 client.17079 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.816+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556fb809cb20 con 0x556fb8025e90 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/11707262 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60400dd70 con 0x556fb8025e90 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/11707262 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc60c0419a0 con 0x556fb8025e90 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc61abfd740 10 client.17079 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc61abfd740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc61abfd740 20 client.17079 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc61abfd740 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb809ceb0 0x556fb809f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc61abfd740 4 client.17079 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fb7f84a70 con 0x556fb809ceb0 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.817+0000 7fc616111640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb809ceb0 0x556fb809f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc616111640 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb809ceb0 0x556fb809f2a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edaef6273c8ce236 server_cookie=4aa0e4a3678314e9 in_seq=0 out_seq=0 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc5f9ffb640 10 client.17079 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc5f9ffb640 1 -- 172.21.15.66:0/11707262 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556fb7f84a70 con 0x556fb809ceb0 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc5f9ffb640 10 client.17079 handle_command_reply: tid=0 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 client.17079 shutdown 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 msgr2=0x556fb802a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb802a810 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009900 tx=0x7fc60c045040 comp rx=0 tx=0).stop 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 shutdown_connections 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556fb809ceb0 0x556fb809f2a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556fb8025e90 0x556fb802a810 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fb801d2b0 0x556fb802a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc5f9ffb640 0 client.17079 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 --2- 172.21.15.66:0/11707262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556fb801c990 0x556fb8029d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 >> 172.21.15.66:0/11707262 conn(0x556fb80170c0 msgr2=0x556fb8011d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 shutdown_connections 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 1 -- 172.21.15.66:0/11707262 wait complete. 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.818+0000 7fc61abfd740 20 client.17079 trim_cache size 0 max 16384 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stdout: "eca9aeee-dad4-4cd5-b3d4-1ad84ebad14b": { 2024-04-21T09:13:08.824 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:13:08.825 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "eca9aeee-dad4-4cd5-b3d4-1ad84ebad14b", 2024-04-21T09:13:08.825 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:13:08.825 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:13:08.825 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:13:08.825 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc5fc0a3d20 msgr2=0x7fc5fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc5fc0a3d20 0x7fc5fc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc60809baa0 tx=0x7fc60800e040 comp rx=0 tx=0).stop 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc610108060 msgr2=0x7fc61019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc610108060 0x7fc61019cfa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b970 tx=0x7fc60800be30 comp rx=0 tx=0).stop 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 shutdown_connections 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc5fc0a3d20 0x7fc5fc0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc610108060 0x7fc61019cfa0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61006af70 0x7fc610198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 --2- 172.21.15.66:0/3340272916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc61006a5b0 0x7fc6101986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 >> 172.21.15.66:0/3340272916 conn(0x7fc610077c40 msgr2=0x7fc6100ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 shutdown_connections 2024-04-21T09:13:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:08.819+0000 7fc618375640 1 -- 172.21.15.66:0/3340272916 wait complete. 2024-04-21T09:13:08.829 INFO:tasks.cephfs.filesystem:scrub status for tag:eca9aeee-dad4-4cd5-b3d4-1ad84ebad14b - {'path': '/', 'tag': 'eca9aeee-dad4-4cd5-b3d4-1ad84ebad14b', 'options': 'recursive,force'} 2024-04-21T09:13:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:09 smithi186 ceph-mon[31505]: pgmap v151: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:13:09.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:09 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:13:09.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:09 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:09.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:09 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:13:09.839 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:09 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:09 smithi083 ceph-mon[31413]: pgmap v151: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:13:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:09 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:13:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:09 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:09.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:09 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:13:09.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:09 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:09.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:09 smithi066 ceph-mon[27953]: pgmap v151: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:13:09.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:09 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:13:09.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:09 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:09.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:09 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:13:09.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:09 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:11 smithi186 ceph-mon[31505]: pgmap v152: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:13:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:11 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:13:11.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:11 smithi083 ceph-mon[31413]: pgmap v152: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:13:11.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:11 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:13:11.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:11 smithi066 ceph-mon[27953]: pgmap v152: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:13:11.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:11 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:13:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:13 smithi186 ceph-mon[31505]: pgmap v153: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:13:13.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:13 smithi083 ceph-mon[31413]: pgmap v153: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:13:13.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:13 smithi066 ceph-mon[27953]: pgmap v153: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:13:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:15 smithi083 ceph-mon[31413]: pgmap v154: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:13:15.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:15 smithi066 ceph-mon[27953]: pgmap v154: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:13:16.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:15 smithi186 ceph-mon[31505]: pgmap v154: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-21T09:13:17.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:17 smithi083 ceph-mon[31413]: pgmap v155: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T09:13:17.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:17 smithi066 ceph-mon[27953]: pgmap v155: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T09:13:18.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:17 smithi186 ceph-mon[31505]: pgmap v155: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T09:13:19.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:19 smithi083 ceph-mon[31413]: pgmap v156: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:13:19.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:19 smithi066 ceph-mon[27953]: pgmap v156: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:13:19.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:19 smithi186 ceph-mon[31505]: pgmap v156: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:13:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:21 smithi083 ceph-mon[31413]: pgmap v157: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:21.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:21 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:13:21.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:21 smithi066 ceph-mon[27953]: pgmap v157: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:21.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:21 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:13:22.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:21 smithi186 ceph-mon[31505]: pgmap v157: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:22.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:21 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:13:22.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:22 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:13:22.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:22 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:13:22.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:22 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:13:22.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:22 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:13:22.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:22 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:13:22.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:22 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:13:23.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:22 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:13:23.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:22 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:13:23.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:22 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:13:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:23 smithi083 ceph-mon[31413]: pgmap v158: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:23.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:23 smithi066 ceph-mon[27953]: pgmap v158: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:24.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:23 smithi186 ceph-mon[31505]: pgmap v158: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:25.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:25 smithi083 ceph-mon[31413]: pgmap v159: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:25.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:25 smithi066 ceph-mon[27953]: pgmap v159: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:26.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:25 smithi186 ceph-mon[31505]: pgmap v159: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:27.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:27 smithi083 ceph-mon[31413]: pgmap v160: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:27.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:27 smithi066 ceph-mon[27953]: pgmap v160: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:27 smithi186 ceph-mon[31505]: pgmap v160: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:29.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:29 smithi066 ceph-mon[27953]: pgmap v161: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:30.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:29 smithi186 ceph-mon[31505]: pgmap v161: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:30.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:29 smithi083 ceph-mon[31413]: pgmap v161: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:31.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:31 smithi066 ceph-mon[27953]: pgmap v162: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:32.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:31 smithi186 ceph-mon[31505]: pgmap v162: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:32.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:31 smithi083 ceph-mon[31413]: pgmap v162: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:33.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:33 smithi066 ceph-mon[27953]: pgmap v163: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:34.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:33 smithi186 ceph-mon[31505]: pgmap v163: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:34.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:33 smithi083 ceph-mon[31413]: pgmap v163: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:35.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:35 smithi066 ceph-mon[27953]: pgmap v164: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:36.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:35 smithi186 ceph-mon[31505]: pgmap v164: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:36.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:35 smithi083 ceph-mon[31413]: pgmap v164: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:38.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:37 smithi186 ceph-mon[31505]: pgmap v165: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:38.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:37 smithi083 ceph-mon[31413]: pgmap v165: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:38.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:37 smithi066 ceph-mon[27953]: pgmap v165: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:38.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:13:38.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:13:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 -- 172.21.15.66:0/2892253381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101a30 msgr2=0x7fe868105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/2892253381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101a30 0x7fe868105a70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe850009900 tx=0x7fe85002bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 -- 172.21.15.66:0/2892253381 shutdown_connections 2024-04-21T09:13:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/2892253381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe868106150 0x7fe86810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/2892253381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101a30 0x7fe868105a70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/2892253381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe868101080 0x7fe868101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 -- 172.21.15.66:0/2892253381 >> 172.21.15.66:0/2892253381 conn(0x7fe868078000 msgr2=0x7fe868078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 -- 172.21.15.66:0/2892253381 shutdown_connections 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.948+0000 7fe86dc7a640 1 -- 172.21.15.66:0/2892253381 wait complete. 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 Processor -- start 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 -- start start 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101080 0x7fe868102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe868106150 0x7fe868103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8681047b0 0x7fe868103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe868111090 con 0x7fe868101080 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe868110f10 con 0x7fe8681047b0 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe86dc7a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe868111210 con 0x7fe868106150 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101080 0x7fe868102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101080 0x7fe868102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45396/0 (socket says 172.21.15.66:45396) 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe8677fe640 1 -- 172.21.15.66:0/3191374084 learned_addr learned my addr 172.21.15.66:0/3191374084 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe867fff640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8681047b0 0x7fe868103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.949+0000 7fe866ffd640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe868106150 0x7fe868103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe8677fe640 1 -- 172.21.15.66:0/3191374084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe868106150 msgr2=0x7fe868103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe8677fe640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe868106150 0x7fe868103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe8677fe640 1 -- 172.21.15.66:0/3191374084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8681047b0 msgr2=0x7fe868103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe8677fe640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8681047b0 0x7fe868103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe8677fe640 1 -- 172.21.15.66:0/3191374084 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8681ab3e0 con 0x7fe868101080 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe8677fe640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101080 0x7fe868102b10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe85800d950 tx=0x7fe85800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe864ff9640 1 -- 172.21.15.66:0/3191374084 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858017070 con 0x7fe868101080 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8681ab6d0 con 0x7fe868101080 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe864ff9640 1 -- 172.21.15.66:0/3191374084 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe858009c30 con 0x7fe868101080 2024-04-21T09:13:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe864ff9640 1 -- 172.21.15.66:0/3191374084 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85800f620 con 0x7fe868101080 2024-04-21T09:13:38.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.950+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8681abc40 con 0x7fe868101080 2024-04-21T09:13:38.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe864ff9640 1 -- 172.21.15.66:0/3191374084 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe858010460 con 0x7fe868101080 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 Processor -- start 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 -- start start 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabd8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabe1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabe3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555bfab3d1e0 con 0x555bfabd8580 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555bfaada390 con 0x555bfabd8ea0 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe87086a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555bfab391a0 con 0x555bfabe1a80 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe864ff9640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe8540a3d20 0x7fe8540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe867fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabd8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabe1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe867fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabd8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45412/0 (socket says 172.21.15.66:45412) 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe867fff640 1 -- 172.21.15.66:0/2010515994 learned_addr learned my addr 172.21.15.66:0/2010515994 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe867fff640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe8540a3d20 0x7fe8540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.953+0000 7fe866ffd640 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabe3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe864ff9640 1 -- 172.21.15.66:0/3191374084 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe8580d0700 con 0x7fe868101080 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe867fff640 1 -- 172.21.15.66:0/2010515994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 msgr2=0x555bfabe3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe867fff640 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabe3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe867fff640 1 -- 172.21.15.66:0/2010515994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 msgr2=0x555bfabe1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe867fff640 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabe1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe867fff640 1 -- 172.21.15.66:0/2010515994 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfa9197d0 con 0x555bfabd8580 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe8677fe640 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabe1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe867fff640 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabd8960 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe860012d20 tx=0x7fe860010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f5be938590f473 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe867fff640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe8540a3d20 0x7fe8540a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe860010c40 tx=0x7fe860017000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:38.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe85dffb640 1 -- 172.21.15.66:0/2010515994 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860012800 con 0x555bfabd8580 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe85dffb640 1 -- 172.21.15.66:0/2010515994 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe860009ce0 con 0x555bfabd8580 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.954+0000 7fe85dffb640 1 -- 172.21.15.66:0/2010515994 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86001e890 con 0x555bfabd8580 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 -- 172.21.15.66:0/2010515994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 msgr2=0x555bfabd8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabd8960 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe860012d20 tx=0x7fe860010740 comp rx=0 tx=0).stop 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 -- 172.21.15.66:0/2010515994 shutdown_connections 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabe3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabe1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 --2- 172.21.15.66:0/2010515994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabd8960 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 -- 172.21.15.66:0/2010515994 >> 172.21.15.66:0/2010515994 conn(0x555bfabd2cb0 msgr2=0x555bfabd3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 -- 172.21.15.66:0/2010515994 shutdown_connections 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 -- 172.21.15.66:0/2010515994 wait complete. 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.955+0000 7fe87086a740 1 -- start start 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 1 Processor -- start 2024-04-21T09:13:38.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabdea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8ea0 0x555bfabdef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabdf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555bfabe4810 con 0x555bfabd8ea0 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555bfaaf62f0 con 0x555bfabd8580 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555bfabe4990 con 0x555bfabe1a80 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe87086a740 10 client.0 _ll_register_callbacks cb 0x555bfab03580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8ea0 0x555bfabdef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8ea0 0x555bfabdef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45424/0 (socket says 172.21.15.66:45424) 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 -- 172.21.15.66:0/2787561938 learned_addr learned my addr 172.21.15.66:0/2787561938 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe8677fe640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabdea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe867fff640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabdf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 -- 172.21.15.66:0/2787561938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 msgr2=0x555bfabdf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabdf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 -- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 msgr2=0x555bfabdea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabdea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 -- 172.21.15.66:0/2787561938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfa84f3f0 con 0x555bfabd8ea0 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe8677fe640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabdea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe866ffd640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8ea0 0x555bfabdef80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe850009900 tx=0x7fe85002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5291d9e481075c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.956+0000 7fe846ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe846ffd640 1 -- 172.21.15.66:0/2787561938 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85002c610 con 0x555bfabd8ea0 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe846ffd640 1 -- 172.21.15.66:0/2787561938 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe850005030 con 0x555bfabd8ea0 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe860009ce0 con 0x555bfabd8ea0 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe846ffd640 1 -- 172.21.15.66:0/2787561938 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe850052050 con 0x555bfabd8ea0 2024-04-21T09:13:38.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 10 client.17091 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bfa80f0f0 con 0x555bfabd8ea0 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe846ffd640 1 -- 172.21.15.66:0/2787561938 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe860009ce0 con 0x555bfabd8ea0 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe846ffd640 1 -- 172.21.15.66:0/2787561938 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe8500035f0 con 0x555bfabd8ea0 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 10 client.17091 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 20 client.17091 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfabf3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 4 client.17091 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bfab4ac00 con 0x555bfabf1af0 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.957+0000 7fe8677fe640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfabf3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.958+0000 7fe846ffd640 10 client.17091 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.958+0000 7fe8677fe640 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfabf3f30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39c2f05b73923b2 server_cookie=4a96609602e60077 in_seq=0 out_seq=0 2024-04-21T09:13:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe846ffd640 1 -- 172.21.15.66:0/2787561938 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bfab4ac00 con 0x555bfabf1af0 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe846ffd640 10 client.17091 handle_command_reply: tid=0 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 client.17091 shutdown 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8ea0 msgr2=0x555bfabdef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8ea0 0x555bfabdef80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe850009900 tx=0x7fe85002c9f0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 shutdown_connections 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfabf3f30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabdf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8ea0 0x555bfabdef80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe846ffd640 0 client.17091 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 --2- 172.21.15.66:0/2787561938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabdea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 >> 172.21.15.66:0/2787561938 conn(0x555bfabd2cb0 msgr2=0x555bfabd4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 shutdown_connections 2024-04-21T09:13:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 1 -- 172.21.15.66:0/2787561938 wait complete. 2024-04-21T09:13:38.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.959+0000 7fe87086a740 20 client.17091 trim_cache size 0 max 16384 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.977+0000 7fe87086a740 1 Processor -- start 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.977+0000 7fe87086a740 1 -- start start 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe87086a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabe1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe87086a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabdd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe87086a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe87086a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681b80b0 con 0x555bfabd8580 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe87086a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681b5980 con 0x555bfabd8ea0 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe87086a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681b37b0 con 0x555bfabe1a80 2024-04-21T09:13:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabe1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabe1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45430/0 (socket says 172.21.15.66:45430) 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe8677fe640 1 -- 172.21.15.66:0/4272295652 learned_addr learned my addr 172.21.15.66:0/4272295652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe866ffd640 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabdd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe867fff640 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe866ffd640 1 -- 172.21.15.66:0/4272295652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 msgr2=0x555bfabed6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe866ffd640 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabed6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe866ffd640 1 -- 172.21.15.66:0/4272295652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 msgr2=0x555bfabe1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe866ffd640 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabe1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.978+0000 7fe866ffd640 1 -- 172.21.15.66:0/4272295652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfabd2ae0 con 0x555bfabd8ea0 2024-04-21T09:13:38.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.979+0000 7fe866ffd640 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabdd980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8500039e0 tx=0x7fe85002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f7d7f4e32722fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:38.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.979+0000 7fe85d7fa640 1 -- 172.21.15.66:0/4272295652 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85005b070 con 0x555bfabd8ea0 2024-04-21T09:13:38.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.979+0000 7fe85d7fa640 1 -- 172.21.15.66:0/4272295652 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8500035f0 con 0x555bfabd8ea0 2024-04-21T09:13:38.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.979+0000 7fe85d7fa640 1 -- 172.21.15.66:0/4272295652 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850005070 con 0x555bfabd8ea0 2024-04-21T09:13:38.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 -- 172.21.15.66:0/4272295652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 msgr2=0x555bfabdd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabdd980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8500039e0 tx=0x7fe85002bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 -- 172.21.15.66:0/4272295652 shutdown_connections 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabe1a80 0x555bfabed6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8ea0 0x555bfabdd980 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 --2- 172.21.15.66:0/4272295652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabd8580 0x555bfabe1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 -- 172.21.15.66:0/4272295652 >> 172.21.15.66:0/4272295652 conn(0x555bfabd2cb0 msgr2=0x555bfabdf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 -- 172.21.15.66:0/4272295652 shutdown_connections 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 -- 172.21.15.66:0/4272295652 wait complete. 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.980+0000 7fe87086a740 1 -- start start 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 1 Processor -- start 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabe5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabd8ea0 0x555bfabe62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabe1a80 0x555bfac57070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681b3e40 con 0x555bfabe1a80 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681b52b0 con 0x555bfabd8580 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681b4510 con 0x555bfabd8ea0 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabe5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe866ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabd8ea0 0x555bfabe62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe87086a740 10 client.0 _ll_register_callbacks cb 0x7fe850004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe866ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabd8ea0 0x555bfabe62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:45656/0 (socket says 172.21.15.66:45656) 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe866ffd640 1 -- 172.21.15.66:0/2847049174 learned_addr learned my addr 172.21.15.66:0/2847049174 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe867fff640 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabe1a80 0x555bfac57070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe866ffd640 1 -- 172.21.15.66:0/2847049174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 msgr2=0x555bfabe5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe866ffd640 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabe5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe866ffd640 1 -- 172.21.15.66:0/2847049174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabe1a80 msgr2=0x555bfac57070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:38.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.981+0000 7fe866ffd640 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabe1a80 0x555bfac57070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe866ffd640 1 -- 172.21.15.66:0/2847049174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfac575b0 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe866ffd640 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabd8ea0 0x555bfabe62d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe850004460 tx=0x7fe850005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde01d77d1e060c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe846ffd640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe846ffd640 1 -- 172.21.15.66:0/2847049174 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85005b070 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8500035f0 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe846ffd640 1 -- 172.21.15.66:0/2847049174 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe850004ad0 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe846ffd640 1 -- 172.21.15.66:0/2847049174 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe850066050 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe87086a740 10 client.26257 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bfac588c0 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.982+0000 7fe846ffd640 1 -- 172.21.15.66:0/2847049174 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8500035f0 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe846ffd640 1 -- 172.21.15.66:0/2847049174 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe850034440 con 0x555bfabd8ea0 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe87086a740 10 client.26257 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe87086a740 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:38.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe87086a740 20 client.26257 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe87086a740 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfac5af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe87086a740 4 client.26257 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bfab44a70 con 0x555bfabf1af0 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe8677fe640 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfac5af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe8677fe640 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfac5af60 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16da576ed5db6c2e server_cookie=4761d0feb039f4e2 in_seq=0 out_seq=0 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe846ffd640 10 client.26257 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe846ffd640 1 -- 172.21.15.66:0/2847049174 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555bfab44a70 con 0x555bfabf1af0 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe846ffd640 10 client.26257 handle_command_reply: tid=0 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.983+0000 7fe87086a740 1 client.26257 shutdown 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabd8ea0 msgr2=0x555bfabe62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabd8ea0 0x555bfabe62d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe850004460 tx=0x7fe850005d50 comp rx=0 tx=0).stop 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 shutdown_connections 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555bfabf1af0 0x555bfac5af60 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bfabe1a80 0x555bfac57070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe846ffd640 0 client.26257 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bfabd8ea0 0x555bfabe62d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 --2- 172.21.15.66:0/2847049174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bfabd8580 0x555bfabe5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 >> 172.21.15.66:0/2847049174 conn(0x555bfabd2cb0 msgr2=0x555bfabd3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 shutdown_connections 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 1 -- 172.21.15.66:0/2847049174 wait complete. 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe87086a740 20 client.26257 trim_cache size 0 max 16384 2024-04-21T09:13:38.989 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:13:38.990 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:13:38.990 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:13:38.990 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe8540a3d20 msgr2=0x7fe8540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe8540a3d20 0x7fe8540a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe860010c40 tx=0x7fe860017000 comp rx=0 tx=0).stop 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101080 msgr2=0x7fe868102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.984+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101080 0x7fe868102b10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe85800d950 tx=0x7fe85800de10 comp rx=0 tx=0).stop 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 shutdown_connections 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe8540a3d20 0x7fe8540a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8681047b0 0x7fe868103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe868106150 0x7fe868103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 --2- 172.21.15.66:0/3191374084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe868101080 0x7fe868102b10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 >> 172.21.15.66:0/3191374084 conn(0x7fe868078000 msgr2=0x7fe8681000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 shutdown_connections 2024-04-21T09:13:38.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:38.985+0000 7fe86dc7a640 1 -- 172.21.15.66:0/3191374084 wait complete. 2024-04-21T09:13:38.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:13:38.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:13:39.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 -- 172.21.15.66:0/2064772688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 msgr2=0x7fb2fc10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 --2- 172.21.15.66:0/2064772688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc10dd50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009ca0 tx=0x7fb2f402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 -- 172.21.15.66:0/2064772688 shutdown_connections 2024-04-21T09:13:39.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 --2- 172.21.15.66:0/2064772688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc10dd50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 --2- 172.21.15.66:0/2064772688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2fc101a60 0x7fb2fc105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 --2- 172.21.15.66:0/2064772688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2fc1010b0 0x7fb2fc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 -- 172.21.15.66:0/2064772688 >> 172.21.15.66:0/2064772688 conn(0x7fb2fc0771d0 msgr2=0x7fb2fc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 -- 172.21.15.66:0/2064772688 shutdown_connections 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.099+0000 7fb30217e640 1 -- 172.21.15.66:0/2064772688 wait complete. 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 Processor -- start 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 -- start start 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2fc1010b0 0x7fb2fc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2fc19cee0 0x7fb2fc19d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc112200 con 0x7fb2fc106180 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc112080 con 0x7fb2fc1010b0 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb30217e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc112380 con 0x7fb2fc19cee0 2024-04-21T09:13:39.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb2faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb2faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45466/0 (socket says 172.21.15.66:45466) 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.100+0000 7fb2faffd640 1 -- 172.21.15.66:0/1445804932 learned_addr learned my addr 172.21.15.66:0/1445804932 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2fb7fe640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2fc1010b0 0x7fb2fc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2fbfff640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2fc19cee0 0x7fb2fc19d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2faffd640 1 -- 172.21.15.66:0/1445804932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2fc19cee0 msgr2=0x7fb2fc19d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2faffd640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2fc19cee0 0x7fb2fc19d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2faffd640 1 -- 172.21.15.66:0/1445804932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2fc1010b0 msgr2=0x7fb2fc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2faffd640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2fc1010b0 0x7fb2fc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2faffd640 1 -- 172.21.15.66:0/1445804932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc19d9d0 con 0x7fb2fc106180 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2fb7fe640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2fc1010b0 0x7fb2fc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2fbfff640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2fc19cee0 0x7fb2fc19d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2faffd640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc198b60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000d770 tx=0x7fb2f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2f8ff9640 1 -- 172.21.15.66:0/1445804932 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0017070 con 0x7fb2fc106180 2024-04-21T09:13:39.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2f8ff9640 1 -- 172.21.15.66:0/1445804932 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f0009c30 con 0x7fb2fc106180 2024-04-21T09:13:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb2f8ff9640 1 -- 172.21.15.66:0/1445804932 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000f620 con 0x7fb2fc106180 2024-04-21T09:13:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc19dcc0 con 0x7fb2fc106180 2024-04-21T09:13:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.101+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc1a54b0 con 0x7fb2fc106180 2024-04-21T09:13:39.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.102+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc069230 con 0x7fb2fc106180 2024-04-21T09:13:39.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.103+0000 7fb2f8ff9640 1 -- 172.21.15.66:0/1445804932 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb2f001a070 con 0x7fb2fc106180 2024-04-21T09:13:39.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.104+0000 7fb2f8ff9640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb2e40a3d20 0x7fb2e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.104+0000 7fb2fb7fe640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb2e40a3d20 0x7fb2e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.104+0000 7fb2f8ff9640 1 -- 172.21.15.66:0/1445804932 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb2f00d0700 con 0x7fb2fc106180 2024-04-21T09:13:39.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.104+0000 7fb2fb7fe640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb2e40a3d20 0x7fb2e40a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00baa0 tx=0x7fb2ec005d90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.109+0000 7fb2f8ff9640 1 -- 172.21.15.66:0/1445804932 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2f0009de0 con 0x7fb2fc106180 2024-04-21T09:13:39.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.337+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2fc101a60 con 0x7fb2fc106180 2024-04-21T09:13:39.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.338+0000 7fb2f8ff9640 1 -- 172.21.15.66:0/1445804932 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7fb2f009ca30 con 0x7fb2fc106180 2024-04-21T09:13:39.337 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:13:39.338 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:13:39.338 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:13:39.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb2e40a3d20 msgr2=0x7fb2e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb2e40a3d20 0x7fb2e40a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00baa0 tx=0x7fb2ec005d90 comp rx=0 tx=0).stop 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 msgr2=0x7fb2fc198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc198b60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000d770 tx=0x7fb2f000dc30 comp rx=0 tx=0).stop 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 shutdown_connections 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb2e40a3d20 0x7fb2e40a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2fc19cee0 0x7fb2fc19d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2fc106180 0x7fb2fc198b60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 --2- 172.21.15.66:0/1445804932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2fc1010b0 0x7fb2fc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 >> 172.21.15.66:0/1445804932 conn(0x7fb2fc0771d0 msgr2=0x7fb2fc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 shutdown_connections 2024-04-21T09:13:39.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.340+0000 7fb30217e640 1 -- 172.21.15.66:0/1445804932 wait complete. 2024-04-21T09:13:39.350 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:13:39.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 -- 172.21.15.66:0/2779778694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 msgr2=0x7fbe7010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 --2- 172.21.15.66:0/2779778694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe7010ee40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009ca0 tx=0x7fbe6802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 -- 172.21.15.66:0/2779778694 shutdown_connections 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 --2- 172.21.15.66:0/2779778694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe7010ee40 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 --2- 172.21.15.66:0/2779778694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7006b1f0 0x7fbe70106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 --2- 172.21.15.66:0/2779778694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7006a840 0x7fbe7006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 -- 172.21.15.66:0/2779778694 >> 172.21.15.66:0/2779778694 conn(0x7fbe70077eb0 msgr2=0x7fbe700782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 -- 172.21.15.66:0/2779778694 shutdown_connections 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.455+0000 7fbe75c32640 1 -- 172.21.15.66:0/2779778694 wait complete. 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 Processor -- start 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 -- start start 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7006a840 0x7fbe701986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe70198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7019cf90 0x7fbe7019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701121d0 con 0x7fbe70107270 2024-04-21T09:13:39.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112050 con 0x7fbe7019cf90 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe75c32640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112350 con 0x7fbe7006a840 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe70198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe70198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45484/0 (socket says 172.21.15.66:45484) 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 -- 172.21.15.66:0/284078099 learned_addr learned my addr 172.21.15.66:0/284078099 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7019cf90 0x7fbe7019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7006a840 0x7fbe701986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 -- 172.21.15.66:0/284078099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7006a840 msgr2=0x7fbe701986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7006a840 0x7fbe701986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 -- 172.21.15.66:0/284078099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7019cf90 msgr2=0x7fbe7019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7019cf90 0x7fbe7019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 -- 172.21.15.66:0/284078099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7019da80 con 0x7fbe70107270 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.456+0000 7fbe6effd640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe70198c10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400e9f0 tx=0x7fbe6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.457+0000 7fbe6cff9640 1 -- 172.21.15.66:0/284078099 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6400ccc0 con 0x7fbe70107270 2024-04-21T09:13:39.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.457+0000 7fbe6cff9640 1 -- 172.21.15.66:0/284078099 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640054d0 con 0x7fbe70107270 2024-04-21T09:13:39.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.457+0000 7fbe6cff9640 1 -- 172.21.15.66:0/284078099 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64010430 con 0x7fbe70107270 2024-04-21T09:13:39.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.457+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7019dd70 con 0x7fbe70107270 2024-04-21T09:13:39.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.457+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a5560 con 0x7fbe70107270 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe6cff9640 1 -- 172.21.15.66:0/284078099 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbe640046c0 con 0x7fbe70107270 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 Processor -- start 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 -- start start 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x5577596798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759682430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759684e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5577595de1f0 con 0x557759682a00 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55775957b4b0 con 0x557759679e80 2024-04-21T09:13:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.459+0000 7fbe7881b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5577595da1b0 con 0x5577596794d0 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759684e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6cff9640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbe580a3da0 0x7fbe580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759684e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45500/0 (socket says 172.21.15.66:45500) 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 -- 172.21.15.66:0/750753401 learned_addr learned my addr 172.21.15.66:0/750753401 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbe580a3da0 0x7fbe580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759682430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x5577596798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6cff9640 1 -- 172.21.15.66:0/284078099 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbe640d0ed0 con 0x7fbe70107270 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 -- 172.21.15.66:0/750753401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 msgr2=0x5577596798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x5577596798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 -- 172.21.15.66:0/750753401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 msgr2=0x557759682430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759682430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 -- 172.21.15.66:0/750753401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775956d1b0 con 0x557759682a00 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x5577596798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759682430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6effd640 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759684e50 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbe64099490 tx=0x7fbe640d6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686d992a45107780 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe55ffb640 1 -- 172.21.15.66:0/750753401 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6409d1c0 con 0x557759682a00 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbe580a3da0 0x7fbe580a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe68004af0 tx=0x7fbe6802e040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe55ffb640 1 -- 172.21.15.66:0/750753401 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640a0e70 con 0x557759682a00 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.460+0000 7fbe55ffb640 1 -- 172.21.15.66:0/750753401 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe640ce4f0 con 0x557759682a00 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 -- 172.21.15.66:0/750753401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 msgr2=0x557759684e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759684e50 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbe64099490 tx=0x7fbe640d6040 comp rx=0 tx=0).stop 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 -- 172.21.15.66:0/750753401 shutdown_connections 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759684e50 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759682430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 --2- 172.21.15.66:0/750753401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x5577596798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 -- 172.21.15.66:0/750753401 >> 172.21.15.66:0/750753401 conn(0x557759673ba0 msgr2=0x557759673f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 -- 172.21.15.66:0/750753401 shutdown_connections 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 -- 172.21.15.66:0/750753401 wait complete. 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.461+0000 7fbe7881b740 1 -- start start 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 1 Processor -- start 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577596794d0 0x557759682380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759679e80 0x55775967f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759682a00 0x55775967fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5577596858b0 con 0x557759679e80 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557759597500 con 0x5577596794d0 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557759685a30 con 0x557759682a00 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe7881b740 10 client.0 _ll_register_callbacks cb 0x557759565b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759679e80 0x55775967f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759679e80 0x55775967f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45504/0 (socket says 172.21.15.66:45504) 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 -- 172.21.15.66:0/4118852277 learned_addr learned my addr 172.21.15.66:0/4118852277 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577596794d0 0x557759682380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 -- 172.21.15.66:0/4118852277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759682a00 msgr2=0x55775967fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759682a00 0x55775967fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 -- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577596794d0 msgr2=0x557759682380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577596794d0 0x557759682380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6effd640 1 -- 172.21.15.66:0/4118852277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577592f03f0 con 0x557759679e80 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.462+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577596794d0 0x557759682380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe6effd640 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759679e80 0x55775967f710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbe640a5200 tx=0x7fbe64098de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f978b690e3a1e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe4effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe4effd640 1 -- 172.21.15.66:0/4118852277 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6406d020 con 0x557759679e80 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe4effd640 1 -- 172.21.15.66:0/4118852277 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbe640991a0 con 0x557759679e80 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe640a0e70 con 0x557759679e80 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe4effd640 1 -- 172.21.15.66:0/4118852277 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe6409d1c0 con 0x557759679e80 2024-04-21T09:13:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 10 client.17115 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577593ba7d0 con 0x557759679e80 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe4effd640 1 -- 172.21.15.66:0/4118852277 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640a0e70 con 0x557759679e80 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe4effd640 1 -- 172.21.15.66:0/4118852277 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe64098810 con 0x557759679e80 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 10 client.17115 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 10 client.17115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 20 client.17115 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x557759692a00 0x557759694df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 4 client.17115 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.463+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55775957d930 con 0x557759692a00 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.464+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x557759692a00 0x557759694df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.464+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x557759692a00 0x557759694df0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a83a6fe48834063 server_cookie=e796e859aa88b60f in_seq=0 out_seq=0 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.464+0000 7fbe4effd640 10 client.17115 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:39.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe4effd640 1 -- 172.21.15.66:0/4118852277 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55775957d930 con 0x557759692a00 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe4effd640 10 client.17115 handle_command_reply: tid=0 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 client.17115 shutdown 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759679e80 msgr2=0x55775967f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759679e80 0x55775967f710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbe640a5200 tx=0x7fbe64098de0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 shutdown_connections 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x557759692a00 0x557759694df0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759682a00 0x55775967fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe4effd640 0 client.17115 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759679e80 0x55775967f710 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 --2- 172.21.15.66:0/4118852277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577596794d0 0x557759682380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 >> 172.21.15.66:0/4118852277 conn(0x557759673ba0 msgr2=0x5577596758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 shutdown_connections 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 1 -- 172.21.15.66:0/4118852277 wait complete. 2024-04-21T09:13:39.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.465+0000 7fbe7881b740 20 client.17115 trim_cache size 0 max 16384 2024-04-21T09:13:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.478+0000 7fbe7881b740 1 Processor -- start 2024-04-21T09:13:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe7881b740 1 -- start start 2024-04-21T09:13:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x55775968e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759681e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759682380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe7881b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701b1b80 con 0x557759682a00 2024-04-21T09:13:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe7881b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701af450 con 0x557759679e80 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe7881b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701ad280 con 0x5577596794d0 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759682380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759682380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45520/0 (socket says 172.21.15.66:45520) 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 -- 172.21.15.66:0/1078576698 learned_addr learned my addr 172.21.15.66:0/1078576698 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x55775968e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6effd640 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759681e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 -- 172.21.15.66:0/1078576698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 msgr2=0x55775968e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x55775968e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 -- 172.21.15.66:0/1078576698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 msgr2=0x557759681e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759681e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 -- 172.21.15.66:0/1078576698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577596733f0 con 0x557759682a00 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759682380 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbe68036000 tx=0x7fbe68005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cc2e8968d88781 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.479+0000 7fbe557fa640 1 -- 172.21.15.66:0/1078576698 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68044030 con 0x557759682a00 2024-04-21T09:13:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe557fa640 1 -- 172.21.15.66:0/1078576698 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe68050070 con 0x557759682a00 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe557fa640 1 -- 172.21.15.66:0/1078576698 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68039040 con 0x557759682a00 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 -- 172.21.15.66:0/1078576698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 msgr2=0x557759682380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759682380 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbe68036000 tx=0x7fbe68005c50 comp rx=0 tx=0).stop 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 -- 172.21.15.66:0/1078576698 shutdown_connections 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557759682a00 0x557759682380 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759679e80 0x557759681e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 --2- 172.21.15.66:0/1078576698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5577596794d0 0x55775968e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 -- 172.21.15.66:0/1078576698 >> 172.21.15.66:0/1078576698 conn(0x557759673ba0 msgr2=0x55775968dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 -- 172.21.15.66:0/1078576698 shutdown_connections 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 -- 172.21.15.66:0/1078576698 wait complete. 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.480+0000 7fbe7881b740 1 -- start start 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 1 Processor -- start 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577596794d0 0x557759698160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759679e80 0x5577596954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759682a00 0x557759695a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701ad910 con 0x5577596794d0 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701aed80 con 0x557759682a00 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701adfe0 con 0x557759679e80 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe7881b740 10 client.0 _ll_register_callbacks cb 0x7fbe640a5730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577596794d0 0x557759698160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577596794d0 0x557759698160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45534/0 (socket says 172.21.15.66:45534) 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 -- 172.21.15.66:0/3575700020 learned_addr learned my addr 172.21.15.66:0/3575700020 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6ffff640 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759682a00 0x557759695a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6effd640 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759679e80 0x5577596954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 -- 172.21.15.66:0/3575700020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759679e80 msgr2=0x5577596954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759679e80 0x5577596954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 -- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759682a00 msgr2=0x557759695a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759682a00 0x557759695a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 -- 172.21.15.66:0/3575700020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577596961e0 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe6f7fe640 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577596794d0 0x557759698160 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000bcb0 tx=0x7fbe6001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d63bd18c89a3d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.481+0000 7fbe4effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe4effd640 1 -- 172.21.15.66:0/3575700020 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60004440 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe4effd640 1 -- 172.21.15.66:0/3575700020 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbe600035f0 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe68050070 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe4effd640 1 -- 172.21.15.66:0/3575700020 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe600037f0 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 10 client.17121 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577596f8bc0 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe4effd640 1 -- 172.21.15.66:0/3575700020 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe68050070 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe4effd640 1 -- 172.21.15.66:0/3575700020 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe60022020 con 0x5577596794d0 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 10 client.17121 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:13:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 20 client.17121 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577596f92f0 0x5577596fb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 4 client.17121 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577595ebc10 con 0x5577596f92f0 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.482+0000 7fbe6effd640 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577596f92f0 0x5577596fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe4effd640 10 client.17121 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe6effd640 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577596f92f0 0x5577596fb710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c6face89dc3f7a5 server_cookie=1df14d4a224dd767 in_seq=0 out_seq=0 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe4effd640 1 -- 172.21.15.66:0/3575700020 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577595ebc10 con 0x5577596f92f0 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe4effd640 10 client.17121 handle_command_reply: tid=0 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 client.17121 shutdown 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577596794d0 msgr2=0x557759698160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577596794d0 0x557759698160 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000bcb0 tx=0x7fbe6001bf90 comp rx=0 tx=0).stop 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 shutdown_connections 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5577596f92f0 0x5577596fb710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557759682a00 0x557759695a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557759679e80 0x5577596954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 --2- 172.21.15.66:0/3575700020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577596794d0 0x557759698160 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe4effd640 0 client.17121 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 >> 172.21.15.66:0/3575700020 conn(0x557759673ba0 msgr2=0x557759672c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 shutdown_connections 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 1 -- 172.21.15.66:0/3575700020 wait complete. 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.483+0000 7fbe7881b740 20 client.17121 trim_cache size 0 max 16384 2024-04-21T09:13:39.489 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:13:39.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbe580a3da0 msgr2=0x7fbe580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbe580a3da0 0x7fbe580a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe68004af0 tx=0x7fbe6802e040 comp rx=0 tx=0).stop 2024-04-21T09:13:39.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 msgr2=0x7fbe70198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe70198c10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400e9f0 tx=0x7fbe6400eeb0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 shutdown_connections 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbe580a3da0 0x7fbe580a6260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe7019cf90 0x7fbe7019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe70107270 0x7fbe70198c10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 --2- 172.21.15.66:0/284078099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe7006a840 0x7fbe701986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 >> 172.21.15.66:0/284078099 conn(0x7fbe70077eb0 msgr2=0x7fbe70100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 shutdown_connections 2024-04-21T09:13:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.484+0000 7fbe75c32640 1 -- 172.21.15.66:0/284078099 wait complete. 2024-04-21T09:13:39.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:13:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.598+0000 7f51c86f1640 1 -- 172.21.15.66:0/391610365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 msgr2=0x7f51c006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.598+0000 7f51c86f1640 1 --2- 172.21.15.66:0/391610365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c006a9b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009900 tx=0x7f51bc02bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.598+0000 7f51c86f1640 1 -- 172.21.15.66:0/391610365 shutdown_connections 2024-04-21T09:13:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.598+0000 7f51c86f1640 1 --2- 172.21.15.66:0/391610365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c010e960 0x7f51c0110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.598+0000 7f51c86f1640 1 --2- 172.21.15.66:0/391610365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c006aef0 0x7f51c010e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.598+0000 7f51c86f1640 1 --2- 172.21.15.66:0/391610365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c006a9b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.598+0000 7f51c86f1640 1 -- 172.21.15.66:0/391610365 >> 172.21.15.66:0/391610365 conn(0x7f51c0077eb0 msgr2=0x7f51c00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 -- 172.21.15.66:0/391610365 shutdown_connections 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 -- 172.21.15.66:0/391610365 wait complete. 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 Processor -- start 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 -- start start 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c006aef0 0x7f51c01a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c010e960 0x7f51c019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01143e0 con 0x7f51c006a5d0 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0114260 con 0x7f51c010e960 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c86f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0114560 con 0x7f51c006aef0 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c01a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45560/0 (socket says 172.21.15.66:45560) 2024-04-21T09:13:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.599+0000 7f51c648d640 1 -- 172.21.15.66:0/997906320 learned_addr learned my addr 172.21.15.66:0/997906320 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c006aef0 0x7f51c01a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c010e960 0x7f51c019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c648d640 1 -- 172.21.15.66:0/997906320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c006aef0 msgr2=0x7f51c01a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c648d640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c006aef0 0x7f51c01a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c648d640 1 -- 172.21.15.66:0/997906320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c010e960 msgr2=0x7f51c019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c648d640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c010e960 0x7f51c019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c648d640 1 -- 172.21.15.66:0/997906320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c019afe0 con 0x7f51c006a5d0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c010e960 0x7f51c019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c648d640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c01a0660 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005940 tx=0x7f51bc02c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51b77fe640 1 -- 172.21.15.66:0/997906320 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc043020 con 0x7f51c006a5d0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c019b270 con 0x7f51c006a5d0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51b77fe640 1 -- 172.21.15.66:0/997906320 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51bc02cc60 con 0x7f51c006a5d0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51b77fe640 1 -- 172.21.15.66:0/997906320 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc003c30 con 0x7f51c006a5d0 2024-04-21T09:13:39.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.600+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01a75e0 con 0x7f51c006a5d0 2024-04-21T09:13:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.602+0000 7f51b77fe640 1 -- 172.21.15.66:0/997906320 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f51bc046020 con 0x7f51c006a5d0 2024-04-21T09:13:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 Processor -- start 2024-04-21T09:13:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 -- start start 2024-04-21T09:13:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa3c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa45430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5588bf9a11f0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588bf93e4b0 con 0x5588bfa3ce80 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5588bf99d1b0 con 0x5588bfa45a00 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51c6c8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa3c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51b77fe640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51c6c8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa3c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45562/0 (socket says 172.21.15.66:45562) 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51c6c8e640 1 -- 172.21.15.66:0/234914096 learned_addr learned my addr 172.21.15.66:0/234914096 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51c648d640 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa45430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.603+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51b77fe640 1 -- 172.21.15.66:0/997906320 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f51bc0f2f20 con 0x7f51c006a5d0 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51c6c8e640 1 -- 172.21.15.66:0/234914096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 msgr2=0x5588bfa47e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa47e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51c6c8e640 1 -- 172.21.15.66:0/234914096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 msgr2=0x5588bfa45430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa45430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51c6c8e640 1 -- 172.21.15.66:0/234914096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588bf9301b0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa3c8b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f51b8014980 tx=0x7f51b8014e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3095145a548535b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51b4ff9640 1 -- 172.21.15.66:0/234914096 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8021070 con 0x5588bfa3c4d0 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51b4ff9640 1 -- 172.21.15.66:0/234914096 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51b8011e30 con 0x5588bfa3c4d0 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.604+0000 7f51b4ff9640 1 -- 172.21.15.66:0/234914096 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8013bf0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 -- 172.21.15.66:0/234914096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 msgr2=0x5588bfa3c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51b801e000 tx=0x7f51b801a000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa3c8b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f51b8014980 tx=0x7f51b8014e40 comp rx=0 tx=0).stop 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 -- 172.21.15.66:0/234914096 shutdown_connections 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa47e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa45430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/234914096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa3c8b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 -- 172.21.15.66:0/234914096 >> 172.21.15.66:0/234914096 conn(0x5588bfa36ba0 msgr2=0x5588bfa36f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 -- 172.21.15.66:0/234914096 shutdown_connections 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 -- 172.21.15.66:0/234914096 wait complete. 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 -- start start 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.605+0000 7f51cb3b7740 1 Processor -- start 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa42650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa42bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5588bfa48970 con 0x5588bfa3c4d0 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588bf95a500 con 0x5588bfa45a00 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5588bfa48af0 con 0x5588bfa3ce80 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 10 client.0 _ll_register_callbacks cb 0x5588bf928b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45570/0 (socket says 172.21.15.66:45570) 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 -- 172.21.15.66:0/3474114127 learned_addr learned my addr 172.21.15.66:0/3474114127 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa42650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa42bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 -- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 msgr2=0x5588bfa42650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa42650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 -- 172.21.15.66:0/3474114127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 msgr2=0x5588bfa42bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa42bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 -- 172.21.15.66:0/3474114127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588bf6b33f0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51c648d640 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa452c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0f0430 tx=0x7f51bc08e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26900702d962428 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3474114127 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0bf320 con 0x5588bfa3c4d0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3474114127 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f51bc0f0540 con 0x5588bfa3c4d0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b8011e30 con 0x5588bfa3c4d0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3474114127 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51bc0f0740 con 0x5588bfa3c4d0 2024-04-21T09:13:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 10 client.17133 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.606+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588bf77d7d0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3474114127 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51b8011e30 con 0x5588bfa3c4d0 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3474114127 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f51bc08d1e0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51cb3b7740 10 client.17133 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51cb3b7740 10 client.17133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51cb3b7740 20 client.17133 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfa55a60 0x5588bfa57ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51cb3b7740 4 client.17133 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588bf940930 con 0x5588bfa55a60 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.607+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfa55a60 0x5588bfa57ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfa55a60 0x5588bfa57ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d420e7b22816fce server_cookie=4fb5fc382aace88c in_seq=0 out_seq=0 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51a9ffb640 10 client.17133 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3474114127 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588bf940930 con 0x5588bfa55a60 2024-04-21T09:13:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51a9ffb640 10 client.17133 handle_command_reply: tid=0 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 client.17133 shutdown 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 msgr2=0x5588bfa452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa452c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0f0430 tx=0x7f51bc08e500 comp rx=0 tx=0).stop 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 shutdown_connections 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa42bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa42650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfa55a60 0x5588bfa57ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3474114127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa452c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.608+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 >> 172.21.15.66:0/3474114127 conn(0x5588bfa36ba0 msgr2=0x5588bfa388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.609+0000 7f51a9ffb640 0 client.17133 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.609+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 shutdown_connections 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.609+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3474114127 wait complete. 2024-04-21T09:13:39.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.609+0000 7f51cb3b7740 20 client.17133 trim_cache size 0 max 16384 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 Processor -- start 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 -- start start 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa51830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa44d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b3da0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b1670 con 0x5588bfa3ce80 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01af4a0 con 0x5588bfa45a00 2024-04-21T09:13:39.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa51830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa51830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45578/0 (socket says 172.21.15.66:45578) 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.622+0000 7f51c648d640 1 -- 172.21.15.66:0/3790976639 learned_addr learned my addr 172.21.15.66:0/3790976639 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa44d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c648d640 1 -- 172.21.15.66:0/3790976639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 msgr2=0x5588bfa44d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c648d640 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa44d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c648d640 1 -- 172.21.15.66:0/3790976639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 msgr2=0x5588bfa386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c648d640 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c648d640 1 -- 172.21.15.66:0/3790976639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588bfa42f20 con 0x5588bfa3c4d0 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa44d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51c648d640 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa51830 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0bb080 tx=0x7f51bc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b2cd3d06b63186 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51abfff640 1 -- 172.21.15.66:0/3790976639 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0bec20 con 0x5588bfa3c4d0 2024-04-21T09:13:39.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51abfff640 1 -- 172.21.15.66:0/3790976639 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51bc0c2fd0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51abfff640 1 -- 172.21.15.66:0/3790976639 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0f0710 con 0x5588bfa3c4d0 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3790976639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 msgr2=0x5588bfa51830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.623+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa51830 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0bb080 tx=0x7f51bc005c00 comp rx=0 tx=0).stop 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3790976639 shutdown_connections 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa45a00 0x5588bfa44d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa3ce80 0x5588bfa386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3790976639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa51830 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3790976639 >> 172.21.15.66:0/3790976639 conn(0x5588bfa36ba0 msgr2=0x5588bfa37fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3790976639 shutdown_connections 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3790976639 wait complete. 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- start start 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 Processor -- start 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa4df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa3eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa3f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01afb30 con 0x5588bfa3c4d0 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b0fa0 con 0x5588bfa45a00 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b0200 con 0x5588bfa3ce80 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51cb3b7740 10 client.0 _ll_register_callbacks cb 0x7f51bc0be760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa4df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51c648d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa4df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45590/0 (socket says 172.21.15.66:45590) 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51c648d640 1 -- 172.21.15.66:0/3175319157 learned_addr learned my addr 172.21.15.66:0/3175319157 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa3f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.624+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa3eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51c648d640 1 -- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 msgr2=0x5588bfa3eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51c648d640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa3eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51c648d640 1 -- 172.21.15.66:0/3175319157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 msgr2=0x5588bfa3f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51c648d640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa3f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51c648d640 1 -- 172.21.15.66:0/3175319157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588bfa3f920 con 0x5588bfa3c4d0 2024-04-21T09:13:39.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51c6c8e640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa3f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:39.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51c648d640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa4df30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000c00 tx=0x7f51bc0df390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d841d04630a7333 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.625+0000 7f51a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:39.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3175319157 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0d7030 con 0x5588bfa3c4d0 2024-04-21T09:13:39.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51bc0c2fd0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3175319157 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f51bc0f04b0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3175319157 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51bc0bec20 con 0x5588bfa3c4d0 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 10 client.17139 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588bfa3f920 con 0x5588bfa3c4d0 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3175319157 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51bc0c2fd0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3175319157 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f51bc0bdae0 con 0x5588bfa3c4d0 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 10 client.17139 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 10 client.17139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 20 client.17139 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfabbf50 0x5588bfabe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 4 client.17139 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.628+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588bf9aec10 con 0x5588bfabbf50 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfabbf50 0x5588bfabe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51c5c8c640 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfabbf50 0x5588bfabe340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=130651e33bbb6b6e server_cookie=206aa42770d40cd2 in_seq=0 out_seq=0 2024-04-21T09:13:39.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51a9ffb640 10 client.17139 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51a9ffb640 1 -- 172.21.15.66:0/3175319157 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588bf9aec10 con 0x5588bfabbf50 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51a9ffb640 10 client.17139 handle_command_reply: tid=0 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 client.17139 shutdown 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 msgr2=0x5588bfa4df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa4df30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000c00 tx=0x7f51bc0df390 comp rx=0 tx=0).stop 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 shutdown_connections 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588bfa45a00 0x5588bfa3f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5588bfa3ce80 0x5588bfa3eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5588bfabbf50 0x5588bfabe340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 --2- 172.21.15.66:0/3175319157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5588bfa3c4d0 0x5588bfa4df30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51a9ffb640 0 client.17139 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 >> 172.21.15.66:0/3175319157 conn(0x5588bfa36ba0 msgr2=0x5588bfa35c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.629+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 shutdown_connections 2024-04-21T09:13:39.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51cb3b7740 1 -- 172.21.15.66:0/3175319157 wait complete. 2024-04-21T09:13:39.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51cb3b7740 20 client.17139 trim_cache size 0 max 16384 2024-04-21T09:13:39.634 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51ac0a3d20 msgr2=0x7f51ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51b801e000 tx=0x7f51b801a000 comp rx=0 tx=0).stop 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 msgr2=0x7f51c01a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c01a0660 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005940 tx=0x7f51bc02c860 comp rx=0 tx=0).stop 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 shutdown_connections 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c010e960 0x7f51c019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c006aef0 0x7f51c01a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 --2- 172.21.15.66:0/997906320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c006a5d0 0x7f51c01a0660 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 >> 172.21.15.66:0/997906320 conn(0x7f51c0077eb0 msgr2=0x7f51c010fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 shutdown_connections 2024-04-21T09:13:39.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.630+0000 7f51c86f1640 1 -- 172.21.15.66:0/997906320 wait complete. 2024-04-21T09:13:39.640 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:13:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.744+0000 7f6c213f4640 1 -- 172.21.15.66:0/1234553247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c107270 msgr2=0x7f6c1c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.744+0000 7f6c213f4640 1 --2- 172.21.15.66:0/1234553247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c107270 0x7f6c1c10ee40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009ca0 tx=0x7f6c1402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.744+0000 7f6c213f4640 1 -- 172.21.15.66:0/1234553247 shutdown_connections 2024-04-21T09:13:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.744+0000 7f6c213f4640 1 --2- 172.21.15.66:0/1234553247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c107270 0x7f6c1c10ee40 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.744+0000 7f6c213f4640 1 --2- 172.21.15.66:0/1234553247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c1c06b1f0 0x7f6c1c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.744+0000 7f6c213f4640 1 --2- 172.21.15.66:0/1234553247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c1c06a840 0x7f6c1c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.744+0000 7f6c213f4640 1 -- 172.21.15.66:0/1234553247 >> 172.21.15.66:0/1234553247 conn(0x7f6c1c077eb0 msgr2=0x7f6c1c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 -- 172.21.15.66:0/1234553247 shutdown_connections 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 -- 172.21.15.66:0/1234553247 wait complete. 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 Processor -- start 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 -- start start 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c1c06a840 0x7f6c1c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c1c107270 0x7f6c1c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c19cf40 0x7f6c1c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1121b0 con 0x7f6c1c19cf40 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c112030 con 0x7f6c1c107270 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c213f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c112330 con 0x7f6c1c06a840 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c1b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c19cf40 0x7f6c1c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c1b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c19cf40 0x7f6c1c19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45612/0 (socket says 172.21.15.66:45612) 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.745+0000 7f6c1b7fe640 1 -- 172.21.15.66:0/3505293765 learned_addr learned my addr 172.21.15.66:0/3505293765 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c13fff640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c1c107270 0x7f6c1c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c1b7fe640 1 -- 172.21.15.66:0/3505293765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c1c06a840 msgr2=0x7f6c1c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c1b7fe640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c1c06a840 0x7f6c1c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c1b7fe640 1 -- 172.21.15.66:0/3505293765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c1c107270 msgr2=0x7f6c1c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c1b7fe640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c1c107270 0x7f6c1c198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c1b7fe640 1 -- 172.21.15.66:0/3505293765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c19da30 con 0x7f6c1c19cf40 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c13fff640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c1c107270 0x7f6c1c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c1b7fe640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c19cf40 0x7f6c1c19d3f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005e30 tx=0x7f6c14032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c18ff9640 1 -- 172.21.15.66:0/3505293765 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14041020 con 0x7f6c1c19cf40 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c19dcc0 con 0x7f6c1c19cf40 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c18ff9640 1 -- 172.21.15.66:0/3505293765 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c14032970 con 0x7f6c1c19cf40 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c18ff9640 1 -- 172.21.15.66:0/3505293765 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14039040 con 0x7f6c1c19cf40 2024-04-21T09:13:39.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.746+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c1a5480 con 0x7f6c1c19cf40 2024-04-21T09:13:39.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c18ff9640 1 -- 172.21.15.66:0/3505293765 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6c14044020 con 0x7f6c1c19cf40 2024-04-21T09:13:39.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 Processor -- start 2024-04-21T09:13:39.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 -- start start 2024-04-21T09:13:39.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da383790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da38d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5629da2e81f0 con 0x5629da3833b0 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5629da2854b0 con 0x5629da383d60 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.749+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5629da2e41b0 con 0x5629da38ce60 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da383790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da383790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45620/0 (socket says 172.21.15.66:45620) 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 -- 172.21.15.66:0/546282618 learned_addr learned my addr 172.21.15.66:0/546282618 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c1affd640 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da38d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c18ff9640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6c080a3d20 0x7f6c080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 -- 172.21.15.66:0/546282618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 msgr2=0x5629da38d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da38d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 -- 172.21.15.66:0/546282618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 msgr2=0x5629da38c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 -- 172.21.15.66:0/546282618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629da2771b0 con 0x5629da3833b0 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c1affd640 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c1b7fe640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6c080a3d20 0x7f6c080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c13fff640 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da383790 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6c04012390 tx=0x7f6c04009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9531d825a3fb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c18ff9640 1 -- 172.21.15.66:0/3505293765 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6c140f2960 con 0x7f6c1c19cf40 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c117fa640 1 -- 172.21.15.66:0/546282618 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04004030 con 0x5629da3833b0 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c117fa640 1 -- 172.21.15.66:0/546282618 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0401b070 con 0x5629da3833b0 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.750+0000 7f6c117fa640 1 -- 172.21.15.66:0/546282618 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04003e20 con 0x5629da3833b0 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 -- 172.21.15.66:0/546282618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 msgr2=0x5629da383790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c1b7fe640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6c080a3d20 0x7f6c080a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6c14083fc0 tx=0x7f6c1408df40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da383790 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6c04012390 tx=0x7f6c04009740 comp rx=0 tx=0).stop 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 -- 172.21.15.66:0/546282618 shutdown_connections 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da38d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/546282618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da383790 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 -- 172.21.15.66:0/546282618 >> 172.21.15.66:0/546282618 conn(0x5629da37dba0 msgr2=0x5629da37df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 -- 172.21.15.66:0/546282618 shutdown_connections 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.751+0000 7f6c23fe8740 1 -- 172.21.15.66:0/546282618 wait complete. 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 -- start start 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 Processor -- start 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da3833b0 0x5629da393210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da3905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da38ce60 0x5629da390ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5629da393da0 con 0x5629da38ce60 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5629da2a1500 con 0x5629da383d60 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5629da393f20 con 0x5629da3833b0 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c23fe8740 10 client.0 _ll_register_callbacks cb 0x5629da26fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:39.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da38ce60 0x5629da390ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da38ce60 0x5629da390ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45630/0 (socket says 172.21.15.66:45630) 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 -- 172.21.15.66:0/631787561 learned_addr learned my addr 172.21.15.66:0/631787561 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c1affd640 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da3905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da3833b0 0x5629da393210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 -- 172.21.15.66:0/631787561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da3833b0 msgr2=0x5629da393210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da3833b0 0x5629da393210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 -- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 msgr2=0x5629da3905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da3905a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 -- 172.21.15.66:0/631787561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d9ffa3f0 con 0x5629da38ce60 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c13fff640 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da38ce60 0x5629da390ae0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005c20 tx=0x7f6c04003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2aea766663bc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.752+0000 7f6c027fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c027fc640 1 -- 172.21.15.66:0/631787561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040035f0 con 0x5629da38ce60 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c027fc640 1 -- 172.21.15.66:0/631787561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6c04005020 con 0x5629da38ce60 2024-04-21T09:13:39.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0401b070 con 0x5629da38ce60 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c027fc640 1 -- 172.21.15.66:0/631787561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c04031050 con 0x5629da38ce60 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 10 client.17151 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629da0c47d0 con 0x5629da38ce60 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c027fc640 1 -- 172.21.15.66:0/631787561 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0401b070 con 0x5629da38ce60 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c027fc640 1 -- 172.21.15.66:0/631787561 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6c0400f040 con 0x5629da38ce60 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 10 client.17151 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 20 client.17151 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da3a0f80 0x5629da3a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 4 client.17151 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629da287930 con 0x5629da3a0f80 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.753+0000 7f6c1affd640 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da3a0f80 0x5629da3a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.754+0000 7f6c1affd640 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da3a0f80 0x5629da3a3370 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821a1f52f996d7fd server_cookie=87a1ca65ff590a27 in_seq=0 out_seq=0 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.754+0000 7f6c027fc640 10 client.17151 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:39.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.754+0000 7f6c027fc640 1 -- 172.21.15.66:0/631787561 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629da287930 con 0x5629da3a0f80 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.754+0000 7f6c027fc640 10 client.17151 handle_command_reply: tid=0 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 client.17151 shutdown 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da38ce60 msgr2=0x5629da390ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da38ce60 0x5629da390ae0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005c20 tx=0x7f6c04003960 comp rx=0 tx=0).stop 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 shutdown_connections 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da3a0f80 0x5629da3a3370 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da38ce60 0x5629da390ae0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da3905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/631787561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da3833b0 0x5629da393210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c027fc640 0 client.17151 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 >> 172.21.15.66:0/631787561 conn(0x5629da37dba0 msgr2=0x5629da37f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 shutdown_connections 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 1 -- 172.21.15.66:0/631787561 wait complete. 2024-04-21T09:13:39.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.755+0000 7f6c23fe8740 20 client.17151 trim_cache size 0 max 16384 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 Processor -- start 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 -- start start 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da39ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da392cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1af910 con 0x5629da3833b0 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1ad1e0 con 0x5629da383d60 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.768+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1ab010 con 0x5629da38ce60 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da39ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da39ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45642/0 (socket says 172.21.15.66:45642) 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 -- 172.21.15.66:0/20926587 learned_addr learned my addr 172.21.15.66:0/20926587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c1affd640 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 -- 172.21.15.66:0/20926587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 msgr2=0x5629da392cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da392cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 -- 172.21.15.66:0/20926587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 msgr2=0x5629da38f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 -- 172.21.15.66:0/20926587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629da383b00 con 0x5629da3833b0 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c1affd640 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c13fff640 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da39ccd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6c04004be0 tx=0x7f6c04005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba422730b1303369 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c10ff9640 1 -- 172.21.15.66:0/20926587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04012ce0 con 0x5629da3833b0 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c10ff9640 1 -- 172.21.15.66:0/20926587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0401b040 con 0x5629da3833b0 2024-04-21T09:13:39.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.769+0000 7f6c10ff9640 1 -- 172.21.15.66:0/20926587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04009790 con 0x5629da3833b0 2024-04-21T09:13:39.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 -- 172.21.15.66:0/20926587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 msgr2=0x5629da39ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da39ccd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6c04004be0 tx=0x7f6c04005d30 comp rx=0 tx=0).stop 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 -- 172.21.15.66:0/20926587 shutdown_connections 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da392cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da38f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/20926587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da39ccd0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 -- 172.21.15.66:0/20926587 >> 172.21.15.66:0/20926587 conn(0x5629da37dba0 msgr2=0x5629da38ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 -- 172.21.15.66:0/20926587 shutdown_connections 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 -- 172.21.15.66:0/20926587 wait complete. 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 -- start start 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.770+0000 7f6c23fe8740 1 Processor -- start 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da3995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da391c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da392190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1ab6a0 con 0x5629da3833b0 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1acb10 con 0x5629da383d60 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1abd70 con 0x5629da38ce60 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 10 client.0 _ll_register_callbacks cb 0x7f6c04003ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da3995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da3995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45658/0 (socket says 172.21.15.66:45658) 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 -- 172.21.15.66:0/2627102957 learned_addr learned my addr 172.21.15.66:0/2627102957 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c1affd640 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da391c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 -- 172.21.15.66:0/2627102957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 msgr2=0x5629da392190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da392190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 -- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 msgr2=0x5629da391c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da391c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 -- 172.21.15.66:0/2627102957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629da392970 con 0x5629da3833b0 2024-04-21T09:13:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c1affd640 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da391c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c13fff640 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da3995f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6c040059d0 tx=0x7f6c04012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7918356de4588041 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c027fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c027fc640 1 -- 172.21.15.66:0/2627102957 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04009530 con 0x5629da3833b0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0401b040 con 0x5629da3833b0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c027fc640 1 -- 172.21.15.66:0/2627102957 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f6c040096d0 con 0x5629da3833b0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c027fc640 1 -- 172.21.15.66:0/2627102957 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c040045c0 con 0x5629da3833b0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 10 client.17157 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.771+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629da406870 con 0x5629da3833b0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c027fc640 1 -- 172.21.15.66:0/2627102957 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0401b040 con 0x5629da3833b0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c027fc640 1 -- 172.21.15.66:0/2627102957 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6c04004c60 con 0x5629da3833b0 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c23fe8740 10 client.17157 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c23fe8740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c23fe8740 20 client.17157 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da406e80 0x5629da409270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c23fe8740 4 client.17157 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629da2f5c10 con 0x5629da406e80 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c1affd640 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da406e80 0x5629da409270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c027fc640 10 client.17157 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.772+0000 7f6c1affd640 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da406e80 0x5629da409270 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5676d9e3d13ff8 server_cookie=544e55fb74a5ea46 in_seq=0 out_seq=0 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c027fc640 1 -- 172.21.15.66:0/2627102957 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629da2f5c10 con 0x5629da406e80 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c027fc640 10 client.17157 handle_command_reply: tid=0 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 client.17157 shutdown 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 msgr2=0x5629da3995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da3995f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6c040059d0 tx=0x7f6c04012e20 comp rx=0 tx=0).stop 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 shutdown_connections 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5629da406e80 0x5629da409270 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629da38ce60 0x5629da392190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629da383d60 0x5629da391c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c027fc640 0 client.17157 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 --2- 172.21.15.66:0/2627102957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629da3833b0 0x5629da3995f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 >> 172.21.15.66:0/2627102957 conn(0x5629da37dba0 msgr2=0x5629da37eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 shutdown_connections 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 1 -- 172.21.15.66:0/2627102957 wait complete. 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c23fe8740 20 client.17157 trim_cache size 0 max 16384 2024-04-21T09:13:39.779 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:13:39.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6c080a3d20 msgr2=0x7f6c080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c213f4640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6c080a3d20 0x7f6c080a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6c14083fc0 tx=0x7f6c1408df40 comp rx=0 tx=0).stop 2024-04-21T09:13:39.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c19cf40 msgr2=0x7f6c1c19d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:39.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.773+0000 7f6c213f4640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c19cf40 0x7f6c1c19d3f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005e30 tx=0x7f6c14032420 comp rx=0 tx=0).stop 2024-04-21T09:13:39.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 shutdown_connections 2024-04-21T09:13:39.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f6c080a3d20 0x7f6c080a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c1c19cf40 0x7f6c1c19d3f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c1c107270 0x7f6c1c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 --2- 172.21.15.66:0/3505293765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c1c06a840 0x7f6c1c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 >> 172.21.15.66:0/3505293765 conn(0x7f6c1c077eb0 msgr2=0x7f6c1c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 shutdown_connections 2024-04-21T09:13:39.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:39.774+0000 7f6c213f4640 1 -- 172.21.15.66:0/3505293765 wait complete. 2024-04-21T09:13:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:39 smithi186 ceph-mon[31505]: pgmap v166: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:40.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:39 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1445804932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:13:40.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:39 smithi083 ceph-mon[31413]: pgmap v166: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:40.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:39 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1445804932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:13:40.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:39 smithi066 ceph-mon[27953]: pgmap v166: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:40.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:39 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1445804932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:13:40.785 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:13:40.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:13:40.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:13:40.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 -- 172.21.15.66:0/2432245589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc100af0 msgr2=0x7f79cc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 --2- 172.21.15.66:0/2432245589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc100af0 0x7f79cc100ed0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009900 tx=0x7f79b402bc30 comp rx=0 tx=0).stop 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 -- 172.21.15.66:0/2432245589 shutdown_connections 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 --2- 172.21.15.66:0/2432245589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc1072b0 0x7f79cc10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 --2- 172.21.15.66:0/2432245589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79cc1014a0 0x7f79cc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 --2- 172.21.15.66:0/2432245589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc100af0 0x7f79cc100ed0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 -- 172.21.15.66:0/2432245589 >> 172.21.15.66:0/2432245589 conn(0x7f79cc0760e0 msgr2=0x7f79cc0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 -- 172.21.15.66:0/2432245589 shutdown_connections 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.905+0000 7f79d13db640 1 -- 172.21.15.66:0/2432245589 wait complete. 2024-04-21T09:13:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 Processor -- start 2024-04-21T09:13:41.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 -- start start 2024-04-21T09:13:41.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:40 smithi083 ceph-mon[31413]: pgmap v167: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:41.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:40 smithi186 ceph-mon[31505]: pgmap v167: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc100af0 0x7f79cc19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79cc1072b0 0x7f79cc19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc198660 0x7f79cc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc112230 con 0x7f79cc198660 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1120b0 con 0x7f79cc100af0 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79d13db640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1123b0 con 0x7f79cc1072b0 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc198660 0x7f79cc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc198660 0x7f79cc198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45686/0 (socket says 172.21.15.66:45686) 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79cb7fe640 1 -- 172.21.15.66:0/923619201 learned_addr learned my addr 172.21.15.66:0/923619201 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.906+0000 7f79caffd640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc100af0 0x7f79cc19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79cb7fe640 1 -- 172.21.15.66:0/923619201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79cc1072b0 msgr2=0x7f79cc19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79cc1072b0 0x7f79cc19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79cb7fe640 1 -- 172.21.15.66:0/923619201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc100af0 msgr2=0x7f79cc19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc100af0 0x7f79cc19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79cb7fe640 1 -- 172.21.15.66:0/923619201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc199180 con 0x7f79cc198660 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc198660 0x7f79cc198b10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f79c400eb10 tx=0x7f79c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79c3fff640 1 -- 172.21.15.66:0/923619201 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4004b20 con 0x7f79cc198660 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79c3fff640 1 -- 172.21.15.66:0/923619201 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c4004cc0 con 0x7f79cc198660 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc199470 con 0x7f79cc198660 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79c3fff640 1 -- 172.21.15.66:0/923619201 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4010670 con 0x7f79cc198660 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.907+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc1a5620 con 0x7f79cc198660 2024-04-21T09:13:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.909+0000 7f79c3fff640 1 -- 172.21.15.66:0/923619201 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f79c4005020 con 0x7f79cc198660 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 Processor -- start 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 -- start start 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79c3fff640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f79b80a3d20 0x7f79b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1b7c0 0x556250d1bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d24ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d25270 0x556250d256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556250c7ff00 con 0x556250d25270 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556250c1d590 con 0x556250d1c170 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556250c7bec0 con 0x556250d1b7c0 2024-04-21T09:13:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d25270 0x556250d256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f79b80a3d20 0x7f79b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d25270 0x556250d256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45688/0 (socket says 172.21.15.66:45688) 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79cb7fe640 1 -- 172.21.15.66:0/36073951 learned_addr learned my addr 172.21.15.66:0/36073951 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79c3fff640 1 -- 172.21.15.66:0/923619201 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f79c40d1120 con 0x7f79cc198660 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.910+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d24ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79cb7fe640 1 -- 172.21.15.66:0/36073951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1b7c0 msgr2=0x556250d1bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1b7c0 0x556250d1bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79cb7fe640 1 -- 172.21.15.66:0/36073951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 msgr2=0x556250d24ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d24ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79cb7fe640 1 -- 172.21.15.66:0/36073951 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556250a5a7e0 con 0x556250d25270 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f79b80a3d20 0x7f79b80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79cc100540 tx=0x7f79bc0147e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d25270 0x556250d256d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005850 tx=0x7f79c400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe9fc4595bd3e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79c17fa640 1 -- 172.21.15.66:0/36073951 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c409c6b0 con 0x556250d25270 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79c17fa640 1 -- 172.21.15.66:0/36073951 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c409d450 con 0x556250d25270 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.911+0000 7f79c17fa640 1 -- 172.21.15.66:0/36073951 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40897b0 con 0x556250d25270 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 -- 172.21.15.66:0/36073951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d25270 msgr2=0x556250d256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d25270 0x556250d256d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005850 tx=0x7f79c400cf70 comp rx=0 tx=0).stop 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 -- 172.21.15.66:0/36073951 shutdown_connections 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d25270 0x556250d256d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d24ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/36073951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1b7c0 0x556250d1bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 -- 172.21.15.66:0/36073951 >> 172.21.15.66:0/36073951 conn(0x556250d15fb0 msgr2=0x556250d16390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 -- 172.21.15.66:0/36073951 shutdown_connections 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 -- 172.21.15.66:0/36073951 wait complete. 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 -- start start 2024-04-21T09:13:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.912+0000 7f79d3fc5740 1 Processor -- start 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d25270 0x556250d29760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556250d2c1b0 con 0x556250d1b7c0 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556250c39580 con 0x556250d1c170 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556250d2c330 con 0x556250d25270 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 10 client.0 _ll_register_callbacks cb 0x556250c3f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d28ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d28ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45690/0 (socket says 172.21.15.66:45690) 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 -- 172.21.15.66:0/1267393810 learned_addr learned my addr 172.21.15.66:0/1267393810 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d25270 0x556250d29760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 -- 172.21.15.66:0/1267393810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d25270 msgr2=0x556250d29760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d25270 0x556250d29760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 -- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 msgr2=0x556250d29220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d29220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 -- 172.21.15.66:0/1267393810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556250abaf70 con 0x556250d1b7c0 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d25270 0x556250d29760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79caffd640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d28ce0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009900 tx=0x7f79b402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee8788264ddfe94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79b27fc640 1 -- 172.21.15.66:0/1267393810 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4045020 con 0x556250d1b7c0 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79b27fc640 1 -- 172.21.15.66:0/1267393810 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f79b4005040 con 0x556250d1b7c0 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.913+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c409d450 con 0x556250d1b7c0 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79b27fc640 1 -- 172.21.15.66:0/1267393810 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79b4050050 con 0x556250d1b7c0 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 10 client.17169 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:41.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556250952170 con 0x556250d1b7c0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79b27fc640 1 -- 172.21.15.66:0/1267393810 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c409d450 con 0x556250d1b7c0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79b27fc640 1 -- 172.21.15.66:0/1267393810 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79b4004bf0 con 0x556250d1b7c0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 10 client.17169 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 20 client.17169 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250d396f0 0x556250d3bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 4 client.17169 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556250c8d920 con 0x556250d396f0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.914+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250d396f0 0x556250d3bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.915+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250d396f0 0x556250d3bbb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3334470e8f2ace server_cookie=a01770324532b5db in_seq=0 out_seq=0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.915+0000 7f79b27fc640 10 client.17169 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.915+0000 7f79b27fc640 1 -- 172.21.15.66:0/1267393810 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556250c8d920 con 0x556250d396f0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.915+0000 7f79b27fc640 10 client.17169 handle_command_reply: tid=0 2024-04-21T09:13:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.915+0000 7f79d3fc5740 1 client.17169 shutdown 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 msgr2=0x556250d28ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d28ce0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009900 tx=0x7f79b402ca10 comp rx=0 tx=0).stop 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 shutdown_connections 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250d396f0 0x556250d3bbb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d25270 0x556250d29760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d1c170 0x556250d29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79b27fc640 0 client.17169 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1267393810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d28ce0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 >> 172.21.15.66:0/1267393810 conn(0x556250d15fb0 msgr2=0x556250d17d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 shutdown_connections 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1267393810 wait complete. 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.916+0000 7f79d3fc5740 20 client.17169 trim_cache size 0 max 16384 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 Processor -- start 2024-04-21T09:13:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 -- start start 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 0x556250d27e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1afa10 con 0x556250d1b7c0 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ad2e0 con 0x556250d25270 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ab110 con 0x556250d1c170 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45706/0 (socket says 172.21.15.66:45706) 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 -- 172.21.15.66:0/4002487483 learned_addr learned my addr 172.21.15.66:0/4002487483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 -- 172.21.15.66:0/4002487483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 msgr2=0x556250d27e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 0x556250d27e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 -- 172.21.15.66:0/4002487483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 msgr2=0x556250d26680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d26680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79caffd640 1 -- 172.21.15.66:0/4002487483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556250d1c5e0 con 0x556250d1b7c0 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.934+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d26680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79caffd640 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d350b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f79b402bc70 tx=0x7f79b4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239e065428eb781d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79c0ff9640 1 -- 172.21.15.66:0/4002487483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4045070 con 0x556250d1b7c0 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79c0ff9640 1 -- 172.21.15.66:0/4002487483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4005440 con 0x556250d1b7c0 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79c0ff9640 1 -- 172.21.15.66:0/4002487483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4000e60 con 0x556250d1b7c0 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 -- 172.21.15.66:0/4002487483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 msgr2=0x556250d350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d350b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f79b402bc70 tx=0x7f79b4004610 comp rx=0 tx=0).stop 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 -- 172.21.15.66:0/4002487483 shutdown_connections 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d26680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 0x556250d27e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/4002487483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d350b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 -- 172.21.15.66:0/4002487483 >> 172.21.15.66:0/4002487483 conn(0x556250d15fb0 msgr2=0x556250d173b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.935+0000 7f79d3fc5740 1 -- 172.21.15.66:0/4002487483 shutdown_connections 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 -- 172.21.15.66:0/4002487483 wait complete. 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 -- start start 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 Processor -- start 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 0x556250d31b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ab7a0 con 0x556250d1b7c0 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1acc10 con 0x556250d25270 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1abe70 con 0x556250d1c170 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79d3fc5740 10 client.0 _ll_register_callbacks cb 0x7f79b4004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45712/0 (socket says 172.21.15.66:45712) 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 -- 172.21.15.66:0/1851440221 learned_addr learned my addr 172.21.15.66:0/1851440221 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 -- 172.21.15.66:0/1851440221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 msgr2=0x556250d31b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 0x556250d31b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 -- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 msgr2=0x556250d2e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d2e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79caffd640 1 -- 172.21.15.66:0/1851440221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556250d2ed40 con 0x556250d1b7c0 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.936+0000 7f79cb7fe640 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79caffd640 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d315c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f79b4003da0 tx=0x7f79b402c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753a2d1327e380fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79b27fc640 1 -- 172.21.15.66:0/1851440221 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4045070 con 0x556250d1b7c0 2024-04-21T09:13:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79b27fc640 1 -- 172.21.15.66:0/1851440221 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f79b4004f90 con 0x556250d1b7c0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4005440 con 0x556250d1b7c0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79b27fc640 1 -- 172.21.15.66:0/1851440221 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79b4062050 con 0x556250d1b7c0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 10 client.17175 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556250da0760 con 0x556250d1b7c0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79b27fc640 1 -- 172.21.15.66:0/1851440221 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4005440 con 0x556250d1b7c0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79b27fc640 1 -- 172.21.15.66:0/1851440221 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79b4041510 con 0x556250d1b7c0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 10 client.17175 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 20 client.17175 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250da0db0 0x556250da31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 4 client.17175 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.937+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556250c87790 con 0x556250da0db0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.938+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250da0db0 0x556250da31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.938+0000 7f79b27fc640 10 client.17175 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.938+0000 7f79ca7fc640 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250da0db0 0x556250da31f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13289930d020b078 server_cookie=22d3b39fe61d31a0 in_seq=0 out_seq=0 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79b27fc640 1 -- 172.21.15.66:0/1851440221 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556250c87790 con 0x556250da0db0 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79b27fc640 10 client.17175 handle_command_reply: tid=0 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 client.17175 shutdown 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 msgr2=0x556250d315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d315c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f79b4003da0 tx=0x7f79b402c610 comp rx=0 tx=0).stop 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 shutdown_connections 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556250da0db0 0x556250da31f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79b27fc640 0 client.17175 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556250d25270 0x556250d2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556250d1c170 0x556250d31b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 --2- 172.21.15.66:0/1851440221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556250d1b7c0 0x556250d315c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 >> 172.21.15.66:0/1851440221 conn(0x556250d15fb0 msgr2=0x556250d15010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 shutdown_connections 2024-04-21T09:13:41.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 1 -- 172.21.15.66:0/1851440221 wait complete. 2024-04-21T09:13:41.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.939+0000 7f79d3fc5740 20 client.17175 trim_cache size 0 max 16384 2024-04-21T09:13:41.280 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:13:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:13:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "e934501b-a5a7-4490-8081-28ad6d2aa506", 2024-04-21T09:13:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:13:41.281 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:13:41.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f79b80a3d20 msgr2=0x7f79b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f79b80a3d20 0x7f79b80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79cc100540 tx=0x7f79bc0147e0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc198660 msgr2=0x7f79cc198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc198660 0x7f79cc198b10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f79c400eb10 tx=0x7f79c400c2d0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 shutdown_connections 2024-04-21T09:13:41.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f79b80a3d20 0x7f79b80a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79cc198660 0x7f79cc198b10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79cc1072b0 0x7f79cc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 --2- 172.21.15.66:0/923619201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc100af0 0x7f79cc19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 >> 172.21.15.66:0/923619201 conn(0x7f79cc0760e0 msgr2=0x7f79cc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 shutdown_connections 2024-04-21T09:13:41.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:40.940+0000 7f79d13db640 1 -- 172.21.15.66:0/923619201 wait complete. 2024-04-21T09:13:41.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:13:41.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:13:41.285 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:40 smithi066 ceph-mon[27953]: pgmap v167: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:41.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/556330610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 msgr2=0x7fbc741035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/556330610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc741035a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009900 tx=0x7fbc7002bd60 comp rx=0 tx=0).stop 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/556330610 shutdown_connections 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/556330610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc7410c500 0x7fbc7410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/556330610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc74103ae0 0x7fbc7410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/556330610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc741035a0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/556330610 >> 172.21.15.66:0/556330610 conn(0x7fbc74077fd0 msgr2=0x7fbc740783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/556330610 shutdown_connections 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.394+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/556330610 wait complete. 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 Processor -- start 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 -- start start 2024-04-21T09:13:41.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc74198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc74103ae0 0x7fbc74198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc7410c500 0x7fbc7419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc74112170 con 0x7fbc741031c0 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc74111ff0 con 0x7fbc74103ae0 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc7bd3b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741122f0 con 0x7fbc7410c500 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc79ad7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc74198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc79ad7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc74198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45728/0 (socket says 172.21.15.66:45728) 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc79ad7640 1 -- 172.21.15.66:0/1384610890 learned_addr learned my addr 172.21.15.66:0/1384610890 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc792d6640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc74103ae0 0x7fbc74198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc79ad7640 1 -- 172.21.15.66:0/1384610890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc7410c500 msgr2=0x7fbc7419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.395+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc7410c500 0x7fbc7419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc79ad7640 1 -- 172.21.15.66:0/1384610890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc74103ae0 msgr2=0x7fbc74198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc74103ae0 0x7fbc74198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc79ad7640 1 -- 172.21.15.66:0/1384610890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7419d6d0 con 0x7fbc741031c0 2024-04-21T09:13:41.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc792d6640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc74103ae0 0x7fbc74198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:13:41.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc74198730 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005940 tx=0x7fbc700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc6affd640 1 -- 172.21.15.66:0/1384610890 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70043020 con 0x7fbc741031c0 2024-04-21T09:13:41.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc6affd640 1 -- 172.21.15.66:0/1384610890 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc7002dc30 con 0x7fbc741031c0 2024-04-21T09:13:41.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7419d960 con 0x7fbc741031c0 2024-04-21T09:13:41.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc6affd640 1 -- 172.21.15.66:0/1384610890 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003a10 con 0x7fbc741031c0 2024-04-21T09:13:41.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.396+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc741a5250 con 0x7fbc741031c0 2024-04-21T09:13:41.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc6affd640 1 -- 172.21.15.66:0/1384610890 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbc70046020 con 0x7fbc741031c0 2024-04-21T09:13:41.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 Processor -- start 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 -- start start 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d3d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d4a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b9f4ca21e0 con 0x55b9f4d3d910 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9f4c3f390 con 0x55b9f4d4ae90 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.399+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b9f4c9e1a0 con 0x55b9f4d3cf60 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d4a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d4a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45738/0 (socket says 172.21.15.66:45738) 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 -- 172.21.15.66:0/844251830 learned_addr learned my addr 172.21.15.66:0/844251830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc6affd640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc600a3da0 0x7fbc600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc792d6640 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc7a2d8640 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d3d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc7a2d8640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc600a3da0 0x7fbc600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc6affd640 1 -- 172.21.15.66:0/1384610890 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbc700f2ba0 con 0x7fbc741031c0 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 -- 172.21.15.66:0/844251830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 msgr2=0x55b9f4d3d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d3d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 -- 172.21.15.66:0/844251830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 msgr2=0x55b9f4d4d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 -- 172.21.15.66:0/844251830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f4a7e7d0 con 0x55b9f4d3d910 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d4a950 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbc700be5e0 tx=0x7fbc700056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e8e93292aba6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc5ffff640 1 -- 172.21.15.66:0/844251830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc700be8e0 con 0x55b9f4d3d910 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.400+0000 7fbc7a2d8640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc600a3da0 0x7fbc600a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c005fd0 tx=0x7fbc6c004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc5ffff640 1 -- 172.21.15.66:0/844251830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc700c2c90 con 0x55b9f4d3d910 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc5ffff640 1 -- 172.21.15.66:0/844251830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc700f0200 con 0x55b9f4d3d910 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 -- 172.21.15.66:0/844251830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 msgr2=0x55b9f4d4a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d4a950 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbc700be5e0 tx=0x7fbc700056c0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 -- 172.21.15.66:0/844251830 shutdown_connections 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d4a950 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/844251830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d3d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 -- 172.21.15.66:0/844251830 >> 172.21.15.66:0/844251830 conn(0x55b9f4d37cb0 msgr2=0x55b9f4d38090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 -- 172.21.15.66:0/844251830 shutdown_connections 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.401+0000 7fbc7ea05740 1 -- 172.21.15.66:0/844251830 wait complete. 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 -- start start 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 Processor -- start 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d47e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d483b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b9f4ca21e0 con 0x55b9f4d3d910 2024-04-21T09:13:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9f4c5b2f0 con 0x55b9f4d4ae90 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b9f4d4da10 con 0x55b9f4d3cf60 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc7ea05740 10 client.0 _ll_register_callbacks cb 0x55b9f4c68580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc792d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc792d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45750/0 (socket says 172.21.15.66:45750) 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.402+0000 7fbc792d6640 1 -- 172.21.15.66:0/2572743892 learned_addr learned my addr 172.21.15.66:0/2572743892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d47e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc7a2d8640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc792d6640 1 -- 172.21.15.66:0/2572743892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 msgr2=0x55b9f4d47e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc792d6640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d47e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc792d6640 1 -- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 msgr2=0x55b9f4d488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc792d6640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc792d6640 1 -- 172.21.15.66:0/2572743892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f49b43f0 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d47e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc792d6640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d483b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b8f0 tx=0x7fbc6400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d132b23169da3de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/2572743892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6401b440 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/2572743892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbc6401b5e0 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc700c2c90 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/2572743892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc64024050 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc7ea05740 10 client.17187 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b9f49740f0 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/2572743892 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc700c2c90 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/2572743892 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc64019020 con 0x55b9f4d3d910 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc7ea05740 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.403+0000 7fbc7ea05740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:41.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.404+0000 7fbc7ea05740 20 client.17187 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.404+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4d5aae0 0x55b9f4d5cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.404+0000 7fbc7ea05740 4 client.17187 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.404+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9f4cafc00 con 0x55b9f4d5aae0 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.404+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4d5aae0 0x55b9f4d5cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.404+0000 7fbc5d7fa640 10 client.17187 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.404+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4d5aae0 0x55b9f4d5cef0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e97b3e6f8f45c1 server_cookie=5b2f716422be98af in_seq=0 out_seq=0 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/2572743892 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9f4cafc00 con 0x55b9f4d5aae0 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc5d7fa640 10 client.17187 handle_command_reply: tid=0 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 client.17187 shutdown 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 msgr2=0x55b9f4d483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d483b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b8f0 tx=0x7fbc6400bdb0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 shutdown_connections 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4d5aae0 0x55b9f4d5cef0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d483b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc5d7fa640 0 client.17187 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/2572743892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d47e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 >> 172.21.15.66:0/2572743892 conn(0x55b9f4d37cb0 msgr2=0x55b9f4d39a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 shutdown_connections 2024-04-21T09:13:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.405+0000 7fbc7ea05740 1 -- 172.21.15.66:0/2572743892 wait complete. 2024-04-21T09:13:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.406+0000 7fbc7ea05740 20 client.17187 trim_cache size 0 max 16384 2024-04-21T09:13:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 Processor -- start 2024-04-21T09:13:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 -- start start 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d4a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d56790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741af9c0 con 0x55b9f4d3d910 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741ad290 con 0x55b9f4d3cf60 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741ab0c0 con 0x55b9f4d4ae90 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc792d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d56790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc792d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d56790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45766/0 (socket says 172.21.15.66:45766) 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc792d6640 1 -- 172.21.15.66:0/4093816452 learned_addr learned my addr 172.21.15.66:0/4093816452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc792d6640 1 -- 172.21.15.66:0/4093816452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d4ae90 msgr2=0x55b9f4d62800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc792d6640 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.423+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d4a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc792d6640 1 -- 172.21.15.66:0/4093816452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d3cf60 msgr2=0x55b9f4d4a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc792d6640 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d4a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc792d6640 1 -- 172.21.15.66:0/4093816452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f4d42c60 con 0x55b9f4d3d910 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d4a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc792d6640 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d56790 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400f3f0 tx=0x7fbc64005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e055a1512a50cca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc5f7fe640 1 -- 172.21.15.66:0/4093816452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640039c0 con 0x55b9f4d3d910 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc5f7fe640 1 -- 172.21.15.66:0/4093816452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc6402d070 con 0x55b9f4d3d910 2024-04-21T09:13:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc5f7fe640 1 -- 172.21.15.66:0/4093816452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640054b0 con 0x55b9f4d3d910 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4093816452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 msgr2=0x55b9f4d56790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d56790 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400f3f0 tx=0x7fbc64005990 comp rx=0 tx=0).stop 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4093816452 shutdown_connections 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d62800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3d910 0x55b9f4d56790 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4093816452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d4a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4093816452 >> 172.21.15.66:0/4093816452 conn(0x55b9f4d37cb0 msgr2=0x55b9f4d486c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.424+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4093816452 shutdown_connections 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4093816452 wait complete. 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 -- start start 2024-04-21T09:13:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 Processor -- start 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d52ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3d910 0x55b9f4d531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741ab750 con 0x55b9f4d3cf60 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741acbc0 con 0x55b9f4d4ae90 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741abe20 con 0x55b9f4d3d910 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc7ea05740 10 client.0 _ll_register_callbacks cb 0x7fbc640042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc79ad7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d52ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc79ad7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d52ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45770/0 (socket says 172.21.15.66:45770) 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.425+0000 7fbc79ad7640 1 -- 172.21.15.66:0/4099823814 learned_addr learned my addr 172.21.15.66:0/4099823814 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7a2d8640 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc79ad7640 1 -- 172.21.15.66:0/4099823814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3d910 msgr2=0x55b9f4d531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3d910 0x55b9f4d531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc79ad7640 1 -- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 msgr2=0x55b9f4d4fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc79ad7640 1 -- 172.21.15.66:0/4099823814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f4d502e0 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc79ad7640 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d52ca0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbc700be5e0 tx=0x7fbc70000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf26774a102cbe5d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/4099823814 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc700bab80 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/4099823814 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fbc700baf50 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc6402d070 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/4099823814 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc700c2c90 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 10 client.17193 fetch_fsmap learned FSMap version 25 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b9f4dc1960 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/4099823814 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc6402d070 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/4099823814 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc700c70c0 con 0x55b9f4d3cf60 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 10 client.17193 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:13:41.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 20 client.17193 populate_metadata read hostname 'smithi066' 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4dc1f00 0x55b9f4dc4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 4 client.17193 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.426+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9f4ca9a70 con 0x55b9f4dc1f00 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc792d6640 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4dc1f00 0x55b9f4dc4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc792d6640 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4dc1f00 0x55b9f4dc4320 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e3e384674f588a server_cookie=7093ebe33c791456 in_seq=0 out_seq=0 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc5d7fa640 10 client.17193 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc5d7fa640 1 -- 172.21.15.66:0/4099823814 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b9f4ca9a70 con 0x55b9f4dc1f00 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc5d7fa640 10 client.17193 handle_command_reply: tid=0 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc7ea05740 1 client.17193 shutdown 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3cf60 msgr2=0x55b9f4d52ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.427+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d52ca0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbc700be5e0 tx=0x7fbc70000bc0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 shutdown_connections 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b9f4dc1f00 0x55b9f4dc4320 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f4d4ae90 0x55b9f4d4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9f4d3d910 0x55b9f4d531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 --2- 172.21.15.66:0/4099823814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9f4d3cf60 0x55b9f4d52ca0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc5d7fa640 0 client.17193 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 >> 172.21.15.66:0/4099823814 conn(0x55b9f4d37cb0 msgr2=0x55b9f4d47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 shutdown_connections 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 1 -- 172.21.15.66:0/4099823814 wait complete. 2024-04-21T09:13:41.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7ea05740 20 client.17193 trim_cache size 0 max 16384 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: "e934501b-a5a7-4490-8081-28ad6d2aa506": { 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "e934501b-a5a7-4490-8081-28ad6d2aa506", 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:13:41.436 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:13:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc600a3da0 msgr2=0x7fbc600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc600a3da0 0x7fbc600a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c005fd0 tx=0x7fbc6c004a40 comp rx=0 tx=0).stop 2024-04-21T09:13:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 msgr2=0x7fbc74198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:13:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc74198730 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005940 tx=0x7fbc700045e0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 shutdown_connections 2024-04-21T09:13:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc600a3da0 0x7fbc600a6260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc7410c500 0x7fbc7419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc74103ae0 0x7fbc74198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 --2- 172.21.15.66:0/1384610890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc741031c0 0x7fbc74198730 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:13:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 >> 172.21.15.66:0/1384610890 conn(0x7fbc74077fd0 msgr2=0x7fbc740fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:13:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 shutdown_connections 2024-04-21T09:13:41.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:13:41.428+0000 7fbc7bd3b640 1 -- 172.21.15.66:0/1384610890 wait complete. 2024-04-21T09:13:41.439 INFO:tasks.cephfs.filesystem:scrub status for tag:e934501b-a5a7-4490-8081-28ad6d2aa506 - {'path': '/', 'tag': 'e934501b-a5a7-4490-8081-28ad6d2aa506', 'options': 'recursive,force'} 2024-04-21T09:13:42.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:41 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:13:42.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:41 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:42.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:41 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:13:42.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:41 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:42.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:41 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:13:42.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:41 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:42.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:41 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:13:42.360 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:41 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:42.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:41 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:13:42.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:41 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:42.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:41 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:13:42.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:41 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:13:43.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:42 smithi186 ceph-mon[31505]: pgmap v168: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:43.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:42 smithi083 ceph-mon[31413]: pgmap v168: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:43.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:42 smithi066 ceph-mon[27953]: pgmap v168: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:45 smithi186 ceph-mon[31505]: pgmap v169: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:45 smithi083 ceph-mon[31413]: pgmap v169: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:45 smithi066 ceph-mon[27953]: pgmap v169: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:46.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:46 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:13:46.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:46 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:13:46.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:46 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:13:47.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:47 smithi186 ceph-mon[31505]: pgmap v170: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:47 smithi083 ceph-mon[31413]: pgmap v170: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:47.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:47 smithi066 ceph-mon[27953]: pgmap v170: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:49 smithi186 ceph-mon[31505]: pgmap v171: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:49.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:49 smithi083 ceph-mon[31413]: pgmap v171: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:49 smithi066 ceph-mon[27953]: pgmap v171: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:51 smithi186 ceph-mon[31505]: pgmap v172: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:51.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:51 smithi083 ceph-mon[31413]: pgmap v172: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:51 smithi066 ceph-mon[27953]: pgmap v172: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:53 smithi186 ceph-mon[31505]: pgmap v173: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:53.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:53 smithi083 ceph-mon[31413]: pgmap v173: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:53 smithi066 ceph-mon[27953]: pgmap v173: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:13:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:55 smithi186 ceph-mon[31505]: pgmap v174: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:55 smithi083 ceph-mon[31413]: pgmap v174: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:55.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:55 smithi066 ceph-mon[27953]: pgmap v174: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:13:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:57 smithi186 ceph-mon[31505]: pgmap v175: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:57.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:57 smithi083 ceph-mon[31413]: pgmap v175: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:57 smithi066 ceph-mon[27953]: pgmap v175: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:13:59.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:13:59 smithi186 ceph-mon[31505]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:59.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:13:59 smithi083 ceph-mon[31413]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:13:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:13:59 smithi066 ceph-mon[27953]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:01.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:01 smithi186 ceph-mon[31505]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:01.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:01 smithi083 ceph-mon[31413]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:01.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:01 smithi066 ceph-mon[27953]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:03 smithi186 ceph-mon[31505]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:03 smithi083 ceph-mon[31413]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:03 smithi066 ceph-mon[27953]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:05.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:05 smithi186 ceph-mon[31505]: pgmap v179: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:05.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:05 smithi083 ceph-mon[31413]: pgmap v179: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:05.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:05 smithi066 ceph-mon[27953]: pgmap v179: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:07.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:07 smithi186 ceph-mon[31505]: pgmap v180: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-21T09:14:07.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:07 smithi083 ceph-mon[31413]: pgmap v180: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-21T09:14:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:07 smithi066 ceph-mon[27953]: pgmap v180: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-21T09:14:09.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:09 smithi186 ceph-mon[31505]: pgmap v181: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:09.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:09 smithi083 ceph-mon[31413]: pgmap v181: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:09.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:09 smithi066 ceph-mon[27953]: pgmap v181: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:11.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:14:11.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:14:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 -- 172.21.15.66:0/3907654159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f541043a0 msgr2=0x7f1f54104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 --2- 172.21.15.66:0/3907654159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f541043a0 0x7f1f54104780 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009900 tx=0x7f1f4402bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 -- 172.21.15.66:0/3907654159 shutdown_connections 2024-04-21T09:14:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 --2- 172.21.15.66:0/3907654159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f54109470 0x7f1f54111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 --2- 172.21.15.66:0/3907654159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f54104d50 0x7f1f54108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 --2- 172.21.15.66:0/3907654159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f541043a0 0x7f1f54104780 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 -- 172.21.15.66:0/3907654159 >> 172.21.15.66:0/3907654159 conn(0x7f1f540ffc20 msgr2=0x7f1f54102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 -- 172.21.15.66:0/3907654159 shutdown_connections 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.557+0000 7f1f5932e640 1 -- 172.21.15.66:0/3907654159 wait complete. 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 Processor -- start 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 -- start start 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f541043a0 0x7f1f541a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f54109470 0x7f1f541a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f5419a730 0x7f1f5419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541143b0 con 0x7f1f54109470 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54114230 con 0x7f1f541043a0 2024-04-21T09:14:11.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f5932e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54114530 con 0x7f1f5419a730 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f54109470 0x7f1f541a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f54109470 0x7f1f541a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49198/0 (socket says 172.21.15.66:49198) 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.558+0000 7f1f4bfff640 1 -- 172.21.15.66:0/1226515184 learned_addr learned my addr 172.21.15.66:0/1226515184 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f541043a0 0x7f1f541a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f4bfff640 1 -- 172.21.15.66:0/1226515184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f5419a730 msgr2=0x7f1f5419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f537fe640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f5419a730 0x7f1f5419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f5419a730 0x7f1f5419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f4bfff640 1 -- 172.21.15.66:0/1226515184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f541043a0 msgr2=0x7f1f541a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f541043a0 0x7f1f541a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f4bfff640 1 -- 172.21.15.66:0/1226515184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5419b330 con 0x7f1f54109470 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f537fe640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f5419a730 0x7f1f5419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f54109470 0x7f1f541a0bb0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c970 tx=0x7f1f3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f50ff9640 1 -- 172.21.15.66:0/1226515184 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c014070 con 0x7f1f54109470 2024-04-21T09:14:11.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f541a7410 con 0x7f1f54109470 2024-04-21T09:14:11.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f50ff9640 1 -- 172.21.15.66:0/1226515184 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c004440 con 0x7f1f54109470 2024-04-21T09:14:11.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f50ff9640 1 -- 172.21.15.66:0/1226515184 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c003e50 con 0x7f1f54109470 2024-04-21T09:14:11.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.559+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a7820 con 0x7f1f54109470 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.561+0000 7f1f50ff9640 1 -- 172.21.15.66:0/1226515184 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1f3c017070 con 0x7f1f54109470 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 Processor -- start 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 -- start start 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f50ff9640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1f400a3ec0 0x7f1f400a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f537fe640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1f400a3ec0 0x7f1f400a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f50ff9640 1 -- 172.21.15.66:0/1226515184 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1f3c0d09f0 con 0x7f1f54109470 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b77960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b80540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b80a80 0x559c34b82e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559c34adc1e0 con 0x559c34b77ea0 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c34a79390 con 0x559c34b77580 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559c34ad81a0 con 0x559c34b80a80 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.562+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b80540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b80540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49210/0 (socket says 172.21.15.66:49210) 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 -- 172.21.15.66:0/975980632 learned_addr learned my addr 172.21.15.66:0/975980632 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b77960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f537fe640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1f400a3ec0 0x7f1f400a6380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005fd0 tx=0x7f1f4c0093c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f537fe640 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b80a80 0x559c34b82e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 -- 172.21.15.66:0/975980632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b80a80 msgr2=0x559c34b82e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b80a80 0x559c34b82e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 -- 172.21.15.66:0/975980632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 msgr2=0x559c34b77960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b77960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 -- 172.21.15.66:0/975980632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c348b87d0 con 0x559c34b77ea0 2024-04-21T09:14:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f537fe640 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b80a80 0x559c34b82e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b80540 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00afd0 tx=0x7f1f3c09bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fe41995513f1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f497fa640 1 -- 172.21.15.66:0/975980632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0a09d0 con 0x559c34b77ea0 2024-04-21T09:14:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f497fa640 1 -- 172.21.15.66:0/975980632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c0a5290 con 0x559c34b77ea0 2024-04-21T09:14:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.563+0000 7f1f497fa640 1 -- 172.21.15.66:0/975980632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c085b60 con 0x559c34b77ea0 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 -- 172.21.15.66:0/975980632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 msgr2=0x559c34b80540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b80540 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00afd0 tx=0x7f1f3c09bf80 comp rx=0 tx=0).stop 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 -- 172.21.15.66:0/975980632 shutdown_connections 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b80a80 0x559c34b82e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b80540 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/975980632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b77960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 -- 172.21.15.66:0/975980632 >> 172.21.15.66:0/975980632 conn(0x559c34b71cb0 msgr2=0x559c34b72090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 -- 172.21.15.66:0/975980632 shutdown_connections 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 -- 172.21.15.66:0/975980632 wait complete. 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 -- start start 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.564+0000 7f1f5bff5740 1 Processor -- start 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 0x559c34b7e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b80a80 0x559c34b7e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559c34b83870 con 0x559c34b80a80 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c34a952f0 con 0x559c34b77580 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559c34b839f0 con 0x559c34b77ea0 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f5bff5740 10 client.0 _ll_register_callbacks cb 0x559c34aa2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b80a80 0x559c34b7e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b80a80 0x559c34b7e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49224/0 (socket says 172.21.15.66:49224) 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 -- 172.21.15.66:0/783488414 learned_addr learned my addr 172.21.15.66:0/783488414 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 0x559c34b7e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 -- 172.21.15.66:0/783488414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 msgr2=0x559c34b7e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 0x559c34b7e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 -- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 msgr2=0x559c34b7db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b7db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 -- 172.21.15.66:0/783488414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c347ee3f0 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f537fe640 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b80a80 0x559c34b7e5c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011690 tx=0x7f1f4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaef8adffd8bf92 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.565+0000 7f1f3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/783488414 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c010b10 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/783488414 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1f4c010cb0 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3c0a5290 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/783488414 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f4c01e420 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 10 client.17205 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c347ae0f0 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/783488414 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c0a5290 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/783488414 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f4c0265c0 con 0x559c34b80a80 2024-04-21T09:14:11.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 10 client.17205 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 20 client.17205 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34b90ad0 0x559c34b92f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 4 client.17205 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c34ae9c00 con 0x559c34b90ad0 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.566+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34b90ad0 0x559c34b92f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.567+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34b90ad0 0x559c34b92f00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8a18f1beafcf67 server_cookie=560023f46e71e0fd in_seq=0 out_seq=0 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.567+0000 7f1f3a7fc640 10 client.17205 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/783488414 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c34ae9c00 con 0x559c34b90ad0 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f3a7fc640 10 client.17205 handle_command_reply: tid=0 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 client.17205 shutdown 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b80a80 msgr2=0x559c34b7e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b80a80 0x559c34b7e5c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011690 tx=0x7f1f4c004a40 comp rx=0 tx=0).stop 2024-04-21T09:14:11.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 shutdown_connections 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34b90ad0 0x559c34b92f00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b80a80 0x559c34b7e5c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 0x559c34b7e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f3a7fc640 0 client.17205 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/783488414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b77580 0x559c34b7db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 >> 172.21.15.66:0/783488414 conn(0x559c34b71cb0 msgr2=0x559c34b73a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 shutdown_connections 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 1 -- 172.21.15.66:0/783488414 wait complete. 2024-04-21T09:14:11.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.568+0000 7f1f5bff5740 20 client.17205 trim_cache size 0 max 16384 2024-04-21T09:14:11.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:11 smithi186 ceph-mon[31505]: pgmap v182: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:14:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 Processor -- start 2024-04-21T09:14:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 -- start start 2024-04-21T09:14:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77580 0x559c34b8c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 0x559c34b7d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 0x559c34b987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b3d70 con 0x559c34b77580 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b1640 con 0x559c34b80a80 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541af470 con 0x559c34b77ea0 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77580 0x559c34b8c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77580 0x559c34b8c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49240/0 (socket says 172.21.15.66:49240) 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 -- 172.21.15.66:0/1591157097 learned_addr learned my addr 172.21.15.66:0/1591157097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 -- 172.21.15.66:0/1591157097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 msgr2=0x559c34b7d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/1591157097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 0x559c34b7d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 -- 172.21.15.66:0/1591157097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 msgr2=0x559c34b987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/1591157097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 0x559c34b987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.586+0000 7f1f52ffd640 1 -- 172.21.15.66:0/1591157097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c34b82e70 con 0x559c34b77580 2024-04-21T09:14:11.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/1591157097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77580 0x559c34b8c730 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f4402c170 tx=0x7f1f44009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b8fccdd0d78a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f48ff9640 1 -- 172.21.15.66:0/1591157097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44049050 con 0x559c34b77580 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f48ff9640 1 -- 172.21.15.66:0/1591157097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f44050070 con 0x559c34b77580 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f48ff9640 1 -- 172.21.15.66:0/1591157097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44005420 con 0x559c34b77580 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 -- 172.21.15.66:0/1591157097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77580 msgr2=0x559c34b8c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/1591157097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77580 0x559c34b8c730 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f4402c170 tx=0x7f1f44009560 comp rx=0 tx=0).stop 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 -- 172.21.15.66:0/1591157097 shutdown_connections 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/1591157097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 0x559c34b987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/1591157097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77ea0 0x559c34b7d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/1591157097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77580 0x559c34b8c730 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 -- 172.21.15.66:0/1591157097 >> 172.21.15.66:0/1591157097 conn(0x559c34b71cb0 msgr2=0x559c34b70d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.587+0000 7f1f5bff5740 1 -- 172.21.15.66:0/1591157097 shutdown_connections 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 -- 172.21.15.66:0/1591157097 wait complete. 2024-04-21T09:14:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 -- start start 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 Processor -- start 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77580 0x559c34b84760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b84ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 0x559c34b851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541afb00 con 0x559c34b77ea0 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b0f70 con 0x559c34b80a80 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b01d0 con 0x559c34b77580 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f5bff5740 10 client.0 _ll_register_callbacks cb 0x7f1f4c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b84ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b84ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49250/0 (socket says 172.21.15.66:49250) 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f4bfff640 1 -- 172.21.15.66:0/3023585698 learned_addr learned my addr 172.21.15.66:0/3023585698 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f537fe640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 0x559c34b851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77580 0x559c34b84760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f4bfff640 1 -- 172.21.15.66:0/3023585698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77580 msgr2=0x559c34b84760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77580 0x559c34b84760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.588+0000 7f1f4bfff640 1 -- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 msgr2=0x559c34b851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 0x559c34b851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f4bfff640 1 -- 172.21.15.66:0/3023585698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c34bf6140 con 0x559c34b77ea0 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77580 0x559c34b84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f4bfff640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b84ca0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c09c2d0 tx=0x7f1f3c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1471dce22cf1d5f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/3023585698 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0a5290 con 0x559c34b77ea0 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/3023585698 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f1f3c0bb2d0 con 0x559c34b77ea0 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f44050070 con 0x559c34b77ea0 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/3023585698 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f3c0bb4d0 con 0x559c34b77ea0 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 10 client.17211 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c34bf7450 con 0x559c34b77ea0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/3023585698 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f44050070 con 0x559c34b77ea0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/3023585698 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f3c0983e0 con 0x559c34b77ea0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 10 client.17211 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 20 client.17211 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34bf77e0 0x559c34bf9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 4 client.17211 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.589+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c34ae3a70 con 0x559c34bf77e0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34bf77e0 0x559c34bf9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f3a7fc640 10 client.17211 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f52ffd640 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34bf77e0 0x559c34bf9c00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ad001af9124929 server_cookie=24791d4004fadd36 in_seq=0 out_seq=0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f3a7fc640 1 -- 172.21.15.66:0/3023585698 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559c34ae3a70 con 0x559c34bf77e0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f3a7fc640 10 client.17211 handle_command_reply: tid=0 2024-04-21T09:14:11.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 client.17211 shutdown 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 msgr2=0x559c34b84ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b84ca0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c09c2d0 tx=0x7f1f3c007f90 comp rx=0 tx=0).stop 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 shutdown_connections 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559c34bf77e0 0x559c34bf9c00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c34b80a80 0x559c34b851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c34b77ea0 0x559c34b84ca0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f3a7fc640 0 client.17211 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 --2- 172.21.15.66:0/3023585698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559c34b77580 0x559c34b84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 >> 172.21.15.66:0/3023585698 conn(0x559c34b71cb0 msgr2=0x559c34b7f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 shutdown_connections 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.590+0000 7f1f5bff5740 1 -- 172.21.15.66:0/3023585698 wait complete. 2024-04-21T09:14:11.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5bff5740 20 client.17211 trim_cache size 0 max 16384 2024-04-21T09:14:11.599 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:14:11.599 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:14:11.599 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:14:11.599 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1f400a3ec0 msgr2=0x7f1f400a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1f400a3ec0 0x7f1f400a6380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005fd0 tx=0x7f1f4c0093c0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f54109470 msgr2=0x7f1f541a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f54109470 0x7f1f541a0bb0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c970 tx=0x7f1f3c00ce30 comp rx=0 tx=0).stop 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 shutdown_connections 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1f400a3ec0 0x7f1f400a6380 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f5419a730 0x7f1f5419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f54109470 0x7f1f541a0bb0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 --2- 172.21.15.66:0/1226515184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f541043a0 0x7f1f541a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 >> 172.21.15.66:0/1226515184 conn(0x7f1f540ffc20 msgr2=0x7f1f54101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 shutdown_connections 2024-04-21T09:14:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.591+0000 7f1f5932e640 1 -- 172.21.15.66:0/1226515184 wait complete. 2024-04-21T09:14:11.601 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:11 smithi066 ceph-mon[27953]: pgmap v182: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:14:11.602 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:14:11.603 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:14:11.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:11 smithi083 ceph-mon[31413]: pgmap v182: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.704+0000 7ff12aa25640 1 -- 172.21.15.66:0/4291227327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1241011a0 msgr2=0x7ff124101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.704+0000 7ff12aa25640 1 --2- 172.21.15.66:0/4291227327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1241011a0 0x7ff124101620 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff118009930 tx=0x7ff11802bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 -- 172.21.15.66:0/4291227327 shutdown_connections 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 --2- 172.21.15.66:0/4291227327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124101b60 0x7ff12410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 --2- 172.21.15.66:0/4291227327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1241011a0 0x7ff124101620 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 --2- 172.21.15.66:0/4291227327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff124105e30 0x7ff124106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 -- 172.21.15.66:0/4291227327 >> 172.21.15.66:0/4291227327 conn(0x7ff124077e30 msgr2=0x7ff124078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 -- 172.21.15.66:0/4291227327 shutdown_connections 2024-04-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 -- 172.21.15.66:0/4291227327 wait complete. 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 Processor -- start 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.705+0000 7ff12aa25640 1 -- start start 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1241011a0 0x7ff12419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124101b60 0x7ff12419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124105e30 0x7ff1241985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff124112240 con 0x7ff124105e30 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1241120c0 con 0x7ff124101b60 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff1241123c0 con 0x7ff1241011a0 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124105e30 0x7ff1241985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124105e30 0x7ff1241985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49258/0 (socket says 172.21.15.66:49258) 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 -- 172.21.15.66:0/1638852679 learned_addr learned my addr 172.21.15.66:0/1638852679 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff123fff640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1241011a0 0x7ff12419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff1237fe640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124101b60 0x7ff12419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 -- 172.21.15.66:0/1638852679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1241011a0 msgr2=0x7ff12419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1241011a0 0x7ff12419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 -- 172.21.15.66:0/1638852679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124101b60 msgr2=0x7ff12419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124101b60 0x7ff12419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 -- 172.21.15.66:0/1638852679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff124198d40 con 0x7ff124105e30 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff1237fe640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124101b60 0x7ff12419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff128fc2640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124105e30 0x7ff1241985c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009fd0 tx=0x7ff11c00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff1217fa640 1 -- 172.21.15.66:0/1638852679 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c017070 con 0x7ff124105e30 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff124199030 con 0x7ff124105e30 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff1217fa640 1 -- 172.21.15.66:0/1638852679 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11c003e20 con 0x7ff124105e30 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff1217fa640 1 -- 172.21.15.66:0/1638852679 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c010460 con 0x7ff124105e30 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.706+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1241a52d0 con 0x7ff124105e30 2024-04-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.708+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff124068de0 con 0x7ff124105e30 2024-04-21T09:14:11.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.709+0000 7ff1217fa640 1 -- 172.21.15.66:0/1638852679 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff11c004020 con 0x7ff124105e30 2024-04-21T09:14:11.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.709+0000 7ff1217fa640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff10c0a3d20 0x7ff10c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:11.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.709+0000 7ff123fff640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff10c0a3d20 0x7ff10c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:11.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.709+0000 7ff1217fa640 1 -- 172.21.15.66:0/1638852679 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff11c0d0960 con 0x7ff124105e30 2024-04-21T09:14:11.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.710+0000 7ff123fff640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff10c0a3d20 0x7ff10c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff114009990 tx=0x7ff114009110 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.714+0000 7ff1217fa640 1 -- 172.21.15.66:0/1638852679 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff11c09cc90 con 0x7ff124105e30 2024-04-21T09:14:11.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.941+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff124109380 con 0x7ff124105e30 2024-04-21T09:14:11.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.941+0000 7ff1217fa640 1 -- 172.21.15.66:0/1638852679 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7ff11c09c590 con 0x7ff124105e30 2024-04-21T09:14:11.942 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:14:11.942 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:14:11.942 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:14:11.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff10c0a3d20 msgr2=0x7ff10c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff10c0a3d20 0x7ff10c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff114009990 tx=0x7ff114009110 comp rx=0 tx=0).stop 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124105e30 msgr2=0x7ff1241985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124105e30 0x7ff1241985c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009fd0 tx=0x7ff11c00def0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 shutdown_connections 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff10c0a3d20 0x7ff10c0a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff124105e30 0x7ff1241985c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff124101b60 0x7ff12419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 --2- 172.21.15.66:0/1638852679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1241011a0 0x7ff12419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 >> 172.21.15.66:0/1638852679 conn(0x7ff124077e30 msgr2=0x7ff1240fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 shutdown_connections 2024-04-21T09:14:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:11.944+0000 7ff12aa25640 1 -- 172.21.15.66:0/1638852679 wait complete. 2024-04-21T09:14:11.955 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:14:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 -- 172.21.15.66:0/3551617265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8101a50 msgr2=0x7ff8c8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3551617265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8101a50 0x7ff8c8105a90 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009900 tx=0x7ff8b802bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 -- 172.21.15.66:0/3551617265 shutdown_connections 2024-04-21T09:14:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3551617265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c8106170 0x7ff8c810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3551617265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8101a50 0x7ff8c8105a90 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3551617265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c81010a0 0x7ff8c8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 -- 172.21.15.66:0/3551617265 >> 172.21.15.66:0/3551617265 conn(0x7ff8c8077eb0 msgr2=0x7ff8c80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 -- 172.21.15.66:0/3551617265 shutdown_connections 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.058+0000 7ff8d0715640 1 -- 172.21.15.66:0/3551617265 wait complete. 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 Processor -- start 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 -- start start 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c81010a0 0x7ff8c8199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8106170 0x7ff8c8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c819a060 0x7ff8c81a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8111090 con 0x7ff8c8106170 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8110f10 con 0x7ff8c819a060 2024-04-21T09:14:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8d0715640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8111210 con 0x7ff8c81010a0 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8cdcb0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8106170 0x7ff8c8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8cdcb0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8106170 0x7ff8c8199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49270/0 (socket says 172.21.15.66:49270) 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.059+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/3336931146 learned_addr learned my addr 172.21.15.66:0/3336931146 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c819a060 0x7ff8c81a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c81010a0 0x7ff8c8199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/3336931146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c81010a0 msgr2=0x7ff8c8199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c81010a0 0x7ff8c8199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/3336931146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c819a060 msgr2=0x7ff8c81a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c819a060 0x7ff8c81a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/3336931146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c81a4de0 con 0x7ff8c8106170 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c819a060 0x7ff8c81a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8106170 0x7ff8c8199ad0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8036040 tx=0x7ff8b802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8bf7fe640 1 -- 172.21.15.66:0/3336931146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8043020 con 0x7ff8c8106170 2024-04-21T09:14:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8bf7fe640 1 -- 172.21.15.66:0/3336931146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b802cbe0 con 0x7ff8c8106170 2024-04-21T09:14:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c81a5070 con 0x7ff8c8106170 2024-04-21T09:14:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8bf7fe640 1 -- 172.21.15.66:0/3336931146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8003bf0 con 0x7ff8c8106170 2024-04-21T09:14:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.060+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a5610 con 0x7ff8c8106170 2024-04-21T09:14:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.062+0000 7ff8bf7fe640 1 -- 172.21.15.66:0/3336931146 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff8b8046020 con 0x7ff8c8106170 2024-04-21T09:14:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 Processor -- start 2024-04-21T09:14:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 -- start start 2024-04-21T09:14:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8bf7fe640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8b40a3da0 0x7ff8b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4e250 0x560a30f4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f5b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 0x560a30f5e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560a30eb31f0 con 0x560a30f4e250 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a30e504b0 con 0x560a30f4ec00 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560a30eaf1b0 con 0x560a30f5be10 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8b40a3da0 0x7ff8b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8bf7fe640 1 -- 172.21.15.66:0/3336931146 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff8b80f2cc0 con 0x7ff8c8106170 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8ce4b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4e250 0x560a30f4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8ce4b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4e250 0x560a30f4e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49274/0 (socket says 172.21.15.66:49274) 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.063+0000 7ff8ce4b1640 1 -- 172.21.15.66:0/2792486018 learned_addr learned my addr 172.21.15.66:0/2792486018 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f5b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8ce4b1640 1 -- 172.21.15.66:0/2792486018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 msgr2=0x560a30f5e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 0x560a30f5e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8ce4b1640 1 -- 172.21.15.66:0/2792486018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 msgr2=0x560a30f5b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f5b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8ce4b1640 1 -- 172.21.15.66:0/2792486018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a30e421b0 con 0x560a30f4e250 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f5b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8b40a3da0 0x7ff8b40a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0005c40 tx=0x7ff8c0009900 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4e250 0x560a30f4e630 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400bc30 tx=0x7ff8c400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf2261d478182f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8bcff9640 1 -- 172.21.15.66:0/2792486018 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4015070 con 0x560a30f4e250 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8bcff9640 1 -- 172.21.15.66:0/2792486018 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c401a910 con 0x560a30f4e250 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.064+0000 7ff8bcff9640 1 -- 172.21.15.66:0/2792486018 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4011800 con 0x560a30f4e250 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2792486018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4e250 msgr2=0x560a30f4e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4e250 0x560a30f4e630 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400bc30 tx=0x7ff8c400f730 comp rx=0 tx=0).stop 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2792486018 shutdown_connections 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 0x560a30f5e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f5b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2792486018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4e250 0x560a30f4e630 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2792486018 >> 172.21.15.66:0/2792486018 conn(0x560a30f48ba0 msgr2=0x560a30f48f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2792486018 shutdown_connections 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2792486018 wait complete. 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- start start 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 Processor -- start 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f5b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4ec00 0x560a30f58a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f58fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560a30f5ed40 con 0x560a30f5be10 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a30e6c500 con 0x560a30f4e250 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.065+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560a30f5eec0 con 0x560a30f4ec00 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8d33dd740 10 client.0 _ll_register_callbacks cb 0x560a30e3ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f58fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f58fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49288/0 (socket says 172.21.15.66:49288) 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 -- 172.21.15.66:0/191554232 learned_addr learned my addr 172.21.15.66:0/191554232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4ec00 0x560a30f58a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f5b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 -- 172.21.15.66:0/191554232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4ec00 msgr2=0x560a30f58a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4ec00 0x560a30f58a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 -- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 msgr2=0x560a30f5b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f5b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 -- 172.21.15.66:0/191554232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a30bc53f0 con 0x560a30f5be10 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f5b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f58fd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0013860 tx=0x7ff8c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9129f8e670824693 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/191554232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0024070 con 0x560a30f5be10 2024-04-21T09:14:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/191554232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff8c000fae0 con 0x560a30f5be10 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8c401a910 con 0x560a30f5be10 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/191554232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8c0030050 con 0x560a30f5be10 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8d33dd740 10 client.17229 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.066+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a30c8f7d0 con 0x560a30f5be10 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/191554232 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c401a910 con 0x560a30f5be10 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/191554232 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff8c000f040 con 0x560a30f5be10 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8d33dd740 10 client.17229 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8d33dd740 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8d33dd740 20 client.17229 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30f6bf00 0x560a30f6e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8d33dd740 4 client.17229 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a30e52930 con 0x560a30f6bf00 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.067+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30f6bf00 0x560a30f6e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.068+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30f6bf00 0x560a30f6e380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7c41eae5a66a192 server_cookie=f8b5d9ef731a5efb in_seq=0 out_seq=0 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.068+0000 7ff8b1ffb640 10 client.17229 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.068+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/191554232 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a30e52930 con 0x560a30f6bf00 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.068+0000 7ff8b1ffb640 10 client.17229 handle_command_reply: tid=0 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.068+0000 7ff8d33dd740 1 client.17229 shutdown 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 msgr2=0x560a30f58fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f58fd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0013860 tx=0x7ff8c0004ab0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 shutdown_connections 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30f6bf00 0x560a30f6e380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f58fd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4ec00 0x560a30f58a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8b1ffb640 0 client.17229 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/191554232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f5b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 >> 172.21.15.66:0/191554232 conn(0x560a30f48ba0 msgr2=0x560a30f4a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 shutdown_connections 2024-04-21T09:14:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 1 -- 172.21.15.66:0/191554232 wait complete. 2024-04-21T09:14:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.069+0000 7ff8d33dd740 20 client.17229 trim_cache size 0 max 16384 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 Processor -- start 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 -- start start 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4e250 0x560a30f67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f57bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f47a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81af9b0 con 0x560a30f5be10 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81ad280 con 0x560a30f4ec00 2024-04-21T09:14:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.082+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81ab0b0 con 0x560a30f4e250 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f47a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f47a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49290/0 (socket says 172.21.15.66:49290) 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 -- 172.21.15.66:0/3601268887 learned_addr learned my addr 172.21.15.66:0/3601268887 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 -- 172.21.15.66:0/3601268887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4e250 msgr2=0x560a30f67c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4e250 0x560a30f67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 -- 172.21.15.66:0/3601268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 msgr2=0x560a30f57bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f57bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f57bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 -- 172.21.15.66:0/3601268887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a30f4e9a0 con 0x560a30f5be10 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8cecb2640 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f47a80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00099e0 tx=0x7ff8c000f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7406af55f5a9701e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8b3fff640 1 -- 172.21.15.66:0/3601268887 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0038070 con 0x560a30f5be10 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8b3fff640 1 -- 172.21.15.66:0/3601268887 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0017070 con 0x560a30f5be10 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.083+0000 7ff8b3fff640 1 -- 172.21.15.66:0/3601268887 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0024050 con 0x560a30f5be10 2024-04-21T09:14:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- 172.21.15.66:0/3601268887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 msgr2=0x560a30f47a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f47a80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00099e0 tx=0x7ff8c000f1f0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- 172.21.15.66:0/3601268887 shutdown_connections 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f5be10 0x560a30f47a80 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4ec00 0x560a30f57bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/3601268887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f4e250 0x560a30f67c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- 172.21.15.66:0/3601268887 >> 172.21.15.66:0/3601268887 conn(0x560a30f48ba0 msgr2=0x560a30f47370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- 172.21.15.66:0/3601268887 shutdown_connections 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- 172.21.15.66:0/3601268887 wait complete. 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- start start 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 Processor -- start 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4ec00 0x560a30f5f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 0x560a30f5fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81ab740 con 0x560a30f4ec00 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81acbb0 con 0x560a30f4e250 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81abe10 con 0x560a30f5be10 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.084+0000 7ff8d33dd740 10 client.0 _ll_register_callbacks cb 0x7ff8c0013560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4ec00 0x560a30f5f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4ec00 0x560a30f5f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49298/0 (socket says 172.21.15.66:49298) 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/2874415183 learned_addr learned my addr 172.21.15.66:0/2874415183 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/2874415183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 msgr2=0x560a30f5fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 0x560a30f5fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 msgr2=0x560a30f64220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f64220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 -- 172.21.15.66:0/2874415183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a30f60190 con 0x560a30f4ec00 2024-04-21T09:14:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f64220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8cdcb0640 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4ec00 0x560a30f5f710 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80bae60 tx=0x7ff8b80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb66de29cf6e838f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/2874415183 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80bf100 con 0x560a30f4ec00 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/2874415183 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ff8b80bb070 con 0x560a30f4ec00 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8c0017070 con 0x560a30f4ec00 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/2874415183 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8b80bea00 con 0x560a30f4ec00 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8d33dd740 10 client.17235 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a30fd2280 con 0x560a30f4ec00 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.085+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/2874415183 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0017070 con 0x560a30f4ec00 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/2874415183 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff8b80ddbc0 con 0x560a30f4ec00 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8d33dd740 10 client.17235 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8d33dd740 10 client.17235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8d33dd740 20 client.17235 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30fd2a30 0x560a30fd4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8d33dd740 4 client.17235 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a30ec0c10 con 0x560a30fd2a30 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30fd2a30 0x560a30fd4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8b1ffb640 10 client.17235 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.086+0000 7ff8ce4b1640 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30fd2a30 0x560a30fd4e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24fc2e4abc962a45 server_cookie=9b8d075e218be7eb in_seq=0 out_seq=0 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8b1ffb640 1 -- 172.21.15.66:0/2874415183 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a30ec0c10 con 0x560a30fd2a30 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8b1ffb640 10 client.17235 handle_command_reply: tid=0 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 client.17235 shutdown 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4ec00 msgr2=0x560a30f5f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4ec00 0x560a30f5f710 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80bae60 tx=0x7ff8b80036f0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 shutdown_connections 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x560a30fd2a30 0x560a30fd4e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a30f5be10 0x560a30f5fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8b1ffb640 0 client.17235 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a30f4ec00 0x560a30f5f710 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 --2- 172.21.15.66:0/2874415183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a30f4e250 0x560a30f64220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 >> 172.21.15.66:0/2874415183 conn(0x560a30f48ba0 msgr2=0x560a30f4a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 shutdown_connections 2024-04-21T09:14:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 1 -- 172.21.15.66:0/2874415183 wait complete. 2024-04-21T09:14:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d33dd740 20 client.17235 trim_cache size 0 max 16384 2024-04-21T09:14:12.094 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:14:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8b40a3da0 msgr2=0x7ff8b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8b40a3da0 0x7ff8b40a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0005c40 tx=0x7ff8c0009900 comp rx=0 tx=0).stop 2024-04-21T09:14:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8106170 msgr2=0x7ff8c8199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8106170 0x7ff8c8199ad0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8036040 tx=0x7ff8b802c7e0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 shutdown_connections 2024-04-21T09:14:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff8b40a3da0 0x7ff8b40a6260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c819a060 0x7ff8c81a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.087+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8c8106170 0x7ff8c8199ad0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.088+0000 7ff8d0715640 1 --2- 172.21.15.66:0/3336931146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c81010a0 0x7ff8c8199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.088+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 >> 172.21.15.66:0/3336931146 conn(0x7ff8c8077eb0 msgr2=0x7ff8c80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.088+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 shutdown_connections 2024-04-21T09:14:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.088+0000 7ff8d0715640 1 -- 172.21.15.66:0/3336931146 wait complete. 2024-04-21T09:14:12.099 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:14:12.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 -- 172.21.15.66:0/2919427784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31141010a0 msgr2=0x7f3114101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 --2- 172.21.15.66:0/2919427784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31141010a0 0x7f3114101480 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3104009ca0 tx=0x7f310402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 -- 172.21.15.66:0/2919427784 shutdown_connections 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 --2- 172.21.15.66:0/2919427784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3114106170 0x7f311410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 --2- 172.21.15.66:0/2919427784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3114101a50 0x7f3114105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 --2- 172.21.15.66:0/2919427784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31141010a0 0x7f3114101480 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 -- 172.21.15.66:0/2919427784 >> 172.21.15.66:0/2919427784 conn(0x7f3114077eb0 msgr2=0x7f31140782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 -- 172.21.15.66:0/2919427784 shutdown_connections 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 -- 172.21.15.66:0/2919427784 wait complete. 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.203+0000 7f311b76d640 1 Processor -- start 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f311b76d640 1 -- start start 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f311b76d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31141010a0 0x7f31141975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f311b76d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3114106170 0x7f3114197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f311b76d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311419bea0 0x7f311419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f311b76d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31141110e0 con 0x7f3114106170 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f311b76d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3114110f60 con 0x7f31141010a0 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f311b76d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3114111260 con 0x7f311419bea0 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3114106170 0x7f3114197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3114106170 0x7f3114197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49304/0 (socket says 172.21.15.66:49304) 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 -- 172.21.15.66:0/2716065841 learned_addr learned my addr 172.21.15.66:0/2716065841 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3119509640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31141010a0 0x7f31141975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3119d0a640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311419bea0 0x7f311419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 -- 172.21.15.66:0/2716065841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311419bea0 msgr2=0x7f311419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311419bea0 0x7f311419c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 -- 172.21.15.66:0/2716065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31141010a0 msgr2=0x7f31141975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31141010a0 0x7f31141975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 -- 172.21.15.66:0/2716065841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311419c990 con 0x7f3114106170 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f3118d08640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3114106170 0x7f3114197b20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f310800d770 tx=0x7f310800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.204+0000 7f31127fc640 1 -- 172.21.15.66:0/2716065841 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108017070 con 0x7f3114106170 2024-04-21T09:14:12.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.205+0000 7f31127fc640 1 -- 172.21.15.66:0/2716065841 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3108009c30 con 0x7f3114106170 2024-04-21T09:14:12.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.205+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311419cc80 con 0x7f3114106170 2024-04-21T09:14:12.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.205+0000 7f31127fc640 1 -- 172.21.15.66:0/2716065841 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310800f620 con 0x7f3114106170 2024-04-21T09:14:12.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.205+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31141a44c0 con 0x7f3114106170 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.207+0000 7f31127fc640 1 -- 172.21.15.66:0/2716065841 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f310801a070 con 0x7f3114106170 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.207+0000 7f311e434740 1 Processor -- start 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f311e434740 1 -- start start 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f311e434740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139154720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f311e434740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154cf0 0x55c13915b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f311e434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f311e434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c1390b91f0 con 0x55c13915bbf0 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f311e434740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c1390564b0 con 0x55c139154cf0 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f311e434740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c1390b51b0 con 0x55c139154340 2024-04-21T09:14:12.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f31127fc640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30fc0a3da0 0x7f30fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49318/0 (socket says 172.21.15.66:49318) 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3119509640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154cf0 0x55c13915b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 -- 172.21.15.66:0/4027557541 learned_addr learned my addr 172.21.15.66:0/4027557541 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3119d0a640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30fc0a3da0 0x7f30fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3119d0a640 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f31127fc640 1 -- 172.21.15.66:0/2716065841 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f31080d0660 con 0x7f3114106170 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 -- 172.21.15.66:0/4027557541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 msgr2=0x55c139154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 -- 172.21.15.66:0/4027557541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154cf0 msgr2=0x55c13915b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154cf0 0x55c13915b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3118d08640 1 -- 172.21.15.66:0/4027557541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1390481b0 con 0x55c13915bbf0 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.208+0000 7f3119d0a640 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:12.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.209+0000 7f3118d08640 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915c070 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f31080cd980 tx=0x7f310800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0259758a754d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.209+0000 7f3119d0a640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30fc0a3da0 0x7f30fc0a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f310c005d80 tx=0x7f310c005cf0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.209+0000 7f30fb7fe640 1 -- 172.21.15.66:0/4027557541 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108021030 con 0x55c13915bbf0 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.209+0000 7f30fb7fe640 1 -- 172.21.15.66:0/4027557541 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310809c290 con 0x55c13915bbf0 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.209+0000 7f30fb7fe640 1 -- 172.21.15.66:0/4027557541 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080cdcc0 con 0x55c13915bbf0 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- 172.21.15.66:0/4027557541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 msgr2=0x55c13915c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915c070 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f31080cd980 tx=0x7f310800de80 comp rx=0 tx=0).stop 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- 172.21.15.66:0/4027557541 shutdown_connections 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915c070 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154cf0 0x55c13915b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 --2- 172.21.15.66:0/4027557541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- 172.21.15.66:0/4027557541 >> 172.21.15.66:0/4027557541 conn(0x55c13914eba0 msgr2=0x55c13914ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- 172.21.15.66:0/4027557541 shutdown_connections 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- 172.21.15.66:0/4027557541 wait complete. 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- start start 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 Processor -- start 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c139161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154cf0 0x55c13915f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c139162b30 con 0x55c13915bbf0 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c139072500 con 0x55c139154340 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c139162cb0 con 0x55c139154cf0 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f311e434740 10 client.0 _ll_register_callbacks cb 0x55c139040b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.210+0000 7f3119d0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49320/0 (socket says 172.21.15.66:49320) 2024-04-21T09:14:12.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 -- 172.21.15.66:0/771111738 learned_addr learned my addr 172.21.15.66:0/771111738 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119509640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c139161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3118d08640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154cf0 0x55c13915f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 -- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154cf0 msgr2=0x55c13915f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154cf0 0x55c13915f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 -- 172.21.15.66:0/771111738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 msgr2=0x55c139161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c139161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 -- 172.21.15.66:0/771111738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c138dcb3f0 con 0x55c13915bbf0 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119509640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c139161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3118d08640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154cf0 0x55c13915f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f3119d0a640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915f870 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f310c012b80 tx=0x7f310c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534361e10c91ca0b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f30f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f30f8ff9640 1 -- 172.21.15.66:0/771111738 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c024030 con 0x55c13915bbf0 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310809c290 con 0x55c13915bbf0 2024-04-21T09:14:12.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f30f8ff9640 1 -- 172.21.15.66:0/771111738 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f310c012c70 con 0x55c13915bbf0 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f30f8ff9640 1 -- 172.21.15.66:0/771111738 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f310c031050 con 0x55c13915bbf0 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f311e434740 10 client.17247 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c138e957d0 con 0x55c13915bbf0 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.211+0000 7f30f8ff9640 1 -- 172.21.15.66:0/771111738 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310809c290 con 0x55c13915bbf0 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f30f8ff9640 1 -- 172.21.15.66:0/771111738 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f310c01d050 con 0x55c13915bbf0 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f311e434740 10 client.17247 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f311e434740 10 client.17247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f311e434740 20 client.17247 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f311e434740 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c13916fd90 0x55c1391721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f311e434740 4 client.17247 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c139058930 con 0x55c13916fd90 2024-04-21T09:14:12.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f3119509640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c13916fd90 0x55c1391721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f30f8ff9640 10 client.17247 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.212+0000 7f3119509640 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c13916fd90 0x55c1391721f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d02928eb3bef057 server_cookie=d1c6853c3993e0a8 in_seq=0 out_seq=0 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f30f8ff9640 1 -- 172.21.15.66:0/771111738 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c139058930 con 0x55c13916fd90 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f30f8ff9640 10 client.17247 handle_command_reply: tid=0 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 client.17247 shutdown 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 msgr2=0x55c13915f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915f870 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f310c012b80 tx=0x7f310c004730 comp rx=0 tx=0).stop 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 shutdown_connections 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c13916fd90 0x55c1391721f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c13915bbf0 0x55c13915f870 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154cf0 0x55c13915f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f30f8ff9640 0 client.17247 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:12.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 --2- 172.21.15.66:0/771111738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c139161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 >> 172.21.15.66:0/771111738 conn(0x55c13914eba0 msgr2=0x55c1391508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.213+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 shutdown_connections 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.214+0000 7f311e434740 1 -- 172.21.15.66:0/771111738 wait complete. 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.214+0000 7f311e434740 20 client.17247 trim_cache size 0 max 16384 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 Processor -- start 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 -- start start 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c13915ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c13915b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c13915bbf0 0x55c13915b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31141af8e0 con 0x55c139154cf0 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31141ad1b0 con 0x55c139154340 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f311e434740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f31141aafe0 con 0x55c13915bbf0 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c13915b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c13915b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49334/0 (socket says 172.21.15.66:49334) 2024-04-21T09:14:12.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 -- 172.21.15.66:0/3426205570 learned_addr learned my addr 172.21.15.66:0/3426205570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3119509640 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c13915ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 -- 172.21.15.66:0/3426205570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c13915bbf0 msgr2=0x55c13915b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c13915bbf0 0x55c13915b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 -- 172.21.15.66:0/3426205570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 msgr2=0x55c13915ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c13915ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3118d08640 1 -- 172.21.15.66:0/3426205570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c139162bd0 con 0x55c139154cf0 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.227+0000 7f3119509640 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c13915ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f3118d08640 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c13915b050 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3108010bb0 tx=0x7f31080d08e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e11b183fef49f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f30faffd640 1 -- 172.21.15.66:0/3426205570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108021040 con 0x55c139154cf0 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f30faffd640 1 -- 172.21.15.66:0/3426205570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31080a4f00 con 0x55c139154cf0 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f30faffd640 1 -- 172.21.15.66:0/3426205570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108098900 con 0x55c139154cf0 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 -- 172.21.15.66:0/3426205570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 msgr2=0x55c13915b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c13915b050 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3108010bb0 tx=0x7f31080d08e0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 -- 172.21.15.66:0/3426205570 shutdown_connections 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c13915bbf0 0x55c13915b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c13915b050 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 --2- 172.21.15.66:0/3426205570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c139154340 0x55c13915ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 -- 172.21.15.66:0/3426205570 >> 172.21.15.66:0/3426205570 conn(0x55c13914eba0 msgr2=0x55c13915f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 -- 172.21.15.66:0/3426205570 shutdown_connections 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.228+0000 7f311e434740 1 -- 172.21.15.66:0/3426205570 wait complete. 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 -- start start 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 Processor -- start 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139177200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c139177740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c13915bbf0 0x55c139177c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31141ab670 con 0x55c139154cf0 2024-04-21T09:14:12.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31141acae0 con 0x55c13915bbf0 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f31141abd40 con 0x55c139154340 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f311e434740 10 client.0 _ll_register_callbacks cb 0x7f310c012830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c139177740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c139177740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49350/0 (socket says 172.21.15.66:49350) 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 -- 172.21.15.66:0/3396084097 learned_addr learned my addr 172.21.15.66:0/3396084097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3119d0a640 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c13915bbf0 0x55c139177c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3119509640 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139177200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 -- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 msgr2=0x55c139177200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139177200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 -- 172.21.15.66:0/3396084097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c13915bbf0 msgr2=0x55c139177c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c13915bbf0 0x55c139177c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.229+0000 7f3118d08640 1 -- 172.21.15.66:0/3396084097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1391d5530 con 0x55c139154cf0 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f3118d08640 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c139177740 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f310809bf40 tx=0x7f31080a06c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b323a3c9079a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f30f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f30f8ff9640 1 -- 172.21.15.66:0/3396084097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108021070 con 0x55c139154cf0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f30f8ff9640 1 -- 172.21.15.66:0/3396084097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f31080cdcc0 con 0x55c139154cf0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31080a4f00 con 0x55c139154cf0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f30f8ff9640 1 -- 172.21.15.66:0/3396084097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31080cdec0 con 0x55c139154cf0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 10 client.17253 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1391d6810 con 0x55c139154cf0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f30f8ff9640 1 -- 172.21.15.66:0/3396084097 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31080a4f00 con 0x55c139154cf0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f30f8ff9640 1 -- 172.21.15.66:0/3396084097 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f31080a4a70 con 0x55c139154cf0 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 10 client.17253 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 10 client.17253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 20 client.17253 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c1391d6a40 0x55c1391d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 4 client.17253 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.230+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1390c6c10 con 0x55c1391d6a40 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.231+0000 7f3119509640 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c1391d6a40 0x55c1391d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.231+0000 7f30f8ff9640 10 client.17253 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.231+0000 7f3119509640 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c1391d6a40 0x55c1391d8e30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d960e1b1698e8ec server_cookie=63235f8dc220b556 in_seq=0 out_seq=0 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.231+0000 7f30f8ff9640 1 -- 172.21.15.66:0/3396084097 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1390c6c10 con 0x55c1391d6a40 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.231+0000 7f30f8ff9640 10 client.17253 handle_command_reply: tid=0 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.231+0000 7f311e434740 1 client.17253 shutdown 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 msgr2=0x55c139177740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c139177740 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f310809bf40 tx=0x7f31080a06c0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 shutdown_connections 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c13915bbf0 0x55c139177c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55c1391d6a40 0x55c1391d8e30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c139154cf0 0x55c139177740 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 --2- 172.21.15.66:0/3396084097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c139154340 0x55c139177200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 >> 172.21.15.66:0/3396084097 conn(0x55c13914eba0 msgr2=0x55c13915ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f30f8ff9640 0 client.17253 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:12.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 shutdown_connections 2024-04-21T09:14:12.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 1 -- 172.21.15.66:0/3396084097 wait complete. 2024-04-21T09:14:12.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311e434740 20 client.17253 trim_cache size 0 max 16384 2024-04-21T09:14:12.510 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:14:12.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30fc0a3da0 msgr2=0x7f30fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30fc0a3da0 0x7f30fc0a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f310c005d80 tx=0x7f310c005cf0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3114106170 msgr2=0x7f3114197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3114106170 0x7f3114197b20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f310800d770 tx=0x7f310800dc30 comp rx=0 tx=0).stop 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 shutdown_connections 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30fc0a3da0 0x7f30fc0a6260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311419bea0 0x7f311419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3114106170 0x7f3114197b20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 --2- 172.21.15.66:0/2716065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31141010a0 0x7f31141975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 >> 172.21.15.66:0/2716065841 conn(0x7f3114077eb0 msgr2=0x7f31140fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.232+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 shutdown_connections 2024-04-21T09:14:12.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.233+0000 7f311b76d640 1 -- 172.21.15.66:0/2716065841 wait complete. 2024-04-21T09:14:12.513 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:14:12.535 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1638852679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:14:12.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1638852679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:14:12.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1638852679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:14:12.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 -- 172.21.15.66:0/806750582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 msgr2=0x7f89f010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/806750582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f010bfd0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009900 tx=0x7f89d802bd60 comp rx=0 tx=0).stop 2024-04-21T09:14:12.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 -- 172.21.15.66:0/806750582 shutdown_connections 2024-04-21T09:14:12.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/806750582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f010c510 0x7f89f010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/806750582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f010bfd0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/806750582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01031d0 0x7f89f01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 -- 172.21.15.66:0/806750582 >> 172.21.15.66:0/806750582 conn(0x7f89f0077f50 msgr2=0x7f89f0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 -- 172.21.15.66:0/806750582 shutdown_connections 2024-04-21T09:14:12.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.616+0000 7f89f4f6d640 1 -- 172.21.15.66:0/806750582 wait complete. 2024-04-21T09:14:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.617+0000 7f89f4f6d640 1 Processor -- start 2024-04-21T09:14:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.617+0000 7f89f4f6d640 1 -- start start 2024-04-21T09:14:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89f4f6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01031d0 0x7f89f0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89f4f6d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f0198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89f4f6d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f010c510 0x7f89f019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89f4f6d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01121a0 con 0x7f89f0103af0 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89f4f6d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0112020 con 0x7f89f01031d0 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89f4f6d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0112320 con 0x7f89f010c510 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f0198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f0198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49370/0 (socket says 172.21.15.66:49370) 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 -- 172.21.15.66:0/3479506314 learned_addr learned my addr 172.21.15.66:0/3479506314 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89eed9d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f010c510 0x7f89f019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89ee59c640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01031d0 0x7f89f0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 -- 172.21.15.66:0/3479506314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f010c510 msgr2=0x7f89f019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f010c510 0x7f89f019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 -- 172.21.15.66:0/3479506314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01031d0 msgr2=0x7f89f0198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01031d0 0x7f89f0198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 -- 172.21.15.66:0/3479506314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f019d720 con 0x7f89f0103af0 2024-04-21T09:14:12.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f0198cc0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f89d8005940 tx=0x7f89d80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.618+0000 7f89e7fff640 1 -- 172.21.15.66:0/3479506314 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8043020 con 0x7f89f0103af0 2024-04-21T09:14:12.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.619+0000 7f89e7fff640 1 -- 172.21.15.66:0/3479506314 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d802dc30 con 0x7f89f0103af0 2024-04-21T09:14:12.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.619+0000 7f89e7fff640 1 -- 172.21.15.66:0/3479506314 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003a10 con 0x7f89f0103af0 2024-04-21T09:14:12.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.619+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f019d9b0 con 0x7f89f0103af0 2024-04-21T09:14:12.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.619+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f01a52a0 con 0x7f89f0103af0 2024-04-21T09:14:12.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.621+0000 7f89e7fff640 1 -- 172.21.15.66:0/3479506314 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f89d8046020 con 0x7f89f0103af0 2024-04-21T09:14:12.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.621+0000 7f89f7c33740 1 Processor -- start 2024-04-21T09:14:12.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89f7c33740 1 -- start start 2024-04-21T09:14:12.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89e7fff640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f89dc0a3d20 0x7f89dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae20e80 0x564aeae29430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae29a00 0x564aeae2be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89f7c33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564aead851f0 con 0x564aeae204d0 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89eed9d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f89dc0a3d20 0x7f89dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89e7fff640 1 -- 172.21.15.66:0/3479506314 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f89d80f2b20 con 0x7f89f0103af0 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89f7c33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564aead224b0 con 0x564aeae20e80 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89f7c33740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564aead811b0 con 0x564aeae29a00 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49384/0 (socket says 172.21.15.66:49384) 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 -- 172.21.15.66:0/1586154637 learned_addr learned my addr 172.21.15.66:0/1586154637 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae20e80 0x564aeae29430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89eed9d640 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae29a00 0x564aeae2be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 -- 172.21.15.66:0/1586154637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae29a00 msgr2=0x564aeae2be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae29a00 0x564aeae2be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 -- 172.21.15.66:0/1586154637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae20e80 msgr2=0x564aeae29430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae20e80 0x564aeae29430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.622+0000 7f89ee59c640 1 -- 172.21.15.66:0/1586154637 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aead141b0 con 0x564aeae204d0 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.623+0000 7f89eed9d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f89dc0a3d20 0x7f89dc0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009fd0 tx=0x7f89e8014b70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.623+0000 7f89ee59c640 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae208b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009cf0 tx=0x7f89e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c39a1ae266b4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.623+0000 7f89e4d99640 1 -- 172.21.15.66:0/1586154637 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0004220 con 0x564aeae204d0 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.623+0000 7f89e4d99640 1 -- 172.21.15.66:0/1586154637 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e00043c0 con 0x564aeae204d0 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.623+0000 7f89e4d99640 1 -- 172.21.15.66:0/1586154637 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0018730 con 0x564aeae204d0 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- 172.21.15.66:0/1586154637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 msgr2=0x564aeae208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae208b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009cf0 tx=0x7f89e0011810 comp rx=0 tx=0).stop 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- 172.21.15.66:0/1586154637 shutdown_connections 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae29a00 0x564aeae2be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae20e80 0x564aeae29430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 --2- 172.21.15.66:0/1586154637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae208b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- 172.21.15.66:0/1586154637 >> 172.21.15.66:0/1586154637 conn(0x564aeae1aba0 msgr2=0x564aeae1af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- 172.21.15.66:0/1586154637 shutdown_connections 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- 172.21.15.66:0/1586154637 wait complete. 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- start start 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 Processor -- start 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae204d0 0x564aeae292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae26640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae29a00 0x564aeae26b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564aeae2c970 con 0x564aeae29a00 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564aead3e500 con 0x564aeae204d0 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564aeae2caf0 con 0x564aeae20e80 2024-04-21T09:14:12.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89f7c33740 10 client.0 _ll_register_callbacks cb 0x564aead0cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.624+0000 7f89eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae29a00 0x564aeae26b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae29a00 0x564aeae26b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49392/0 (socket says 172.21.15.66:49392) 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 -- 172.21.15.66:0/2164446714 learned_addr learned my addr 172.21.15.66:0/2164446714 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89ee59c640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae204d0 0x564aeae292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae26640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 -- 172.21.15.66:0/2164446714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 msgr2=0x564aeae26640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae26640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 -- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae204d0 msgr2=0x564aeae292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae204d0 0x564aeae292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 -- 172.21.15.66:0/2164446714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aeaa973f0 con 0x564aeae29a00 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae26640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae29a00 0x564aeae26b80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f89e8010ab0 tx=0x7f89e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5680642fb462e534 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2164446714 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8026070 con 0x564aeae29a00 2024-04-21T09:14:12.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2164446714 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f89e8020410 con 0x564aeae29a00 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e00043c0 con 0x564aeae29a00 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2164446714 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89e8032050 con 0x564aeae29a00 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89f7c33740 10 client.17265 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.625+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564aeab617d0 con 0x564aeae29a00 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2164446714 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e00043c0 con 0x564aeae29a00 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2164446714 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89e800f040 con 0x564aeae29a00 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89f7c33740 10 client.17265 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89f7c33740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89f7c33740 20 client.17265 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae39b50 0x564aeae3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89f7c33740 4 client.17265 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aead24930 con 0x564aeae39b50 2024-04-21T09:14:12.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89ee59c640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae39b50 0x564aeae3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89d5ffb640 10 client.17265 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.626+0000 7f89ee59c640 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae39b50 0x564aeae3bfa0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87289e94db0d253 server_cookie=6f57a46f087c0f8a in_seq=0 out_seq=0 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2164446714 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564aead24930 con 0x564aeae39b50 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89d5ffb640 10 client.17265 handle_command_reply: tid=0 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 client.17265 shutdown 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae29a00 msgr2=0x564aeae26b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae29a00 0x564aeae26b80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f89e8010ab0 tx=0x7f89e8004ab0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 shutdown_connections 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae39b50 0x564aeae3bfa0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae29a00 0x564aeae26b80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae26640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89d5ffb640 0 client.17265 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2164446714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae204d0 0x564aeae292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 >> 172.21.15.66:0/2164446714 conn(0x564aeae1aba0 msgr2=0x564aeae1c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 shutdown_connections 2024-04-21T09:14:12.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 1 -- 172.21.15.66:0/2164446714 wait complete. 2024-04-21T09:14:12.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.627+0000 7f89f7c33740 20 client.17265 trim_cache size 0 max 16384 2024-04-21T09:14:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 Processor -- start 2024-04-21T09:14:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 -- start start 2024-04-21T09:14:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae35830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae204d0 0x564aeae28d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae20e80 0x564aeae292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01b1b80 con 0x564aeae20e80 2024-04-21T09:14:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01af450 con 0x564aeae1aba0 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89f7c33740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01ad280 con 0x564aeae204d0 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae20e80 0x564aeae292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae20e80 0x564aeae292b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49394/0 (socket says 172.21.15.66:49394) 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 -- 172.21.15.66:0/2994220124 learned_addr learned my addr 172.21.15.66:0/2994220124 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89ee59c640 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae35830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 -- 172.21.15.66:0/2994220124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae204d0 msgr2=0x564aeae28d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae204d0 0x564aeae28d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 -- 172.21.15.66:0/2994220124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 msgr2=0x564aeae35830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae35830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 -- 172.21.15.66:0/2994220124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aeae257a0 con 0x564aeae20e80 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.641+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae20e80 0x564aeae292b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f89e80096e0 tx=0x7f89e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbd92d6faf1cd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89d7fff640 1 -- 172.21.15.66:0/2994220124 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e802f070 con 0x564aeae20e80 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89d7fff640 1 -- 172.21.15.66:0/2994220124 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e8009b70 con 0x564aeae20e80 2024-04-21T09:14:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89d7fff640 1 -- 172.21.15.66:0/2994220124 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8026040 con 0x564aeae20e80 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 -- 172.21.15.66:0/2994220124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae20e80 msgr2=0x564aeae292b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae20e80 0x564aeae292b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f89e80096e0 tx=0x7f89e8004a40 comp rx=0 tx=0).stop 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 -- 172.21.15.66:0/2994220124 shutdown_connections 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae20e80 0x564aeae292b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae204d0 0x564aeae28d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2994220124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae35830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 -- 172.21.15.66:0/2994220124 >> 172.21.15.66:0/2994220124 conn(0x564aeae26ce0 msgr2=0x564aeae21260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 -- 172.21.15.66:0/2994220124 shutdown_connections 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 -- 172.21.15.66:0/2994220124 wait complete. 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.642+0000 7f89f7c33740 1 -- start start 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 1 Processor -- start 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae23970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01ad910 con 0x564aeae204d0 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01aed80 con 0x564aeae1aba0 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01adfe0 con 0x564aeae20e80 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89f7c33740 10 client.0 _ll_register_callbacks cb 0x7f89e80109c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae23970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae23970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49410/0 (socket says 172.21.15.66:49410) 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 -- 172.21.15.66:0/2017341491 learned_addr learned my addr 172.21.15.66:0/2017341491 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89ee59c640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 -- 172.21.15.66:0/2017341491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 msgr2=0x564aeae23eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae23eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 -- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 msgr2=0x564aeae23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae23430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 -- 172.21.15.66:0/2017341491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aeae9f1b0 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89eed9d640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae23eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89e5d9b640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae23970 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f89d80bac80 tx=0x7f89d80be240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147633c166a75113 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.643+0000 7f89d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2017341491 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80c2bd0 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2017341491 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f89d80bae90 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e8009b70 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2017341491 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89d80c7490 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 10 client.17271 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564aeaea04b0 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2017341491 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e8009b70 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2017341491 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89d80bdc70 con 0x564aeae204d0 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 10 client.17271 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 20 client.17271 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae29a00 0x564aeaea2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 4 client.17271 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564aead92c10 con 0x564aeae29a00 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.644+0000 7f89ee59c640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae29a00 0x564aeaea2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89d5ffb640 10 client.17271 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89ee59c640 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae29a00 0x564aeaea2cc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dde320115d3024a server_cookie=350aa73148a36284 in_seq=0 out_seq=0 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89d5ffb640 1 -- 172.21.15.66:0/2017341491 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564aead92c10 con 0x564aeae29a00 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89d5ffb640 10 client.17271 handle_command_reply: tid=0 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 client.17271 shutdown 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 msgr2=0x564aeae23970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae23970 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f89d80bac80 tx=0x7f89d80be240 comp rx=0 tx=0).stop 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 shutdown_connections 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x564aeae29a00 0x564aeaea2cc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564aeae20e80 0x564aeae23eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564aeae204d0 0x564aeae23970 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89d5ffb640 0 client.17271 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 --2- 172.21.15.66:0/2017341491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aeae1aba0 0x564aeae23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 >> 172.21.15.66:0/2017341491 conn(0x564aeae26ce0 msgr2=0x564aeae21af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 shutdown_connections 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 1 -- 172.21.15.66:0/2017341491 wait complete. 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.645+0000 7f89f7c33740 20 client.17271 trim_cache size 0 max 16384 2024-04-21T09:14:12.653 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:14:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f89dc0a3d20 msgr2=0x7f89dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f89dc0a3d20 0x7f89dc0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009fd0 tx=0x7f89e8014b70 comp rx=0 tx=0).stop 2024-04-21T09:14:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 msgr2=0x7f89f0198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f0198cc0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f89d8005940 tx=0x7f89d80045e0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 shutdown_connections 2024-04-21T09:14:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f89dc0a3d20 0x7f89dc0a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f010c510 0x7f89f019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f0103af0 0x7f89f0198cc0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 --2- 172.21.15.66:0/3479506314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01031d0 0x7f89f0198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 >> 172.21.15.66:0/3479506314 conn(0x7f89f0077f50 msgr2=0x7f89f00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 shutdown_connections 2024-04-21T09:14:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:12.646+0000 7f89f4f6d640 1 -- 172.21.15.66:0/3479506314 wait complete. 2024-04-21T09:14:13.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:13 smithi186 ceph-mon[31505]: pgmap v183: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:14:13.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:13 smithi083 ceph-mon[31413]: pgmap v183: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:14:13.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:13 smithi066 ceph-mon[27953]: pgmap v183: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T09:14:13.657 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:14:13.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:14:13.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:14:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 -- 172.21.15.66:0/4199971793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21006b1f0 msgr2=0x7fc210106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 --2- 172.21.15.66:0/4199971793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21006b1f0 0x7fc210106b90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc200009900 tx=0x7fc20002bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 -- 172.21.15.66:0/4199971793 shutdown_connections 2024-04-21T09:14:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 --2- 172.21.15.66:0/4199971793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc210107270 0x7fc21010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 --2- 172.21.15.66:0/4199971793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21006b1f0 0x7fc210106b90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 --2- 172.21.15.66:0/4199971793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc21006a840 0x7fc21006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 -- 172.21.15.66:0/4199971793 >> 172.21.15.66:0/4199971793 conn(0x7fc210077eb0 msgr2=0x7fc2100782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 -- 172.21.15.66:0/4199971793 shutdown_connections 2024-04-21T09:14:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.776+0000 7fc214fce640 1 -- 172.21.15.66:0/4199971793 wait complete. 2024-04-21T09:14:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 Processor -- start 2024-04-21T09:14:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 -- start start 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc21006a840 0x7fc210198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc210107270 0x7fc210198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21019cff0 0x7fc21019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101121b0 con 0x7fc21019cff0 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc210112030 con 0x7fc21006a840 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc214fce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc210112330 con 0x7fc210107270 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21019cff0 0x7fc21019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21019cff0 0x7fc21019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49424/0 (socket says 172.21.15.66:49424) 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc20ed9d640 1 -- 172.21.15.66:0/1653844253 learned_addr learned my addr 172.21.15.66:0/1653844253 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc20e59c640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc21006a840 0x7fc210198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.777+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc210107270 0x7fc210198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc20ed9d640 1 -- 172.21.15.66:0/1653844253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc210107270 msgr2=0x7fc210198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc20ed9d640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc210107270 0x7fc210198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc20ed9d640 1 -- 172.21.15.66:0/1653844253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc21006a840 msgr2=0x7fc210198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc20ed9d640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc21006a840 0x7fc210198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc20ed9d640 1 -- 172.21.15.66:0/1653844253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21019dae0 con 0x7fc21019cff0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc210107270 0x7fc210198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc20ed9d640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21019cff0 0x7fc21019d4a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc20800eb10 tx=0x7fc20800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc2077fe640 1 -- 172.21.15.66:0/1653844253 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208004b20 con 0x7fc21019cff0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc2077fe640 1 -- 172.21.15.66:0/1653844253 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc208004cc0 con 0x7fc21019cff0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21019ddd0 con 0x7fc21019cff0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc2077fe640 1 -- 172.21.15.66:0/1653844253 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208010670 con 0x7fc21019cff0 2024-04-21T09:14:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.778+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2101a5570 con 0x7fc21019cff0 2024-04-21T09:14:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.781+0000 7fc2077fe640 1 -- 172.21.15.66:0/1653844253 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc208005020 con 0x7fc21019cff0 2024-04-21T09:14:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.781+0000 7fc217bbe740 1 Processor -- start 2024-04-21T09:14:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.781+0000 7fc217bbe740 1 -- start start 2024-04-21T09:14:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.781+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc217bbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55db37b7ff00 con 0x55db37c206a0 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc217bbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db37b1d590 con 0x55db37c1fcf0 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc217bbe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55db37b7bec0 con 0x55db37c29220 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc2077fe640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc1fc0a3d20 0x7fc1fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c28c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49426/0 (socket says 172.21.15.66:49426) 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 -- 172.21.15.66:0/2008825116 learned_addr learned my addr 172.21.15.66:0/2008825116 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20ed9d640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc1fc0a3d20 0x7fc1fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 -- 172.21.15.66:0/2008825116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 msgr2=0x55db37c2b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c2b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 -- 172.21.15.66:0/2008825116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 msgr2=0x55db37c200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 -- 172.21.15.66:0/2008825116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db3795a7e0 con 0x55db37c206a0 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc2077fe640 1 -- 172.21.15.66:0/1653844253 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc2080d1120 con 0x7fc21019cff0 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.782+0000 7fc20e59c640 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c28c50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800b760 tx=0x7fc1f800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef459f4b4aa775e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc204ff9640 1 -- 172.21.15.66:0/2008825116 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8003dd0 con 0x55db37c206a0 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc20ed9d640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc1fc0a3d20 0x7fc1fc0a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc20809ca50 tx=0x7fc2080bd040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc204ff9640 1 -- 172.21.15.66:0/2008825116 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8015070 con 0x55db37c206a0 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc204ff9640 1 -- 172.21.15.66:0/2008825116 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8004d90 con 0x55db37c206a0 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 -- 172.21.15.66:0/2008825116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 msgr2=0x55db37c28c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c28c50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800b760 tx=0x7fc1f800bc20 comp rx=0 tx=0).stop 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 -- 172.21.15.66:0/2008825116 shutdown_connections 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c2b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c28c50 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 --2- 172.21.15.66:0/2008825116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 -- 172.21.15.66:0/2008825116 >> 172.21.15.66:0/2008825116 conn(0x55db37c1a3c0 msgr2=0x55db37c1a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.783+0000 7fc217bbe740 1 -- 172.21.15.66:0/2008825116 shutdown_connections 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 -- 172.21.15.66:0/2008825116 wait complete. 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 -- start start 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 Processor -- start 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c26220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c26ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55db37c2c190 con 0x55db37c206a0 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db37b39580 con 0x55db37c1fcf0 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55db37c2c310 con 0x55db37c29220 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc217bbe740 10 client.0 _ll_register_callbacks cb 0x55db37b3f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c26760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49434/0 (socket says 172.21.15.66:49434) 2024-04-21T09:14:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc20e59c640 1 -- 172.21.15.66:0/923805561 learned_addr learned my addr 172.21.15.66:0/923805561 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.784+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c26220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c26ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20e59c640 1 -- 172.21.15.66:0/923805561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 msgr2=0x55db37c26ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20e59c640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c26ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20e59c640 1 -- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 msgr2=0x55db37c26220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20e59c640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c26220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20e59c640 1 -- 172.21.15.66:0/923805561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db379baf70 con 0x55db37c206a0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c26ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc20e59c640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c26760 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8018000 tx=0x7fc1f8003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4597c5d8c8211a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc1f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/923805561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8028060 con 0x55db37c206a0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f8015070 con 0x55db37c206a0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/923805561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc1f8011850 con 0x55db37c206a0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/923805561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc1f802f050 con 0x55db37c206a0 2024-04-21T09:14:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc217bbe740 10 client.17283 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db37852170 con 0x55db37c206a0 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.785+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/923805561 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8015070 con 0x55db37c206a0 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/923805561 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc1f8022070 con 0x55db37c206a0 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc217bbe740 10 client.17283 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc217bbe740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc217bbe740 20 client.17283 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc217bbe740 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37c39670 0x55db37c3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc217bbe740 4 client.17283 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db37b8d920 con 0x55db37c39670 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37c39670 0x55db37c3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37c39670 0x55db37c3ba60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303caf0c7f1eb3e4 server_cookie=3b8b420e790f4411 in_seq=0 out_seq=0 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.786+0000 7fc1f5ffb640 10 client.17283 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/923805561 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db37b8d920 con 0x55db37c39670 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc1f5ffb640 10 client.17283 handle_command_reply: tid=0 2024-04-21T09:14:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 client.17283 shutdown 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 msgr2=0x55db37c26760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c26760 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8018000 tx=0x7fc1f8003e00 comp rx=0 tx=0).stop 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 shutdown_connections 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37c39670 0x55db37c3ba60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c26ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c26760 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc1f5ffb640 0 client.17283 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 --2- 172.21.15.66:0/923805561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c26220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 >> 172.21.15.66:0/923805561 conn(0x55db37c1a3c0 msgr2=0x55db37c1c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 shutdown_connections 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.787+0000 7fc217bbe740 1 -- 172.21.15.66:0/923805561 wait complete. 2024-04-21T09:14:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.788+0000 7fc217bbe740 20 client.17283 trim_cache size 0 max 16384 2024-04-21T09:14:13.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 Processor -- start 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 -- start start 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c28950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c14e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101afa10 con 0x55db37c206a0 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101ad2e0 con 0x55db37c1fcf0 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc217bbe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101ab110 con 0x55db37c29220 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c35050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49448/0 (socket says 172.21.15.66:49448) 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 -- 172.21.15.66:0/1738094023 learned_addr learned my addr 172.21.15.66:0/1738094023 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c28950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 -- 172.21.15.66:0/1738094023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 msgr2=0x55db37c14e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c14e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 -- 172.21.15.66:0/1738094023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 msgr2=0x55db37c28950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c28950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 -- 172.21.15.66:0/1738094023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db37c24770 con 0x55db37c206a0 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c28950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.806+0000 7fc20e59c640 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c35050 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800b3f0 tx=0x7fc1f800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd329b3c3613f650 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc1f7fff640 1 -- 172.21.15.66:0/1738094023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8028070 con 0x55db37c206a0 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc1f7fff640 1 -- 172.21.15.66:0/1738094023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8015040 con 0x55db37c206a0 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc1f7fff640 1 -- 172.21.15.66:0/1738094023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8011650 con 0x55db37c206a0 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 -- 172.21.15.66:0/1738094023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 msgr2=0x55db37c35050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c35050 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800b3f0 tx=0x7fc1f800b5b0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 -- 172.21.15.66:0/1738094023 shutdown_connections 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c29220 0x55db37c14e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c206a0 0x55db37c35050 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 --2- 172.21.15.66:0/1738094023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c1fcf0 0x55db37c28950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 -- 172.21.15.66:0/1738094023 >> 172.21.15.66:0/1738094023 conn(0x55db37c1a3c0 msgr2=0x55db37c14780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 -- 172.21.15.66:0/1738094023 shutdown_connections 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 -- 172.21.15.66:0/1738094023 wait complete. 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 -- start start 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.807+0000 7fc217bbe740 1 Processor -- start 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c1fcf0 0x55db37c22de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c206a0 0x55db37c23320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc217bbe740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c29220 0x55db37c23860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc217bbe740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101ab7a0 con 0x55db37c29220 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc217bbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101acc10 con 0x55db37c206a0 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc217bbe740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101abe70 con 0x55db37c1fcf0 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc217bbe740 10 client.0 _ll_register_callbacks cb 0x7fc1f8004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c29220 0x55db37c23860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c29220 0x55db37c23860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49458/0 (socket says 172.21.15.66:49458) 2024-04-21T09:14:13.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 -- 172.21.15.66:0/4146315985 learned_addr learned my addr 172.21.15.66:0/4146315985 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c1fcf0 0x55db37c22de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20e59c640 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c206a0 0x55db37c23320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 -- 172.21.15.66:0/4146315985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c1fcf0 msgr2=0x55db37c22de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c1fcf0 0x55db37c22de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 -- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c206a0 msgr2=0x55db37c23320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c206a0 0x55db37c23320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 -- 172.21.15.66:0/4146315985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db37c9e870 con 0x55db37c29220 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc20dd9b640 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c29220 0x55db37c23860 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc200006010 tx=0x7fc200009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f20b2f3bb07feb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc1f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.808+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/4146315985 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20002c9f0 con 0x55db37c29220 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/4146315985 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fc20003a040 con 0x55db37c29220 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f8015040 con 0x55db37c29220 2024-04-21T09:14:13.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/4146315985 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc20002cb90 con 0x55db37c29220 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 10 client.17289 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db37c9fa20 con 0x55db37c29220 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/4146315985 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8015040 con 0x55db37c29220 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/4146315985 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc20004f070 con 0x55db37c29220 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 10 client.17289 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 20 client.17289 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37ca0150 0x55db37ca2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 4 client.17289 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55db37b87790 con 0x55db37ca0150 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.809+0000 7fc20e59c640 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37ca0150 0x55db37ca2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.810+0000 7fc20e59c640 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37ca0150 0x55db37ca2570 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b8adf9b26aa83b server_cookie=7f7fcc2c406d9564 in_seq=0 out_seq=0 2024-04-21T09:14:13.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.810+0000 7fc1f5ffb640 10 client.17289 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc1f5ffb640 1 -- 172.21.15.66:0/4146315985 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55db37b87790 con 0x55db37ca0150 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc1f5ffb640 10 client.17289 handle_command_reply: tid=0 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 client.17289 shutdown 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c29220 msgr2=0x55db37c23860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c29220 0x55db37c23860 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc200006010 tx=0x7fc200009260 comp rx=0 tx=0).stop 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 shutdown_connections 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55db37ca0150 0x55db37ca2570 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db37c29220 0x55db37c23860 secure :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc200006010 tx=0x7fc200009260 comp rx=0 tx=0).stop 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc1f5ffb640 0 client.17289 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db37c206a0 0x55db37c23320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 --2- 172.21.15.66:0/4146315985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37c1fcf0 0x55db37c22de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 >> 172.21.15.66:0/4146315985 conn(0x55db37c1a3c0 msgr2=0x55db37c269e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 shutdown_connections 2024-04-21T09:14:13.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 1 -- 172.21.15.66:0/4146315985 wait complete. 2024-04-21T09:14:13.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.811+0000 7fc217bbe740 20 client.17289 trim_cache size 0 max 16384 2024-04-21T09:14:13.818 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:14:13.818 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:14:13.818 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "b91a62b2-03cc-4d3c-978d-8b95fa669a32", 2024-04-21T09:14:13.818 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:14:13.818 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc1fc0a3d20 msgr2=0x7fc1fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc1fc0a3d20 0x7fc1fc0a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc20809ca50 tx=0x7fc2080bd040 comp rx=0 tx=0).stop 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21019cff0 msgr2=0x7fc21019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21019cff0 0x7fc21019d4a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc20800eb10 tx=0x7fc20800c2d0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 shutdown_connections 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc1fc0a3d20 0x7fc1fc0a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc21019cff0 0x7fc21019d4a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc210107270 0x7fc210198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 --2- 172.21.15.66:0/1653844253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc21006a840 0x7fc210198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 >> 172.21.15.66:0/1653844253 conn(0x7fc210077eb0 msgr2=0x7fc2100ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 shutdown_connections 2024-04-21T09:14:13.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.812+0000 7fc214fce640 1 -- 172.21.15.66:0/1653844253 wait complete. 2024-04-21T09:14:13.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:14:13.823 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:14:13.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 -- 172.21.15.66:0/668355397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8101a30 msgr2=0x7f34e8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 --2- 172.21.15.66:0/668355397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8101a30 0x7f34e8105a70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009900 tx=0x7f34d002bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:13.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 -- 172.21.15.66:0/668355397 shutdown_connections 2024-04-21T09:14:13.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 --2- 172.21.15.66:0/668355397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e8106150 0x7f34e810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 --2- 172.21.15.66:0/668355397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8101a30 0x7f34e8105a70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 --2- 172.21.15.66:0/668355397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e8101080 0x7f34e8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 -- 172.21.15.66:0/668355397 >> 172.21.15.66:0/668355397 conn(0x7f34e8078000 msgr2=0x7f34e8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.926+0000 7f34ede81640 1 -- 172.21.15.66:0/668355397 shutdown_connections 2024-04-21T09:14:13.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 -- 172.21.15.66:0/668355397 wait complete. 2024-04-21T09:14:13.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 Processor -- start 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 -- start start 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e8101080 0x7f34e8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8106150 0x7f34e8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e819cfd0 0x7f34e819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81121d0 con 0x7f34e8106150 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8112050 con 0x7f34e819cfd0 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.927+0000 7f34ede81640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8112350 con 0x7f34e8101080 2024-04-21T09:14:13.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8106150 0x7f34e8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8106150 0x7f34e8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49476/0 (socket says 172.21.15.66:49476) 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 -- 172.21.15.66:0/2819533801 learned_addr learned my addr 172.21.15.66:0/2819533801 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e8101080 0x7f34e8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e7fff640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e819cfd0 0x7f34e819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 -- 172.21.15.66:0/2819533801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e8101080 msgr2=0x7f34e8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e8101080 0x7f34e8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 -- 172.21.15.66:0/2819533801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e819cfd0 msgr2=0x7f34e819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e819cfd0 0x7f34e819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 -- 172.21.15.66:0/2819533801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e819dac0 con 0x7f34e8106150 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8106150 0x7f34e8198c50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f34d0036040 tx=0x7f34d002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e4ff9640 1 -- 172.21.15.66:0/2819533801 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0043020 con 0x7f34e8106150 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e4ff9640 1 -- 172.21.15.66:0/2819533801 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d002cbe0 con 0x7f34e8106150 2024-04-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e819dd50 con 0x7f34e8106150 2024-04-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34e4ff9640 1 -- 172.21.15.66:0/2819533801 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0003bf0 con 0x7f34e8106150 2024-04-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.928+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e81a5570 con 0x7f34e8106150 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.930+0000 7f34e4ff9640 1 -- 172.21.15.66:0/2819533801 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f34d0046020 con 0x7f34e8106150 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 Processor -- start 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 -- start start 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f6751a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67523540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67525e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555f6747f1e0 con 0x555f6751a580 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f6741c390 con 0x555f6751aea0 2024-04-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34f0a74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f6747b1a0 con 0x555f67523a80 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f6751a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e4ff9640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34d40a3d20 0x7f34d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67523540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67523540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:50058/0 (socket says 172.21.15.66:50058) 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 -- 172.21.15.66:0/2677502462 learned_addr learned my addr 172.21.15.66:0/2677502462 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f6751a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49492/0 (socket says 172.21.15.66:49492) 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e7fff640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34d40a3d20 0x7f34d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e4ff9640 1 -- 172.21.15.66:0/2819533801 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f34d00f2bf0 con 0x7f34e8106150 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67525e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 -- 172.21.15.66:0/2677502462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 msgr2=0x555f67525e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67525e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 -- 172.21.15.66:0/2677502462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 msgr2=0x555f6751a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f6751a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.931+0000 7f34e77fe640 1 -- 172.21.15.66:0/2677502462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6725b7d0 con 0x555f6751aea0 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.932+0000 7f34e7fff640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34d40a3d20 0x7f34d40a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34e000be60 tx=0x7f34e00041b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.932+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67523540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f34d80103f0 tx=0x7f34d80107f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b9de2206690478 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.932+0000 7f34ddffb640 1 -- 172.21.15.66:0/2677502462 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8004d20 con 0x555f6751aea0 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.932+0000 7f34ddffb640 1 -- 172.21.15.66:0/2677502462 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d80129c0 con 0x555f6751aea0 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.932+0000 7f34ddffb640 1 -- 172.21.15.66:0/2677502462 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80054f0 con 0x555f6751aea0 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 -- 172.21.15.66:0/2677502462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 msgr2=0x555f67523540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67523540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f34d80103f0 tx=0x7f34d80107f0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 -- 172.21.15.66:0/2677502462 shutdown_connections 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67525e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67523540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2677502462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f6751a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 -- 172.21.15.66:0/2677502462 >> 172.21.15.66:0/2677502462 conn(0x555f67514cb0 msgr2=0x555f67515090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 -- 172.21.15.66:0/2677502462 shutdown_connections 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 -- 172.21.15.66:0/2677502462 wait complete. 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 -- start start 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.933+0000 7f34f0a74740 1 Processor -- start 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f67520a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67520fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67521500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555f67526870 con 0x555f6751a580 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f674382f0 con 0x555f6751aea0 2024-04-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f675269f0 con 0x555f67523a80 2024-04-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f67520a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 10 client.0 _ll_register_callbacks cb 0x555f67445580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f67520a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49504/0 (socket says 172.21.15.66:49504) 2024-04-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 -- 172.21.15.66:0/2455486236 learned_addr learned my addr 172.21.15.66:0/2455486236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67520fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e7fff640 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67521500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 -- 172.21.15.66:0/2455486236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 msgr2=0x555f67521500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67521500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 -- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 msgr2=0x555f67520fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67520fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 -- 172.21.15.66:0/2455486236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f671913f0 con 0x555f6751a580 2024-04-21T09:14:13.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f67520a80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f34d8006010 tx=0x7f34d8012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2abbe820862295 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:13.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34c6ffd640 1 -- 172.21.15.66:0/2455486236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d802a060 con 0x555f6751a580 2024-04-21T09:14:13.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34c6ffd640 1 -- 172.21.15.66:0/2455486236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f34d8003b70 con 0x555f6751a580 2024-04-21T09:14:13.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.934+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d80129c0 con 0x555f6751a580 2024-04-21T09:14:13.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34c6ffd640 1 -- 172.21.15.66:0/2455486236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34d8031050 con 0x555f6751a580 2024-04-21T09:14:13.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 10 client.17301 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:13.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f671510f0 con 0x555f6751a580 2024-04-21T09:14:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34c6ffd640 1 -- 172.21.15.66:0/2455486236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d80129c0 con 0x555f6751a580 2024-04-21T09:14:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34c6ffd640 1 -- 172.21.15.66:0/2455486236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f34d800f040 con 0x555f6751a580 2024-04-21T09:14:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 10 client.17301 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 20 client.17301 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f675339d0 0x555f67535dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 4 client.17301 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f6748cc00 con 0x555f675339d0 2024-04-21T09:14:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.935+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f675339d0 0x555f67535dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f675339d0 0x555f67535dc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97448aec4ce99492 server_cookie=e67375d5b2b00ebc in_seq=0 out_seq=0 2024-04-21T09:14:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34c6ffd640 10 client.17301 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34c6ffd640 1 -- 172.21.15.66:0/2455486236 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f6748cc00 con 0x555f675339d0 2024-04-21T09:14:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34c6ffd640 10 client.17301 handle_command_reply: tid=0 2024-04-21T09:14:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 client.17301 shutdown 2024-04-21T09:14:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 msgr2=0x555f67520a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f67520a80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f34d8006010 tx=0x7f34d8012e20 comp rx=0 tx=0).stop 2024-04-21T09:14:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 shutdown_connections 2024-04-21T09:14:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f675339d0 0x555f67535dc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f67523a80 0x555f67521500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67520fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2455486236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f6751a580 0x555f67520a80 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34c6ffd640 0 client.17301 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.936+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 >> 172.21.15.66:0/2455486236 conn(0x555f67514cb0 msgr2=0x555f67516a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.937+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 shutdown_connections 2024-04-21T09:14:13.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.937+0000 7f34f0a74740 1 -- 172.21.15.66:0/2455486236 wait complete. 2024-04-21T09:14:13.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.937+0000 7f34f0a74740 20 client.17301 trim_cache size 0 max 16384 2024-04-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 Processor -- start 2024-04-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 -- start start 2024-04-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f6752f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67515090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81b1b00 con 0x555f67523a80 2024-04-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81af3d0 con 0x555f6751aea0 2024-04-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34f0a74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81ad200 con 0x555f6751a580 2024-04-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67515090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67515090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49508/0 (socket says 172.21.15.66:49508) 2024-04-21T09:14:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34e7fff640 1 -- 172.21.15.66:0/2548184455 learned_addr learned my addr 172.21.15.66:0/2548184455 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.954+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f6752f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e7fff640 1 -- 172.21.15.66:0/2548184455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 msgr2=0x555f675231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e7fff640 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e7fff640 1 -- 172.21.15.66:0/2548184455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 msgr2=0x555f6752f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e7fff640 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f6752f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e7fff640 1 -- 172.21.15.66:0/2548184455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f67536320 con 0x555f67523a80 2024-04-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f6752f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e77fe640 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34e7fff640 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67515090 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005d00 tx=0x7f34e000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ad488d7838b393 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34dd7fa640 1 -- 172.21.15.66:0/2548184455 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0025070 con 0x555f67523a80 2024-04-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34dd7fa640 1 -- 172.21.15.66:0/2548184455 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e0012dc0 con 0x555f67523a80 2024-04-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.955+0000 7f34dd7fa640 1 -- 172.21.15.66:0/2548184455 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00216b0 con 0x555f67523a80 2024-04-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- 172.21.15.66:0/2548184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 msgr2=0x555f67515090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67515090 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005d00 tx=0x7f34e000bf90 comp rx=0 tx=0).stop 2024-04-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- 172.21.15.66:0/2548184455 shutdown_connections 2024-04-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67515090 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f6752f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 --2- 172.21.15.66:0/2548184455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- 172.21.15.66:0/2548184455 >> 172.21.15.66:0/2548184455 conn(0x555f67514cb0 msgr2=0x555f675134e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- 172.21.15.66:0/2548184455 shutdown_connections 2024-04-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- 172.21.15.66:0/2548184455 wait complete. 2024-04-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- start start 2024-04-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 Processor -- start 2024-04-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67527c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67528170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81ad890 con 0x555f67523a80 2024-04-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81aed00 con 0x555f6751aea0 2024-04-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.956+0000 7f34f0a74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81adf60 con 0x555f6751a580 2024-04-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34f0a74740 10 client.0 _ll_register_callbacks cb 0x7f34d00035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67528170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67528170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49518/0 (socket says 172.21.15.66:49518) 2024-04-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 -- 172.21.15.66:0/4006848396 learned_addr learned my addr 172.21.15.66:0/4006848396 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e77fe640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 -- 172.21.15.66:0/4006848396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 msgr2=0x555f675276f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67527c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675276f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 -- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 msgr2=0x555f67527c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67527c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 -- 172.21.15.66:0/4006848396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f67598eb0 con 0x555f67523a80 2024-04-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e6ffd640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67527c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34e7fff640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67528170 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f34e00042b0 tx=0x7f34e0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a9bd0254cc54da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34c6ffd640 1 -- 172.21.15.66:0/4006848396 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0025040 con 0x555f67523a80 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34c6ffd640 1 -- 172.21.15.66:0/4006848396 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f34e002a660 con 0x555f67523a80 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e0012dc0 con 0x555f67523a80 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34c6ffd640 1 -- 172.21.15.66:0/4006848396 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34e003b050 con 0x555f67523a80 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34f0a74740 10 client.17307 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.957+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f6759a0f0 con 0x555f67523a80 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34c6ffd640 1 -- 172.21.15.66:0/4006848396 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e0012dc0 con 0x555f67523a80 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34c6ffd640 1 -- 172.21.15.66:0/4006848396 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f34e001e040 con 0x555f67523a80 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34f0a74740 10 client.17307 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34f0a74740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34f0a74740 20 client.17307 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34f0a74740 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f6759a7b0 0x555f6759cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34f0a74740 4 client.17307 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f67486a70 con 0x555f6759a7b0 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.958+0000 7f34e77fe640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f6759a7b0 0x555f6759cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34c6ffd640 10 client.17307 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34e77fe640 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f6759a7b0 0x555f6759cbd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64de812d846d4f5d server_cookie=f8bc5345451073de in_seq=0 out_seq=0 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34c6ffd640 1 -- 172.21.15.66:0/4006848396 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555f67486a70 con 0x555f6759a7b0 2024-04-21T09:14:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34c6ffd640 10 client.17307 handle_command_reply: tid=0 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 client.17307 shutdown 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 msgr2=0x555f67528170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67528170 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f34e00042b0 tx=0x7f34e0005270 comp rx=0 tx=0).stop 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 shutdown_connections 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555f6759a7b0 0x555f6759cbd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f67523a80 0x555f67528170 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34c6ffd640 0 client.17307 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f6751aea0 0x555f67527c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 --2- 172.21.15.66:0/4006848396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f6751a580 0x555f675276f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 >> 172.21.15.66:0/4006848396 conn(0x555f67514cb0 msgr2=0x555f675168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 shutdown_connections 2024-04-21T09:14:13.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 1 -- 172.21.15.66:0/4006848396 wait complete. 2024-04-21T09:14:13.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.959+0000 7f34f0a74740 20 client.17307 trim_cache size 0 max 16384 2024-04-21T09:14:13.980 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:14:13.980 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:14:13.980 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:14:13.981 INFO:teuthology.orchestra.run.smithi066.stdout: "b91a62b2-03cc-4d3c-978d-8b95fa669a32": { 2024-04-21T09:14:13.981 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:14:13.981 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "b91a62b2-03cc-4d3c-978d-8b95fa669a32", 2024-04-21T09:14:13.981 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:14:13.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:14:13.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:14:13.981 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:14:13.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34d40a3d20 msgr2=0x7f34d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34d40a3d20 0x7f34d40a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34e000be60 tx=0x7f34e00041b0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8106150 msgr2=0x7f34e8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:13.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8106150 0x7f34e8198c50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f34d0036040 tx=0x7f34d002c7e0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 shutdown_connections 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34d40a3d20 0x7f34d40a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e819cfd0 0x7f34e819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34e8106150 0x7f34e8198c50 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 --2- 172.21.15.66:0/2819533801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e8101080 0x7f34e8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 >> 172.21.15.66:0/2819533801 conn(0x7f34e8078000 msgr2=0x7f34e80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 shutdown_connections 2024-04-21T09:14:13.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:13.960+0000 7f34ede81640 1 -- 172.21.15.66:0/2819533801 wait complete. 2024-04-21T09:14:13.985 INFO:tasks.cephfs.filesystem:scrub status for tag:b91a62b2-03cc-4d3c-978d-8b95fa669a32 - {'path': '/', 'tag': 'b91a62b2-03cc-4d3c-978d-8b95fa669a32', 'options': 'recursive,force'} 2024-04-21T09:14:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:14 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:14:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:14 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:14 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:14:14.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:14 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:14.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:14 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:14:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:14 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:14 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:14:14.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:14 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:14.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:14 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:14:14.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:14 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:14.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:14 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:14:14.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:14 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:15 smithi186 ceph-mon[31505]: pgmap v184: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:15.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:15 smithi083 ceph-mon[31413]: pgmap v184: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:15.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:15 smithi066 ceph-mon[27953]: pgmap v184: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T09:14:16.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:16 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:14:16.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:16 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:14:16.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:16 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:14:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:17 smithi186 ceph-mon[31505]: pgmap v185: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:14:17.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:17 smithi083 ceph-mon[31413]: pgmap v185: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:14:17.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:17 smithi066 ceph-mon[27953]: pgmap v185: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:14:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:19 smithi186 ceph-mon[31505]: pgmap v186: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:14:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:19 smithi083 ceph-mon[31413]: pgmap v186: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:14:19.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:19 smithi066 ceph-mon[27953]: pgmap v186: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:14:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:21 smithi186 ceph-mon[31505]: pgmap v187: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:21.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:21 smithi083 ceph-mon[31413]: pgmap v187: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:21.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:21 smithi066 ceph-mon[27953]: pgmap v187: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:22.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:22 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:14:22.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:22 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:14:22.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:22 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:14:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:23 smithi186 ceph-mon[31505]: pgmap v188: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:23 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:14:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:23 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:14:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:23 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:14:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:23 smithi083 ceph-mon[31413]: pgmap v188: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:23.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:23 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:14:23.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:23 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:14:23.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:23 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:14:23.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:23 smithi066 ceph-mon[27953]: pgmap v188: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:23 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:14:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:23 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:14:23.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:23 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:14:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:25 smithi186 ceph-mon[31505]: pgmap v189: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:25 smithi083 ceph-mon[31413]: pgmap v189: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:25.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:25 smithi066 ceph-mon[27953]: pgmap v189: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:27.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:27 smithi186 ceph-mon[31505]: pgmap v190: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:27.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:27 smithi083 ceph-mon[31413]: pgmap v190: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:27.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:27 smithi066 ceph-mon[27953]: pgmap v190: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:29 smithi186 ceph-mon[31505]: pgmap v191: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:29 smithi083 ceph-mon[31413]: pgmap v191: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:29.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:29 smithi066 ceph-mon[27953]: pgmap v191: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:31 smithi186 ceph-mon[31505]: pgmap v192: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:31.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:31 smithi083 ceph-mon[31413]: pgmap v192: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:31 smithi066 ceph-mon[27953]: pgmap v192: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:33.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:33 smithi186 ceph-mon[31505]: pgmap v193: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:33.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:33 smithi083 ceph-mon[31413]: pgmap v193: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:33 smithi066 ceph-mon[27953]: pgmap v193: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:35 smithi186 ceph-mon[31505]: pgmap v194: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:35 smithi083 ceph-mon[31413]: pgmap v194: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:35.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:35 smithi066 ceph-mon[27953]: pgmap v194: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:37 smithi186 ceph-mon[31505]: pgmap v195: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:37.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:37 smithi083 ceph-mon[31413]: pgmap v195: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:37.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:37 smithi066 ceph-mon[27953]: pgmap v195: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:39 smithi186 ceph-mon[31505]: pgmap v196: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:39 smithi083 ceph-mon[31413]: pgmap v196: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:39.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:39 smithi066 ceph-mon[27953]: pgmap v196: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:41 smithi186 ceph-mon[31505]: pgmap v197: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:41.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:41 smithi083 ceph-mon[31413]: pgmap v197: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:41.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:41 smithi066 ceph-mon[27953]: pgmap v197: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:43.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:43 smithi186 ceph-mon[31505]: pgmap v198: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:43.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:43 smithi083 ceph-mon[31413]: pgmap v198: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:43.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:43 smithi066 ceph-mon[27953]: pgmap v198: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:43.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:14:43.986 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:14:44.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 -- 172.21.15.66:0/1967371237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f810c580 msgr2=0x7fe3f810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1967371237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f810c580 0x7fe3f810e9b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009ca0 tx=0x7fe3f002c180 comp rx=0 tx=0).stop 2024-04-21T09:14:44.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 -- 172.21.15.66:0/1967371237 shutdown_connections 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1967371237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f810c580 0x7fe3f810e9b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1967371237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f8103ad0 0x7fe3f810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1967371237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f81031b0 0x7fe3f8103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 -- 172.21.15.66:0/1967371237 >> 172.21.15.66:0/1967371237 conn(0x7fe3f80771d0 msgr2=0x7fe3f80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 -- 172.21.15.66:0/1967371237 shutdown_connections 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.103+0000 7fe3fe893640 1 -- 172.21.15.66:0/1967371237 wait complete. 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 Processor -- start 2024-04-21T09:14:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 -- start start 2024-04-21T09:14:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f81031b0 0x7fe3f81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f8103ad0 0x7fe3f8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f810c580 0x7fe3f819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81121a0 con 0x7fe3f81031b0 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8112020 con 0x7fe3f8103ad0 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3fe893640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8112320 con 0x7fe3f810c580 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3f7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f81031b0 0x7fe3f81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3f7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f81031b0 0x7fe3f81986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44090/0 (socket says 172.21.15.66:44090) 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.104+0000 7fe3f7fff640 1 -- 172.21.15.66:0/1787031892 learned_addr learned my addr 172.21.15.66:0/1787031892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f8103ad0 0x7fe3f8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f810c580 0x7fe3f819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f7fff640 1 -- 172.21.15.66:0/1787031892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f810c580 msgr2=0x7fe3f819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f810c580 0x7fe3f819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f7fff640 1 -- 172.21.15.66:0/1787031892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f8103ad0 msgr2=0x7fe3f8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f8103ad0 0x7fe3f8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f7fff640 1 -- 172.21.15.66:0/1787031892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f819d690 con 0x7fe3f81031b0 2024-04-21T09:14:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f810c580 0x7fe3f819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f8103ad0 0x7fe3f8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f81031b0 0x7fe3f81986a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d950 tx=0x7fe3e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f57fa640 1 -- 172.21.15.66:0/1787031892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8017070 con 0x7fe3f81031b0 2024-04-21T09:14:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f57fa640 1 -- 172.21.15.66:0/1787031892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e8009c30 con 0x7fe3f81031b0 2024-04-21T09:14:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f819d980 con 0x7fe3f81031b0 2024-04-21T09:14:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3f57fa640 1 -- 172.21.15.66:0/1787031892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800f620 con 0x7fe3f81031b0 2024-04-21T09:14:44.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.105+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f81a5210 con 0x7fe3f81031b0 2024-04-21T09:14:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.107+0000 7fe3f57fa640 1 -- 172.21.15.66:0/1787031892 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe3e8010460 con 0x7fe3f81031b0 2024-04-21T09:14:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 Processor -- start 2024-04-21T09:14:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 -- start start 2024-04-21T09:14:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe3f57fa640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3e00a3d20 0x7fe3e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cbc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cc5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 0x55ef00cc7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ef00c211e0 con 0x55ef00cbcea0 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef00bbe390 con 0x55ef00cbc580 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe401565740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ef00c1d1a0 con 0x55ef00cc5a80 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cc5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3e00a3d20 0x7fe3e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cc5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44104/0 (socket says 172.21.15.66:44104) 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe3f77fe640 1 -- 172.21.15.66:0/2403361158 learned_addr learned my addr 172.21.15.66:0/2403361158 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe3f57fa640 1 -- 172.21.15.66:0/1787031892 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe3e80d0880 con 0x7fe3f81031b0 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.108+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cbc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3f77fe640 1 -- 172.21.15.66:0/2403361158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 msgr2=0x55ef00cc7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 0x55ef00cc7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3f77fe640 1 -- 172.21.15.66:0/2403361158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 msgr2=0x55ef00cbc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cbc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3f77fe640 1 -- 172.21.15.66:0/2403361158 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef009fd7d0 con 0x55ef00cbcea0 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cbc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3e00a3d20 0x7fe3e00a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0037040 tx=0x7fe3f0032340 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cc5540 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00cbe0 tx=0x7fe3ec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da5535feb78d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3de7fc640 1 -- 172.21.15.66:0/2403361158 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec005220 con 0x55ef00cbcea0 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3de7fc640 1 -- 172.21.15.66:0/2403361158 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ec004c50 con 0x55ef00cbcea0 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.109+0000 7fe3de7fc640 1 -- 172.21.15.66:0/2403361158 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec010670 con 0x55ef00cbcea0 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 -- 172.21.15.66:0/2403361158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 msgr2=0x55ef00cc5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cc5540 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00cbe0 tx=0x7fe3ec00e720 comp rx=0 tx=0).stop 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 -- 172.21.15.66:0/2403361158 shutdown_connections 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 0x55ef00cc7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cc5540 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 --2- 172.21.15.66:0/2403361158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cbc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 -- 172.21.15.66:0/2403361158 >> 172.21.15.66:0/2403361158 conn(0x55ef00cb6cb0 msgr2=0x55ef00cb7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 -- 172.21.15.66:0/2403361158 shutdown_connections 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 -- 172.21.15.66:0/2403361158 wait complete. 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 -- start start 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.110+0000 7fe401565740 1 Processor -- start 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe401565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cc2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe401565740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbcea0 0x55ef00cc2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe401565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cc34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe401565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ef00cc8870 con 0x55ef00cc5a80 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe401565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef00bda2f0 con 0x55ef00cbc580 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe401565740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ef00cc89f0 con 0x55ef00cbcea0 2024-04-21T09:14:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe401565740 10 client.0 _ll_register_callbacks cb 0x55ef00be7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cc34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cc34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44110/0 (socket says 172.21.15.66:44110) 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 -- 172.21.15.66:0/1257491136 learned_addr learned my addr 172.21.15.66:0/1257491136 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cc2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbcea0 0x55ef00cc2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 -- 172.21.15.66:0/1257491136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbcea0 msgr2=0x55ef00cc2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbcea0 0x55ef00cc2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 -- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 msgr2=0x55ef00cc2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cc2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 -- 172.21.15.66:0/1257491136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef009333f0 con 0x55ef00cc5a80 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cc34e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009ca0 tx=0x7fe3f00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a757e4b85d675d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.111+0000 7fe3d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1257491136 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0051060 con 0x55ef00cc5a80 2024-04-21T09:14:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1257491136 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe3f0040ab0 con 0x55ef00cc5a80 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3ec004c50 con 0x55ef00cc5a80 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1257491136 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe3f0058050 con 0x55ef00cc5a80 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 10 client.17319 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef008f30f0 con 0x55ef00cc5a80 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1257491136 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ec004c50 con 0x55ef00cc5a80 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1257491136 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe3f0034440 con 0x55ef00cc5a80 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 10 client.17319 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 20 client.17319 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00cd5a40 0x55ef00cd7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 4 client.17319 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.112+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef00c2ec00 con 0x55ef00cd5a40 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.113+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00cd5a40 0x55ef00cd7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.113+0000 7fe3d77fe640 10 client.17319 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.113+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00cd5a40 0x55ef00cd7e50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff0d117f69d21f2 server_cookie=6282a2e55320343 in_seq=0 out_seq=0 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1257491136 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef00c2ec00 con 0x55ef00cd5a40 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe3d77fe640 10 client.17319 handle_command_reply: tid=0 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 client.17319 shutdown 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 msgr2=0x55ef00cc34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cc34e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009ca0 tx=0x7fe3f00095c0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 shutdown_connections 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00cd5a40 0x55ef00cd7e50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cc34e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbcea0 0x55ef00cc2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 --2- 172.21.15.66:0/1257491136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00cc2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe3d77fe640 0 client.17319 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 >> 172.21.15.66:0/1257491136 conn(0x55ef00cb6cb0 msgr2=0x55ef00cb8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 shutdown_connections 2024-04-21T09:14:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 1 -- 172.21.15.66:0/1257491136 wait complete. 2024-04-21T09:14:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.114+0000 7fe401565740 20 client.17319 trim_cache size 0 max 16384 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 Processor -- start 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 -- start start 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbc580 0x55ef00cd1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbcea0 0x55ef00cc19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cdd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81b1ba0 con 0x55ef00cc5a80 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81af470 con 0x55ef00cbcea0 2024-04-21T09:14:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe401565740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81ad2a0 con 0x55ef00cbc580 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cdd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cdd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44112/0 (socket says 172.21.15.66:44112) 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 -- 172.21.15.66:0/3046474272 learned_addr learned my addr 172.21.15.66:0/3046474272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbcea0 0x55ef00cc19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 -- 172.21.15.66:0/3046474272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbc580 msgr2=0x55ef00cd1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbc580 0x55ef00cd1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 -- 172.21.15.66:0/3046474272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbcea0 msgr2=0x55ef00cc19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbcea0 0x55ef00cc19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.132+0000 7fe3fce30640 1 -- 172.21.15.66:0/3046474272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef00cb64f0 con 0x55ef00cc5a80 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe3fce30640 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cdd720 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe3f002c6c0 tx=0x7fe3f0032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27533e927715b97c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe3ddffb640 1 -- 172.21.15.66:0/3046474272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0051040 con 0x55ef00cc5a80 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe3ddffb640 1 -- 172.21.15.66:0/3046474272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f0034e50 con 0x55ef00cc5a80 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe3ddffb640 1 -- 172.21.15.66:0/3046474272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0039030 con 0x55ef00cc5a80 2024-04-21T09:14:44.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 -- 172.21.15.66:0/3046474272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 msgr2=0x55ef00cdd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cdd720 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe3f002c6c0 tx=0x7fe3f0032900 comp rx=0 tx=0).stop 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 -- 172.21.15.66:0/3046474272 shutdown_connections 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cc5a80 0x55ef00cdd720 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbcea0 0x55ef00cc19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 --2- 172.21.15.66:0/3046474272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cbc580 0x55ef00cd1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 -- 172.21.15.66:0/3046474272 >> 172.21.15.66:0/3046474272 conn(0x55ef00cb6cb0 msgr2=0x55ef00cc4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 -- 172.21.15.66:0/3046474272 shutdown_connections 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.133+0000 7fe401565740 1 -- 172.21.15.66:0/3046474272 wait complete. 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 -- start start 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 Processor -- start 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00ccdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 0x55ef00ccac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81ad930 con 0x55ef00cbcea0 2024-04-21T09:14:44.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81aeda0 con 0x55ef00cbc580 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81ae000 con 0x55ef00cc5a80 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe401565740 10 client.0 _ll_register_callbacks cb 0x7fe3f0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44118/0 (socket says 172.21.15.66:44118) 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe3f77fe640 1 -- 172.21.15.66:0/1640149528 learned_addr learned my addr 172.21.15.66:0/1640149528 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.134+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00ccdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3fce30640 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 0x55ef00ccac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3f77fe640 1 -- 172.21.15.66:0/1640149528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 msgr2=0x55ef00ccac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 0x55ef00ccac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3f77fe640 1 -- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 msgr2=0x55ef00ccdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00ccdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3f77fe640 1 -- 172.21.15.66:0/1640149528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef00ccb470 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3f77fe640 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cce1b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec005620 tx=0x7fe3ec010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ada6b6788195f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1640149528 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec004020 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1640149528 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fe3ec0041c0 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f0034e50 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1640149528 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe3ec0043c0 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe401565740 10 client.17325 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef00d3ca70 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1640149528 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f0034e50 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.135+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1640149528 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe3ec026460 con 0x55ef00cbcea0 2024-04-21T09:14:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe401565740 10 client.17325 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe401565740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe401565740 20 client.17325 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe401565740 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00d3d0a0 0x55ef00d3f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe401565740 4 client.17325 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef00c28a70 con 0x55ef00d3d0a0 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00d3d0a0 0x55ef00d3f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe3d77fe640 10 client.17325 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe3f7fff640 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00d3d0a0 0x55ef00d3f4c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3081efd8834f28a server_cookie=504774c7456b311d in_seq=0 out_seq=0 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe3d77fe640 1 -- 172.21.15.66:0/1640149528 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef00c28a70 con 0x55ef00d3d0a0 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe3d77fe640 10 client.17325 handle_command_reply: tid=0 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.136+0000 7fe401565740 1 client.17325 shutdown 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 msgr2=0x55ef00cce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cce1b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec005620 tx=0x7fe3ec010ba0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 shutdown_connections 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55ef00d3d0a0 0x55ef00d3f4c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef00cc5a80 0x55ef00ccac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3d77fe640 0 client.17325 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef00cbcea0 0x55ef00cce1b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 --2- 172.21.15.66:0/1640149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef00cbc580 0x55ef00ccdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 >> 172.21.15.66:0/1640149528 conn(0x55ef00cb6cb0 msgr2=0x55ef00cc2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 shutdown_connections 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 1 -- 172.21.15.66:0/1640149528 wait complete. 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe401565740 20 client.17325 trim_cache size 0 max 16384 2024-04-21T09:14:44.144 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:14:44.145 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:14:44.145 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:14:44.145 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3e00a3d20 msgr2=0x7fe3e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3e00a3d20 0x7fe3e00a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0037040 tx=0x7fe3f0032340 comp rx=0 tx=0).stop 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f81031b0 msgr2=0x7fe3f81986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f81031b0 0x7fe3f81986a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d950 tx=0x7fe3e800de10 comp rx=0 tx=0).stop 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 shutdown_connections 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe3e00a3d20 0x7fe3e00a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f810c580 0x7fe3f819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f8103ad0 0x7fe3f8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 --2- 172.21.15.66:0/1787031892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3f81031b0 0x7fe3f81986a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 >> 172.21.15.66:0/1787031892 conn(0x7fe3f80771d0 msgr2=0x7fe3f80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 shutdown_connections 2024-04-21T09:14:44.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.137+0000 7fe3fe893640 1 -- 172.21.15.66:0/1787031892 wait complete. 2024-04-21T09:14:44.149 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:14:44.149 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:14:44.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 -- 172.21.15.66:0/3302737031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba01021a0 msgr2=0x7ffba0102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 --2- 172.21.15.66:0/3302737031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba01021a0 0x7ffba0102580 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009ca0 tx=0x7ffb9802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 -- 172.21.15.66:0/3302737031 shutdown_connections 2024-04-21T09:14:44.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 --2- 172.21.15.66:0/3302737031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba0107270 0x7ffba010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 --2- 172.21.15.66:0/3302737031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba0102b50 0x7ffba0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 --2- 172.21.15.66:0/3302737031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba01021a0 0x7ffba0102580 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 -- 172.21.15.66:0/3302737031 >> 172.21.15.66:0/3302737031 conn(0x7ffba00793a0 msgr2=0x7ffba00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.250+0000 7ffba7032640 1 -- 172.21.15.66:0/3302737031 shutdown_connections 2024-04-21T09:14:44.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 -- 172.21.15.66:0/3302737031 wait complete. 2024-04-21T09:14:44.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 Processor -- start 2024-04-21T09:14:44.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 -- start start 2024-04-21T09:14:44.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba01021a0 0x7ffba0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba0107270 0x7ffba0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba019d010 0x7ffba019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01121d0 con 0x7ffba019d010 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0112050 con 0x7ffba0107270 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba7032640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0112350 con 0x7ffba01021a0 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba55cf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba019d010 0x7ffba019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba55cf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba019d010 0x7ffba019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44148/0 (socket says 172.21.15.66:44148) 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba55cf640 1 -- 172.21.15.66:0/3105258289 learned_addr learned my addr 172.21.15.66:0/3105258289 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.251+0000 7ffba4dce640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba01021a0 0x7ffba0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffb9ffff640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba0107270 0x7ffba0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba55cf640 1 -- 172.21.15.66:0/3105258289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba01021a0 msgr2=0x7ffba0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba55cf640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba01021a0 0x7ffba0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba55cf640 1 -- 172.21.15.66:0/3105258289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba0107270 msgr2=0x7ffba0198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba55cf640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba0107270 0x7ffba0198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba55cf640 1 -- 172.21.15.66:0/3105258289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba019db00 con 0x7ffba019d010 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffb9ffff640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba0107270 0x7ffba0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba55cf640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba019d010 0x7ffba019d4c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800e970 tx=0x7ffb8800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffb9dffb640 1 -- 172.21.15.66:0/3105258289 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800ccc0 con 0x7ffba019d010 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffb9dffb640 1 -- 172.21.15.66:0/3105258289 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8800ce60 con 0x7ffba019d010 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba019ddf0 con 0x7ffba019d010 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffb9dffb640 1 -- 172.21.15.66:0/3105258289 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88010430 con 0x7ffba019d010 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.252+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba01a5590 con 0x7ffba019d010 2024-04-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.253+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba0069230 con 0x7ffba019d010 2024-04-21T09:14:44.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.254+0000 7ffb9dffb640 1 -- 172.21.15.66:0/3105258289 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ffb88004ff0 con 0x7ffba019d010 2024-04-21T09:14:44.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.255+0000 7ffb9dffb640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ffb8c0a3d20 0x7ffb8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.255+0000 7ffba4dce640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ffb8c0a3d20 0x7ffb8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.255+0000 7ffb9dffb640 1 -- 172.21.15.66:0/3105258289 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7ffb880d0fb0 con 0x7ffba019d010 2024-04-21T09:14:44.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.255+0000 7ffba4dce640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ffb8c0a3d20 0x7ffb8c0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffb98004af0 tx=0x7ffb98005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.260+0000 7ffb9dffb640 1 -- 172.21.15.66:0/3105258289 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb8809d2e0 con 0x7ffba019d010 2024-04-21T09:14:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.490+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffba0102b50 con 0x7ffba019d010 2024-04-21T09:14:44.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.491+0000 7ffb9dffb640 1 -- 172.21.15.66:0/3105258289 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7ffb8809cbe0 con 0x7ffba019d010 2024-04-21T09:14:44.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:14:44.492 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:14:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:14:44.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ffb8c0a3d20 msgr2=0x7ffb8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ffb8c0a3d20 0x7ffb8c0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffb98004af0 tx=0x7ffb98005c50 comp rx=0 tx=0).stop 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba019d010 msgr2=0x7ffba019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba019d010 0x7ffba019d4c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800e970 tx=0x7ffb8800ed70 comp rx=0 tx=0).stop 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 shutdown_connections 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ffb8c0a3d20 0x7ffb8c0a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffba019d010 0x7ffba019d4c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba0107270 0x7ffba0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 --2- 172.21.15.66:0/3105258289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba01021a0 0x7ffba0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 >> 172.21.15.66:0/3105258289 conn(0x7ffba00793a0 msgr2=0x7ffba00fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 shutdown_connections 2024-04-21T09:14:44.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.493+0000 7ffba7032640 1 -- 172.21.15.66:0/3105258289 wait complete. 2024-04-21T09:14:44.505 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:14:44.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.607+0000 7fecb6a8c640 1 -- 172.21.15.66:0/529835160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb010e900 msgr2=0x7fecb0110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.607+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/529835160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb010e900 0x7fecb0110cf0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7feca0009900 tx=0x7feca002bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- 172.21.15.66:0/529835160 shutdown_connections 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/529835160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb010e900 0x7fecb0110cf0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/529835160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb0101670 0x7fecb010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/529835160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0100d50 0x7fecb0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- 172.21.15.66:0/529835160 >> 172.21.15.66:0/529835160 conn(0x7fecb0077f50 msgr2=0x7fecb0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- 172.21.15.66:0/529835160 shutdown_connections 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- 172.21.15.66:0/529835160 wait complete. 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 Processor -- start 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- start start 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0100d50 0x7fecb01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb0101670 0x7fecb01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb010e900 0x7fecb019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0114420 con 0x7fecb0101670 2024-04-21T09:14:44.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01142a0 con 0x7fecb010e900 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.608+0000 7fecb6a8c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01145a0 con 0x7fecb0100d50 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb0101670 0x7fecb01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb0101670 0x7fecb01a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44172/0 (socket says 172.21.15.66:44172) 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 -- 172.21.15.66:0/2156138808 learned_addr learned my addr 172.21.15.66:0/2156138808 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecb4828640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0100d50 0x7fecb01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecb5029640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb010e900 0x7fecb019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 -- 172.21.15.66:0/2156138808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0100d50 msgr2=0x7fecb01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0100d50 0x7fecb01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 -- 172.21.15.66:0/2156138808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb010e900 msgr2=0x7fecb019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb010e900 0x7fecb019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 -- 172.21.15.66:0/2156138808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb019afb0 con 0x7fecb0101670 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecaffff640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb0101670 0x7fecb01a0b60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7feca400d950 tx=0x7feca400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecadffb640 1 -- 172.21.15.66:0/2156138808 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4017070 con 0x7fecb0101670 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb019b2a0 con 0x7fecb0101670 2024-04-21T09:14:44.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecadffb640 1 -- 172.21.15.66:0/2156138808 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca4009c30 con 0x7fecb0101670 2024-04-21T09:14:44.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecadffb640 1 -- 172.21.15.66:0/2156138808 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca400f620 con 0x7fecb0101670 2024-04-21T09:14:44.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.609+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb01a7470 con 0x7fecb0101670 2024-04-21T09:14:44.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.611+0000 7fecadffb640 1 -- 172.21.15.66:0/2156138808 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7feca4010460 con 0x7fecb0101670 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 Processor -- start 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 -- start start 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecadffb640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec980a3d20 0x7fec980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x5622970038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562297003e80 0x56229700c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56229700ca00 0x56229700ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562296f681f0 con 0x56229700ca00 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562296f054b0 con 0x562297003e80 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb9682740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562296f641b0 con 0x5622970034d0 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb5029640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec980a3d20 0x7fec980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecadffb640 1 -- 172.21.15.66:0/2156138808 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7feca40d08a0 con 0x7fecb0101670 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.612+0000 7fecb5029640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56229700ca00 0x56229700ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56229700ca00 0x56229700ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44174/0 (socket says 172.21.15.66:44174) 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 -- 172.21.15.66:0/3867644634 learned_addr learned my addr 172.21.15.66:0/3867644634 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb4828640 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x5622970038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecaffff640 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562297003e80 0x56229700c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 -- 172.21.15.66:0/3867644634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 msgr2=0x5622970038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x5622970038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 -- 172.21.15.66:0/3867644634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562297003e80 msgr2=0x56229700c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562297003e80 0x56229700c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 -- 172.21.15.66:0/3867644634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562296ef71b0 con 0x56229700ca00 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec980a3d20 0x7fec980a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feca0005940 tx=0x7feca002c700 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fecb5029640 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56229700ca00 0x56229700ee50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fecb006b6e0 tx=0x7feca0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab7f83cfd36ae58 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fec96ffd640 1 -- 172.21.15.66:0/3867644634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0052070 con 0x56229700ca00 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fec96ffd640 1 -- 172.21.15.66:0/3867644634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca00044a0 con 0x56229700ca00 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.613+0000 7fec96ffd640 1 -- 172.21.15.66:0/3867644634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca00037f0 con 0x56229700ca00 2024-04-21T09:14:44.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 -- 172.21.15.66:0/3867644634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56229700ca00 msgr2=0x56229700ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56229700ca00 0x56229700ee50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fecb006b6e0 tx=0x7feca0046000 comp rx=0 tx=0).stop 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 -- 172.21.15.66:0/3867644634 shutdown_connections 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56229700ca00 0x56229700ee50 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562297003e80 0x56229700c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 --2- 172.21.15.66:0/3867644634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x5622970038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 -- 172.21.15.66:0/3867644634 >> 172.21.15.66:0/3867644634 conn(0x562296ffdba0 msgr2=0x562296ffdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 -- 172.21.15.66:0/3867644634 shutdown_connections 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 -- 172.21.15.66:0/3867644634 wait complete. 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 -- start start 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.614+0000 7fecb9682740 1 Processor -- start 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622970034d0 0x56229700c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x5622970096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56229700ca00 0x562297009ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56229700f910 con 0x562297003e80 2024-04-21T09:14:44.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562296f21500 con 0x5622970034d0 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56229700fa90 con 0x56229700ca00 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 10 client.0 _ll_register_callbacks cb 0x562296eefb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x5622970096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x5622970096d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44186/0 (socket says 172.21.15.66:44186) 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 -- 172.21.15.66:0/496040492 learned_addr learned my addr 172.21.15.66:0/496040492 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb4828640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622970034d0 0x56229700c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb5029640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56229700ca00 0x562297009ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 -- 172.21.15.66:0/496040492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56229700ca00 msgr2=0x562297009ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56229700ca00 0x562297009ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 -- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622970034d0 msgr2=0x56229700c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622970034d0 0x56229700c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 -- 172.21.15.66:0/496040492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562296c7a3f0 con 0x562297003e80 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb5029640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56229700ca00 0x562297009ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecaffff640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x5622970096d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7feca409c190 tx=0x7feca409cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fe4b2c85fd4fda server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fec8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fec8ffff640 1 -- 172.21.15.66:0/496040492 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4020040 con 0x562297003e80 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fec8ffff640 1 -- 172.21.15.66:0/496040492 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7feca4098b00 con 0x562297003e80 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.615+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca00044a0 con 0x562297003e80 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fec8ffff640 1 -- 172.21.15.66:0/496040492 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feca409c490 con 0x562297003e80 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 10 client.17343 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562296d447d0 con 0x562297003e80 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fec8ffff640 1 -- 172.21.15.66:0/496040492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca00044a0 con 0x562297003e80 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fec8ffff640 1 -- 172.21.15.66:0/496040492 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feca40a4c70 con 0x562297003e80 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 10 client.17343 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 10 client.17343 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 20 client.17343 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56229701cbd0 0x56229701f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 4 client.17343 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:44.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562296f07930 con 0x56229701cbd0 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.616+0000 7fecb4828640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56229701cbd0 0x56229701f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.617+0000 7fecb4828640 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56229701cbd0 0x56229701f000 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3a39110a087f15 server_cookie=1818154949db1e1d in_seq=0 out_seq=0 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.617+0000 7fec8ffff640 10 client.17343 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.617+0000 7fec8ffff640 1 -- 172.21.15.66:0/496040492 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562296f07930 con 0x56229701cbd0 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.617+0000 7fec8ffff640 10 client.17343 handle_command_reply: tid=0 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.617+0000 7fecb9682740 1 client.17343 shutdown 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 msgr2=0x5622970096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x5622970096d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7feca409c190 tx=0x7feca409cc50 comp rx=0 tx=0).stop 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 shutdown_connections 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56229701cbd0 0x56229701f000 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56229700ca00 0x562297009ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x5622970096d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fec8ffff640 0 client.17343 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 --2- 172.21.15.66:0/496040492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622970034d0 0x56229700c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 >> 172.21.15.66:0/496040492 conn(0x562296ffdba0 msgr2=0x562296fff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 shutdown_connections 2024-04-21T09:14:44.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 1 -- 172.21.15.66:0/496040492 wait complete. 2024-04-21T09:14:44.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.618+0000 7fecb9682740 20 client.17343 trim_cache size 0 max 16384 2024-04-21T09:14:44.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.631+0000 7fecb9682740 1 Processor -- start 2024-04-21T09:14:44.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.631+0000 7fecb9682740 1 -- start start 2024-04-21T09:14:44.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.631+0000 7fecb9682740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622970034d0 0x5622970187d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb9682740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562297003e80 0x56229700be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb9682740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x56229700c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb9682740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01b3e00 con 0x5622970034d0 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb9682740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01b16d0 con 0x56229700ca00 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb9682740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01af500 con 0x562297003e80 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622970034d0 0x5622970187d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622970034d0 0x5622970187d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44192/0 (socket says 172.21.15.66:44192) 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 -- 172.21.15.66:0/2150524900 learned_addr learned my addr 172.21.15.66:0/2150524900 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecaffff640 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562297003e80 0x56229700be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb5029640 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x56229700c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 -- 172.21.15.66:0/2150524900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562297003e80 msgr2=0x56229700be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562297003e80 0x56229700be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 -- 172.21.15.66:0/2150524900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 msgr2=0x56229700c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x56229700c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 -- 172.21.15.66:0/2150524900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622970027f0 con 0x5622970034d0 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecaffff640 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562297003e80 0x56229700be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb5029640 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x56229700c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fecb4828640 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622970034d0 0x5622970187d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7feca800a5e0 tx=0x7feca8009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf3d30c237dda3f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fec967fc640 1 -- 172.21.15.66:0/2150524900 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca801e070 con 0x5622970034d0 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fec967fc640 1 -- 172.21.15.66:0/2150524900 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca801a840 con 0x5622970034d0 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.632+0000 7fec967fc640 1 -- 172.21.15.66:0/2150524900 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8019880 con 0x5622970034d0 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 -- 172.21.15.66:0/2150524900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622970034d0 msgr2=0x5622970187d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622970034d0 0x5622970187d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7feca800a5e0 tx=0x7feca8009830 comp rx=0 tx=0).stop 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 -- 172.21.15.66:0/2150524900 shutdown_connections 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x56229700c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562297003e80 0x56229700be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 --2- 172.21.15.66:0/2150524900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622970034d0 0x5622970187d0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 -- 172.21.15.66:0/2150524900 >> 172.21.15.66:0/2150524900 conn(0x562296ffdba0 msgr2=0x562297004af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 -- 172.21.15.66:0/2150524900 shutdown_connections 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 -- 172.21.15.66:0/2150524900 wait complete. 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 -- start start 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.633+0000 7fecb9682740 1 Processor -- start 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x562297010670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x562297010bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x5622970110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01afb90 con 0x562297003e80 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01b1000 con 0x56229700ca00 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01b0260 con 0x5622970034d0 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 10 client.0 _ll_register_callbacks cb 0x7feca409bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x562297010bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x562297010bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44208/0 (socket says 172.21.15.66:44208) 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 -- 172.21.15.66:0/3047944326 learned_addr learned my addr 172.21.15.66:0/3047944326 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb5029640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x5622970110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb4828640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x562297010670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 -- 172.21.15.66:0/3047944326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 msgr2=0x562297010670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x562297010670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 -- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 msgr2=0x5622970110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x5622970110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 -- 172.21.15.66:0/3047944326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562297082140 con 0x562297003e80 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb4828640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x562297010670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecaffff640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x562297010bb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7feca4098010 tx=0x7feca4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85a2e57efcb37ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fec8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fec8ffff640 1 -- 172.21.15.66:0/3047944326 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4020040 con 0x562297003e80 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fec8ffff640 1 -- 172.21.15.66:0/3047944326 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7feca40bb1e0 con 0x562297003e80 2024-04-21T09:14:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca801a840 con 0x562297003e80 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fec8ffff640 1 -- 172.21.15.66:0/3047944326 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feca409bfd0 con 0x562297003e80 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.634+0000 7fecb9682740 10 client.17349 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622970832c0 con 0x562297003e80 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fec8ffff640 1 -- 172.21.15.66:0/3047944326 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca801a840 con 0x562297003e80 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fec8ffff640 1 -- 172.21.15.66:0/3047944326 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feca4010750 con 0x562297003e80 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb9682740 10 client.17349 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb9682740 10 client.17349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb9682740 20 client.17349 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb9682740 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x562297083870 0x562297085c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb9682740 4 client.17349 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562296f75c10 con 0x562297083870 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb4828640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x562297083870 0x562297085c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fec8ffff640 10 client.17349 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.635+0000 7fecb4828640 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x562297083870 0x562297085c90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4f419143767843b server_cookie=64d266a64f5a4da5 in_seq=0 out_seq=0 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fec8ffff640 1 -- 172.21.15.66:0/3047944326 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562296f75c10 con 0x562297083870 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fec8ffff640 10 client.17349 handle_command_reply: tid=0 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 client.17349 shutdown 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 msgr2=0x562297010bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x562297010bb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7feca4098010 tx=0x7feca4010cb0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 shutdown_connections 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x562297083870 0x562297085c90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56229700ca00 0x5622970110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562297003e80 0x562297010bb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 --2- 172.21.15.66:0/3047944326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622970034d0 0x562297010670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fec8ffff640 0 client.17349 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:14:44.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 >> 172.21.15.66:0/3047944326 conn(0x562296ffdba0 msgr2=0x562296ffc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 shutdown_connections 2024-04-21T09:14:44.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 1 -- 172.21.15.66:0/3047944326 wait complete. 2024-04-21T09:14:44.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb9682740 20 client.17349 trim_cache size 0 max 16384 2024-04-21T09:14:44.644 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:14:44.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec980a3d20 msgr2=0x7fec980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.636+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec980a3d20 0x7fec980a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feca0005940 tx=0x7feca002c700 comp rx=0 tx=0).stop 2024-04-21T09:14:44.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb0101670 msgr2=0x7fecb01a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb0101670 0x7fecb01a0b60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7feca400d950 tx=0x7feca400de10 comp rx=0 tx=0).stop 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 shutdown_connections 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fec980a3d20 0x7fec980a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb010e900 0x7fecb019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecb0101670 0x7fecb01a0b60 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 --2- 172.21.15.66:0/2156138808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0100d50 0x7fecb01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 >> 172.21.15.66:0/2156138808 conn(0x7fecb0077f50 msgr2=0x7fecb010ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 shutdown_connections 2024-04-21T09:14:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.637+0000 7fecb6a8c640 1 -- 172.21.15.66:0/2156138808 wait complete. 2024-04-21T09:14:44.647 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:14:44.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 -- 172.21.15.66:0/3529674310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103970 msgr2=0x7fce6810c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 --2- 172.21.15.66:0/3529674310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103970 0x7fce6810c030 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009900 tx=0x7fce5c02bcd0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 -- 172.21.15.66:0/3529674310 shutdown_connections 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 --2- 172.21.15.66:0/3529674310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6810c570 0x7fce6810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 --2- 172.21.15.66:0/3529674310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103970 0x7fce6810c030 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 --2- 172.21.15.66:0/3529674310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce68103050 0x7fce68103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 -- 172.21.15.66:0/3529674310 >> 172.21.15.66:0/3529674310 conn(0x7fce68077fb0 msgr2=0x7fce680783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 -- 172.21.15.66:0/3529674310 shutdown_connections 2024-04-21T09:14:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 -- 172.21.15.66:0/3529674310 wait complete. 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 Processor -- start 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.751+0000 7fce6cf15640 1 -- start start 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6cf15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103050 0x7fce6819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6cf15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce68103970 0x7fce6819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6cf15640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6810c570 0x7fce681985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6cf15640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce681121b0 con 0x7fce68103050 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6cf15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce68112030 con 0x7fce68103970 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6cf15640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fce68112330 con 0x7fce6810c570 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103050 0x7fce6819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103050 0x7fce6819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60492/0 (socket says 172.21.15.66:60492) 2024-04-21T09:14:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 -- 172.21.15.66:0/521763309 learned_addr learned my addr 172.21.15.66:0/521763309 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce65d9b640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce68103970 0x7fce6819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce66d9d640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6810c570 0x7fce681985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 -- 172.21.15.66:0/521763309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6810c570 msgr2=0x7fce681985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6810c570 0x7fce681985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 -- 172.21.15.66:0/521763309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce68103970 msgr2=0x7fce6819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce68103970 0x7fce6819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 -- 172.21.15.66:0/521763309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce68198d30 con 0x7fce68103050 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce66d9d640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6810c570 0x7fce681985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce6659c640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103050 0x7fce6819e400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c970 tx=0x7fce5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.752+0000 7fce577fe640 1 -- 172.21.15.66:0/521763309 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58007bb0 con 0x7fce68103050 2024-04-21T09:14:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.753+0000 7fce577fe640 1 -- 172.21.15.66:0/521763309 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce58007d50 con 0x7fce68103050 2024-04-21T09:14:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.753+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce68199020 con 0x7fce68103050 2024-04-21T09:14:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.753+0000 7fce577fe640 1 -- 172.21.15.66:0/521763309 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5800f450 con 0x7fce68103050 2024-04-21T09:14:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.753+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce681a52c0 con 0x7fce68103050 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce577fe640 1 -- 172.21.15.66:0/521763309 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fce58004f20 con 0x7fce68103050 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 Processor -- start 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 -- start start 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a07f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a07fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a0801220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5603a07561f0 con 0x5603a07f1250 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603a06f34b0 con 0x5603a07f1c00 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5603a07521b0 con 0x5603a07fee10 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.755+0000 7fce577fe640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fce500a3da0 0x7fce500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a07f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a07f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60504/0 (socket says 172.21.15.66:60504) 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 -- 172.21.15.66:0/1737793951 learned_addr learned my addr 172.21.15.66:0/1737793951 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fce500a3da0 0x7fce500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce577fe640 1 -- 172.21.15.66:0/521763309 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fce580d0860 con 0x7fce68103050 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce6659c640 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a07fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce65d9b640 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a0801220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 -- 172.21.15.66:0/1737793951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 msgr2=0x5603a0801220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a0801220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 -- 172.21.15.66:0/1737793951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 msgr2=0x5603a07fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a07fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 -- 172.21.15.66:0/1737793951 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a06e51b0 con 0x5603a07f1250 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a07f1630 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fce60014e50 tx=0x7fce60011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a774ecbb3cf87c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce66d9d640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fce500a3da0 0x7fce500a6260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce60011b90 tx=0x7fce60016000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce54ff9640 1 -- 172.21.15.66:0/1737793951 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60022070 con 0x5603a07f1250 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce54ff9640 1 -- 172.21.15.66:0/1737793951 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce60009cd0 con 0x5603a07f1250 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.756+0000 7fce54ff9640 1 -- 172.21.15.66:0/1737793951 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60005220 con 0x5603a07f1250 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1737793951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 msgr2=0x5603a07f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a07f1630 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fce60014e50 tx=0x7fce60011750 comp rx=0 tx=0).stop 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1737793951 shutdown_connections 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a0801220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a07fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1737793951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a07f1630 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1737793951 >> 172.21.15.66:0/1737793951 conn(0x5603a07ebba0 msgr2=0x5603a07ebf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1737793951 shutdown_connections 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1737793951 wait complete. 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.757+0000 7fce6fbe1740 1 -- start start 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 1 Processor -- start 2024-04-21T09:14:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a07fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 0x5603a07fbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a07fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5603a0801ce0 con 0x5603a07fee10 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603a070f500 con 0x5603a07f1250 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5603a0801e60 con 0x5603a07f1c00 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6fbe1740 10 client.0 _ll_register_callbacks cb 0x5603a06ddb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a07fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a07fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60512/0 (socket says 172.21.15.66:60512) 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 -- 172.21.15.66:0/3694322248 learned_addr learned my addr 172.21.15.66:0/3694322248 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce65d9b640 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 0x5603a07fbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce6659c640 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a07fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 -- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 msgr2=0x5603a07fbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 0x5603a07fbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 -- 172.21.15.66:0/3694322248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 msgr2=0x5603a07fe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a07fe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 -- 172.21.15.66:0/3694322248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a04683f0 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce66d9d640 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a07fc290 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fce60006010 tx=0x7fce60009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e638cea5a16f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.758+0000 7fce4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce4dffb640 1 -- 172.21.15.66:0/3694322248 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60034060 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce4dffb640 1 -- 172.21.15.66:0/3694322248 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fce600147b0 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce60009cd0 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce4dffb640 1 -- 172.21.15.66:0/3694322248 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce6003b050 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 10 client.17361 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603a05327d0 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce4dffb640 1 -- 172.21.15.66:0/3694322248 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce60009cd0 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce4dffb640 1 -- 172.21.15.66:0/3694322248 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fce60004cd0 con 0x5603a07fee10 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 10 client.17361 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 10 client.17361 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 20 client.17361 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a080ef90 0x5603a0811380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 4 client.17361 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.759+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603a06f5930 con 0x5603a080ef90 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.760+0000 7fce6659c640 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a080ef90 0x5603a0811380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.760+0000 7fce4dffb640 10 client.17361 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.760+0000 7fce6659c640 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a080ef90 0x5603a0811380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9f7f1d1226ce702 server_cookie=28098b2d74fb3005 in_seq=0 out_seq=0 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce4dffb640 1 -- 172.21.15.66:0/3694322248 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603a06f5930 con 0x5603a080ef90 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce4dffb640 10 client.17361 handle_command_reply: tid=0 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 client.17361 shutdown 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 msgr2=0x5603a07fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a07fc290 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fce60006010 tx=0x7fce60009570 comp rx=0 tx=0).stop 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 shutdown_connections 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a080ef90 0x5603a0811380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a07fc290 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce4dffb640 0 client.17361 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 0x5603a07fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/3694322248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a07fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 >> 172.21.15.66:0/3694322248 conn(0x5603a07ebba0 msgr2=0x5603a07ed8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 shutdown_connections 2024-04-21T09:14:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.762+0000 7fce6fbe1740 1 -- 172.21.15.66:0/3694322248 wait complete. 2024-04-21T09:14:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.763+0000 7fce6fbe1740 20 client.17361 trim_cache size 0 max 16384 2024-04-21T09:14:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 Processor -- start 2024-04-21T09:14:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 -- start start 2024-04-21T09:14:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a080abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 0x5603a07fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a0816980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce681afa10 con 0x5603a07fee10 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce681ad2e0 con 0x5603a07f1250 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fce681ab110 con 0x5603a07f1c00 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a0816980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a0816980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60514/0 (socket says 172.21.15.66:60514) 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 -- 172.21.15.66:0/2583599337 learned_addr learned my addr 172.21.15.66:0/2583599337 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6659c640 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a080abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 -- 172.21.15.66:0/2583599337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 msgr2=0x5603a07fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 0x5603a07fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 -- 172.21.15.66:0/2583599337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 msgr2=0x5603a080abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a080abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 -- 172.21.15.66:0/2583599337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a0801e50 con 0x5603a07fee10 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce6659c640 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a080abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.776+0000 7fce66d9d640 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a0816980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fce600092f0 tx=0x7fce60011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264c2bf78c8506a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce4ffff640 1 -- 172.21.15.66:0/2583599337 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60034030 con 0x5603a07fee10 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce4ffff640 1 -- 172.21.15.66:0/2583599337 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce60009cd0 con 0x5603a07fee10 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce4ffff640 1 -- 172.21.15.66:0/2583599337 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60022040 con 0x5603a07fee10 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 -- 172.21.15.66:0/2583599337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 msgr2=0x5603a0816980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a0816980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fce600092f0 tx=0x7fce60011bc0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 -- 172.21.15.66:0/2583599337 shutdown_connections 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07fee10 0x5603a0816980 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07f1c00 0x5603a07fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/2583599337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1250 0x5603a080abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 -- 172.21.15.66:0/2583599337 >> 172.21.15.66:0/2583599337 conn(0x5603a07ebba0 msgr2=0x5603a07ed810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 -- 172.21.15.66:0/2583599337 shutdown_connections 2024-04-21T09:14:44.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 -- 172.21.15.66:0/2583599337 wait complete. 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.777+0000 7fce6fbe1740 1 -- start start 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 1 Processor -- start 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a0802b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a08030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a08035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce681ab7a0 con 0x5603a07f1250 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce681acc10 con 0x5603a07f1c00 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fce681abe70 con 0x5603a07fee10 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6fbe1740 10 client.0 _ll_register_callbacks cb 0x7fce60009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a0802b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a0802b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60518/0 (socket says 172.21.15.66:60518) 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 -- 172.21.15.66:0/1982400790 learned_addr learned my addr 172.21.15.66:0/1982400790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce65d9b640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a08030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce66d9d640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a08035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 -- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 msgr2=0x5603a08035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a08035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 -- 172.21.15.66:0/1982400790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 msgr2=0x5603a08030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a08030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 -- 172.21.15.66:0/1982400790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a0874240 con 0x5603a07f1250 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce66d9d640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a08035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce6659c640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a0802b70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fce580058c0 tx=0x7fce5800a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5df02c7b0f96ae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.778+0000 7fce4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce4dffb640 1 -- 172.21.15.66:0/1982400790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5801d040 con 0x5603a07f1250 2024-04-21T09:14:44.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce4dffb640 1 -- 172.21.15.66:0/1982400790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fce58098b00 con 0x5603a07f1250 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce60009cd0 con 0x5603a07f1250 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce4dffb640 1 -- 172.21.15.66:0/1982400790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce580a5100 con 0x5603a07f1250 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 10 client.17367 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603a08754b0 con 0x5603a07f1250 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce4dffb640 1 -- 172.21.15.66:0/1982400790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce60009cd0 con 0x5603a07f1250 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce4dffb640 1 -- 172.21.15.66:0/1982400790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fce58005210 con 0x5603a07f1250 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 10 client.17367 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 10 client.17367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 20 client.17367 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a0875c00 0x5603a0878020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 4 client.17367 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.779+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603a0763c10 con 0x5603a0875c00 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce65d9b640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a0875c00 0x5603a0878020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce65d9b640 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a0875c00 0x5603a0878020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3562bc28f4d72cc server_cookie=a618547df6479dfc in_seq=0 out_seq=0 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce4dffb640 10 client.17367 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce4dffb640 1 -- 172.21.15.66:0/1982400790 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603a0763c10 con 0x5603a0875c00 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce4dffb640 10 client.17367 handle_command_reply: tid=0 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 client.17367 shutdown 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 msgr2=0x5603a0802b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a0802b70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fce580058c0 tx=0x7fce5800a380 comp rx=0 tx=0).stop 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 shutdown_connections 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5603a07fee10 0x5603a08035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603a07f1c00 0x5603a08030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5603a0875c00 0x5603a0878020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 --2- 172.21.15.66:0/1982400790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603a07f1250 0x5603a0802b70 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 >> 172.21.15.66:0/1982400790 conn(0x5603a07ebba0 msgr2=0x5603a07ecf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.780+0000 7fce4dffb640 0 client.17367 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:14:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 shutdown_connections 2024-04-21T09:14:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6fbe1740 1 -- 172.21.15.66:0/1982400790 wait complete. 2024-04-21T09:14:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6fbe1740 20 client.17367 trim_cache size 0 max 16384 2024-04-21T09:14:44.788 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:14:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fce500a3da0 msgr2=0x7fce500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fce500a3da0 0x7fce500a6260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce60011b90 tx=0x7fce60016000 comp rx=0 tx=0).stop 2024-04-21T09:14:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103050 msgr2=0x7fce6819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103050 0x7fce6819e400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c970 tx=0x7fce5800ce30 comp rx=0 tx=0).stop 2024-04-21T09:14:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 shutdown_connections 2024-04-21T09:14:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fce500a3da0 0x7fce500a6260 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6810c570 0x7fce681985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce68103970 0x7fce6819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 --2- 172.21.15.66:0/521763309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce68103050 0x7fce6819e400 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 >> 172.21.15.66:0/521763309 conn(0x7fce68077fb0 msgr2=0x7fce680fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 shutdown_connections 2024-04-21T09:14:44.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.781+0000 7fce6cf15640 1 -- 172.21.15.66:0/521763309 wait complete. 2024-04-21T09:14:44.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 -- 172.21.15.66:0/1735658505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102b40 msgr2=0x7f0d3c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 --2- 172.21.15.66:0/1735658505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102b40 0x7f0d3c106b80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009900 tx=0x7f0d2402bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 -- 172.21.15.66:0/1735658505 shutdown_connections 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 --2- 172.21.15.66:0/1735658505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d3c107260 0x7f0d3c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 --2- 172.21.15.66:0/1735658505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102b40 0x7f0d3c106b80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 --2- 172.21.15.66:0/1735658505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c102190 0x7f0d3c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 -- 172.21.15.66:0/1735658505 >> 172.21.15.66:0/1735658505 conn(0x7f0d3c0fd990 msgr2=0x7f0d3c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 -- 172.21.15.66:0/1735658505 shutdown_connections 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.896+0000 7f0d41700640 1 -- 172.21.15.66:0/1735658505 wait complete. 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 Processor -- start 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 -- start start 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102190 0x7f0d3c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d3c107260 0x7f0d3c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c19d020 0x7f0d3c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c112180 con 0x7f0d3c102190 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c112000 con 0x7f0d3c107260 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d41700640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c112300 con 0x7f0d3c19d020 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102190 0x7f0d3c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102190 0x7f0d3c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60544/0 (socket says 172.21.15.66:60544) 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 -- 172.21.15.66:0/2963235475 learned_addr learned my addr 172.21.15.66:0/2963235475 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d33fff640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d3c107260 0x7f0d3c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 -- 172.21.15.66:0/2963235475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c19d020 msgr2=0x7f0d3c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c19d020 0x7f0d3c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 -- 172.21.15.66:0/2963235475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d3c107260 msgr2=0x7f0d3c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d3c107260 0x7f0d3c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 -- 172.21.15.66:0/2963235475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19db10 con 0x7f0d3c102190 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d33fff640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d3c107260 0x7f0d3c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d3affd640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102190 0x7f0d3c198760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9f0 tx=0x7f0d2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.897+0000 7f0d38ff9640 1 -- 172.21.15.66:0/2963235475 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00cd80 con 0x7f0d3c102190 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.898+0000 7f0d38ff9640 1 -- 172.21.15.66:0/2963235475 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2c0108e0 con 0x7f0d3c102190 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.898+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19de00 con 0x7f0d3c102190 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.898+0000 7f0d38ff9640 1 -- 172.21.15.66:0/2963235475 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c018980 con 0x7f0d3c102190 2024-04-21T09:14:44.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.898+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a5610 con 0x7f0d3c102190 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.900+0000 7f0d38ff9640 1 -- 172.21.15.66:0/2963235475 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0d2c010430 con 0x7f0d3c102190 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.900+0000 7f0d443cd740 1 Processor -- start 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.900+0000 7f0d443cd740 1 -- start start 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19cf1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1c00 0x556f19cfe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cfee10 0x556f19d01220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d443cd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556f19c561f0 con 0x556f19cf1250 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d443cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f19bf34b0 con 0x556f19cf1c00 2024-04-21T09:14:44.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d443cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556f19c521b0 con 0x556f19cfee10 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d3b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19cf1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d3b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19cf1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60546/0 (socket says 172.21.15.66:60546) 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d3b7fe640 1 -- 172.21.15.66:0/1352152864 learned_addr learned my addr 172.21.15.66:0/1352152864 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d38ff9640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d280a3da0 0x7f0d280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d33fff640 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cfee10 0x556f19d01220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d3affd640 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1c00 0x556f19cfe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d280a3da0 0x7f0d280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d38ff9640 1 -- 172.21.15.66:0/2963235475 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f0d2c0d0fa0 con 0x7f0d3c102190 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d33fff640 1 -- 172.21.15.66:0/1352152864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1c00 msgr2=0x556f19cfe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d33fff640 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1c00 0x556f19cfe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d33fff640 1 -- 172.21.15.66:0/1352152864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 msgr2=0x556f19cf1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d33fff640 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19cf1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d33fff640 1 -- 172.21.15.66:0/1352152864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f19be51b0 con 0x556f19cfee10 2024-04-21T09:14:44.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d33fff640 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cfee10 0x556f19d01220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d24036040 tx=0x7f0d24005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7466944ba38b3ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.901+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d280a3da0 0x7f0d280a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d340092f0 tx=0x7f0d34005bd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.902+0000 7f0d317fa640 1 -- 172.21.15.66:0/1352152864 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24044020 con 0x556f19cfee10 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.902+0000 7f0d317fa640 1 -- 172.21.15.66:0/1352152864 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2402cc90 con 0x556f19cfee10 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.902+0000 7f0d317fa640 1 -- 172.21.15.66:0/1352152864 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240046c0 con 0x556f19cfee10 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.902+0000 7f0d443cd740 1 -- 172.21.15.66:0/1352152864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cfee10 msgr2=0x556f19d01220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.902+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cfee10 0x556f19d01220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d24036040 tx=0x7f0d24005e50 comp rx=0 tx=0).stop 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- 172.21.15.66:0/1352152864 shutdown_connections 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cfee10 0x556f19d01220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1c00 0x556f19cfe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1352152864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19cf1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- 172.21.15.66:0/1352152864 >> 172.21.15.66:0/1352152864 conn(0x556f19cebba0 msgr2=0x556f19cebf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- 172.21.15.66:0/1352152864 shutdown_connections 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- 172.21.15.66:0/1352152864 wait complete. 2024-04-21T09:14:44.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- start start 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 Processor -- start 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1250 0x556f19cfe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19cfba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cfee10 0x556f19cfbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556f19d01ce0 con 0x556f19cfee10 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f19c0f500 con 0x556f19cf1250 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556f19d01e60 con 0x556f19cf1c00 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d443cd740 10 client.0 _ll_register_callbacks cb 0x556f19bddb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d3b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cfee10 0x556f19cfbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d3b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cfee10 0x556f19cfbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60558/0 (socket says 172.21.15.66:60558) 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d3b7fe640 1 -- 172.21.15.66:0/814895589 learned_addr learned my addr 172.21.15.66:0/814895589 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d33fff640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19cfba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.903+0000 7f0d3affd640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1250 0x556f19cfe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d3b7fe640 1 -- 172.21.15.66:0/814895589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 msgr2=0x556f19cfba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19cfba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d3b7fe640 1 -- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1250 msgr2=0x556f19cfe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1250 0x556f19cfe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d3b7fe640 1 -- 172.21.15.66:0/814895589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f199683f0 con 0x556f19cfee10 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d3affd640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1250 0x556f19cfe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cfee10 0x556f19cfbf80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0d34006010 tx=0x7f0d34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a72c0c37dd071da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d227fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d227fc640 1 -- 172.21.15.66:0/814895589 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34003b70 con 0x556f19cfee10 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d227fc640 1 -- 172.21.15.66:0/814895589 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f0d34003d10 con 0x556f19cfee10 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d2402cc90 con 0x556f19cfee10 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d227fc640 1 -- 172.21.15.66:0/814895589 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d3402e050 con 0x556f19cfee10 2024-04-21T09:14:44.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d443cd740 10 client.17379 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f19a327d0 con 0x556f19cfee10 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d227fc640 1 -- 172.21.15.66:0/814895589 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2402cc90 con 0x556f19cfee10 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d227fc640 1 -- 172.21.15.66:0/814895589 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d34010050 con 0x556f19cfee10 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d443cd740 10 client.17379 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.904+0000 7f0d443cd740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.905+0000 7f0d443cd740 20 client.17379 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.905+0000 7f0d443cd740 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d0f090 0x556f19d11510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.905+0000 7f0d443cd740 4 client.17379 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.905+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f19bf5930 con 0x556f19d0f090 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.905+0000 7f0d3affd640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d0f090 0x556f19d11510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.905+0000 7f0d227fc640 10 client.17379 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.905+0000 7f0d3affd640 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d0f090 0x556f19d11510 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e9874e01c1d2e0 server_cookie=17bcaaf568056577 in_seq=0 out_seq=0 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d227fc640 1 -- 172.21.15.66:0/814895589 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f19bf5930 con 0x556f19d0f090 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d227fc640 10 client.17379 handle_command_reply: tid=0 2024-04-21T09:14:44.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 client.17379 shutdown 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cfee10 msgr2=0x556f19cfbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cfee10 0x556f19cfbf80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0d34006010 tx=0x7f0d34004a40 comp rx=0 tx=0).stop 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 shutdown_connections 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d0f090 0x556f19d11510 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cfee10 0x556f19cfbf80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19cfba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d227fc640 0 client.17379 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 --2- 172.21.15.66:0/814895589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cf1250 0x556f19cfe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 >> 172.21.15.66:0/814895589 conn(0x556f19cebba0 msgr2=0x556f19ced8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 shutdown_connections 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 1 -- 172.21.15.66:0/814895589 wait complete. 2024-04-21T09:14:44.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.906+0000 7f0d443cd740 20 client.17379 trim_cache size 0 max 16384 2024-04-21T09:14:44.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 Processor -- start 2024-04-21T09:14:44.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 -- start start 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d0abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19cfb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d16de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1af930 con 0x556f19cf1250 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ad200 con 0x556f19cfee10 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d443cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ab030 con 0x556f19cf1c00 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d3affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d0abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d3affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d0abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60562/0 (socket says 172.21.15.66:60562) 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d3affd640 1 -- 172.21.15.66:0/584440143 learned_addr learned my addr 172.21.15.66:0/584440143 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.920+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d16de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d3affd640 1 -- 172.21.15.66:0/584440143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 msgr2=0x556f19cfb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d3affd640 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19cfb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d3affd640 1 -- 172.21.15.66:0/584440143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 msgr2=0x556f19d16de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d3affd640 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d16de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d3affd640 1 -- 172.21.15.66:0/584440143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f19d0db90 con 0x556f19cf1250 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d3affd640 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d0abe0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c06f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a954a95145201f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d30ff9640 1 -- 172.21.15.66:0/584440143 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0a5840 con 0x556f19cf1250 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d30ff9640 1 -- 172.21.15.66:0/584440143 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2c09aed0 con 0x556f19cf1250 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d30ff9640 1 -- 172.21.15.66:0/584440143 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0993b0 con 0x556f19cf1250 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d443cd740 1 -- 172.21.15.66:0/584440143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 msgr2=0x556f19d0abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.921+0000 7f0d443cd740 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d0abe0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c06f780 comp rx=0 tx=0).stop 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- 172.21.15.66:0/584440143 shutdown_connections 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d16de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19cfb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 --2- 172.21.15.66:0/584440143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d0abe0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- 172.21.15.66:0/584440143 >> 172.21.15.66:0/584440143 conn(0x556f19cebba0 msgr2=0x556f19cf7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- 172.21.15.66:0/584440143 shutdown_connections 2024-04-21T09:14:44.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- 172.21.15.66:0/584440143 wait complete. 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- start start 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 Processor -- start 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d07410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19d027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d02d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ab6c0 con 0x556f19cf1250 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1acb30 con 0x556f19cfee10 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1abd90 con 0x556f19cf1c00 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d443cd740 10 client.0 _ll_register_callbacks cb 0x7f0d34012950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d3affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d07410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d3affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d07410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60568/0 (socket says 172.21.15.66:60568) 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.922+0000 7f0d3affd640 1 -- 172.21.15.66:0/1717172962 learned_addr learned my addr 172.21.15.66:0/1717172962 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d02d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3affd640 1 -- 172.21.15.66:0/1717172962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 msgr2=0x556f19d027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3affd640 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19d027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3affd640 1 -- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 msgr2=0x556f19d02d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3affd640 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d02d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3affd640 1 -- 172.21.15.66:0/1717172962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f19d03510 con 0x556f19cf1250 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3b7fe640 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d02d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d3affd640 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d07410 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0a5310 tx=0x7f0d2c010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6842d52657a484 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d227fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d227fc640 1 -- 172.21.15.66:0/1717172962 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0b5020 con 0x556f19cf1250 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d227fc640 1 -- 172.21.15.66:0/1717172962 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f0d2c0ce600 con 0x556f19cf1250 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d2c09aed0 con 0x556f19cf1250 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d227fc640 1 -- 172.21.15.66:0/1717172962 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d2c09d2d0 con 0x556f19cf1250 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 10 client.17385 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f19d755c0 con 0x556f19cf1250 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d227fc640 1 -- 172.21.15.66:0/1717172962 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2c09aed0 con 0x556f19cf1250 2024-04-21T09:14:44.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d227fc640 1 -- 172.21.15.66:0/1717172962 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d2c018b90 con 0x556f19cf1250 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 10 client.17385 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 20 client.17385 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d75c60 0x556f19d78080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 4 client.17385 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.923+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f19c63c10 con 0x556f19d75c60 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d33fff640 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d75c60 0x556f19d78080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d227fc640 10 client.17385 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d33fff640 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d75c60 0x556f19d78080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c917a0133a0906 server_cookie=a993c283a1423fa5 in_seq=0 out_seq=0 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d227fc640 1 -- 172.21.15.66:0/1717172962 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f19c63c10 con 0x556f19d75c60 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d227fc640 10 client.17385 handle_command_reply: tid=0 2024-04-21T09:14:44.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d443cd740 1 client.17385 shutdown 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 msgr2=0x556f19d07410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.924+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d07410 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0a5310 tx=0x7f0d2c010870 comp rx=0 tx=0).stop 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 shutdown_connections 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556f19d75c60 0x556f19d78080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f19cfee10 0x556f19d02d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d227fc640 0 client.17385 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f19cf1c00 0x556f19d027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 --2- 172.21.15.66:0/1717172962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f19cf1250 0x556f19d07410 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 >> 172.21.15.66:0/1717172962 conn(0x556f19cebba0 msgr2=0x556f19cea370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 shutdown_connections 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 1 -- 172.21.15.66:0/1717172962 wait complete. 2024-04-21T09:14:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d443cd740 20 client.17385 trim_cache size 0 max 16384 2024-04-21T09:14:44.933 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:14:44.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d280a3da0 msgr2=0x7f0d280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d280a3da0 0x7f0d280a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d340092f0 tx=0x7f0d34005bd0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102190 msgr2=0x7f0d3c198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102190 0x7f0d3c198760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9f0 tx=0x7f0d2c00eeb0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 shutdown_connections 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d280a3da0 0x7f0d280a6260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c19d020 0x7f0d3c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d3c107260 0x7f0d3c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 --2- 172.21.15.66:0/2963235475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3c102190 0x7f0d3c198760 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 >> 172.21.15.66:0/2963235475 conn(0x7f0d3c0fd990 msgr2=0x7f0d3c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 shutdown_connections 2024-04-21T09:14:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:44.925+0000 7f0d41700640 1 -- 172.21.15.66:0/2963235475 wait complete. 2024-04-21T09:14:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:45 smithi186 ceph-mon[31505]: pgmap v199: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:45 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3105258289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:14:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:45 smithi083 ceph-mon[31413]: pgmap v199: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:45.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:45 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3105258289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:14:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:45 smithi066 ceph-mon[27953]: pgmap v199: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:14:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:45 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3105258289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:14:45.937 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:14:45.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:14:45.938 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 -- 172.21.15.66:0/839347621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d406a840 msgr2=0x7f98d406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 --2- 172.21.15.66:0/839347621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d406a840 0x7f98d406ac20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009900 tx=0x7f98c802bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 -- 172.21.15.66:0/839347621 shutdown_connections 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 --2- 172.21.15.66:0/839347621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d4107270 0x7f98d410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 --2- 172.21.15.66:0/839347621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d406b1f0 0x7f98d4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 --2- 172.21.15.66:0/839347621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d406a840 0x7f98d406ac20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 -- 172.21.15.66:0/839347621 >> 172.21.15.66:0/839347621 conn(0x7f98d4077eb0 msgr2=0x7f98d40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 -- 172.21.15.66:0/839347621 shutdown_connections 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 -- 172.21.15.66:0/839347621 wait complete. 2024-04-21T09:14:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 Processor -- start 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.056+0000 7f98da552640 1 -- start start 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98da552640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d406a840 0x7f98d4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98da552640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d4107270 0x7f98d4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98da552640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d419cfd0 0x7f98d419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98da552640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41121d0 con 0x7f98d419cfd0 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98da552640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98d4112050 con 0x7f98d4107270 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98da552640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f98d4112350 con 0x7f98d406a840 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d419cfd0 0x7f98d419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d419cfd0 0x7f98d419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60582/0 (socket says 172.21.15.66:60582) 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 -- 172.21.15.66:0/3449712843 learned_addr learned my addr 172.21.15.66:0/3449712843 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d3fff640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d406a840 0x7f98d4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d4107270 0x7f98d4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 -- 172.21.15.66:0/3449712843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d406a840 msgr2=0x7f98d4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d406a840 0x7f98d4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 -- 172.21.15.66:0/3449712843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d4107270 msgr2=0x7f98d4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d4107270 0x7f98d4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 -- 172.21.15.66:0/3449712843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d419dac0 con 0x7f98d419cfd0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d4107270 0x7f98d4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d8aef640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d419cfd0 0x7f98d419d480 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009fd0 tx=0x7f98cc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.057+0000 7f98d17fa640 1 -- 172.21.15.66:0/3449712843 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc017070 con 0x7f98d419cfd0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.058+0000 7f98d17fa640 1 -- 172.21.15.66:0/3449712843 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc003e20 con 0x7f98d419cfd0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.058+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d419ddb0 con 0x7f98d419cfd0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.058+0000 7f98d17fa640 1 -- 172.21.15.66:0/3449712843 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc010460 con 0x7f98d419cfd0 2024-04-21T09:14:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.058+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d41a5550 con 0x7f98d419cfd0 2024-04-21T09:14:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.060+0000 7f98d17fa640 1 -- 172.21.15.66:0/3449712843 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f98cc003970 con 0x7f98d419cfd0 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.060+0000 7f98dd224740 1 Processor -- start 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.060+0000 7f98dd224740 1 -- start start 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.060+0000 7f98dd224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98dd224740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98dd224740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98dd224740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a1e2efef00 con 0x55a1e2f9b290 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98dd224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1e2e9c590 con 0x55a1e2f9a8e0 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98dd224740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a1e2efaec0 con 0x55a1e2fa3e10 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60586/0 (socket says 172.21.15.66:60586) 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 -- 172.21.15.66:0/1287815788 learned_addr learned my addr 172.21.15.66:0/1287815788 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d17fa640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3d20 0x7f98bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d37fe640 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d37fe640 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d8aef640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3d20 0x7f98bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 -- 172.21.15.66:0/1287815788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2fa3e10 msgr2=0x55a1e2fa6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 -- 172.21.15.66:0/1287815788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 msgr2=0x55a1e2f9acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 -- 172.21.15.66:0/1287815788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e2cd97e0 con 0x55a1e2f9b290 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d17fa640 1 -- 172.21.15.66:0/3449712843 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f98cc0d0960 con 0x7f98d419cfd0 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.061+0000 7f98d3fff640 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa3840 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009900 tx=0x7f98c8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377043ddf92ef456 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98d8aef640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3d20 0x7f98bc0a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0989f0 tx=0x7f98cc06f080 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98ba7fc640 1 -- 172.21.15.66:0/1287815788 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8044020 con 0x55a1e2f9b290 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98ba7fc640 1 -- 172.21.15.66:0/1287815788 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c802cc90 con 0x55a1e2f9b290 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98ba7fc640 1 -- 172.21.15.66:0/1287815788 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c80046c0 con 0x55a1e2f9b290 2024-04-21T09:14:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 -- 172.21.15.66:0/1287815788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 msgr2=0x55a1e2fa3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa3840 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009900 tx=0x7f98c8005e70 comp rx=0 tx=0).stop 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 -- 172.21.15.66:0/1287815788 shutdown_connections 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa3840 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 --2- 172.21.15.66:0/1287815788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 -- 172.21.15.66:0/1287815788 >> 172.21.15.66:0/1287815788 conn(0x55a1e2f94fb0 msgr2=0x55a1e2f95390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 -- 172.21.15.66:0/1287815788 shutdown_connections 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.062+0000 7f98dd224740 1 -- 172.21.15.66:0/1287815788 wait complete. 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 -- start start 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 Processor -- start 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2fa0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a1e2fa6d20 con 0x55a1e2fa3e10 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1e2eb8580 con 0x55a1e2f9b290 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a1e2fa6ea0 con 0x55a1e2f9a8e0 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98dd224740 10 client.0 _ll_register_callbacks cb 0x55a1e2ebe650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60594/0 (socket says 172.21.15.66:60594) 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98d37fe640 1 -- 172.21.15.66:0/2379265985 learned_addr learned my addr 172.21.15.66:0/2379265985 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.063+0000 7f98d37fe640 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2fa0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98d3fff640 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98d37fe640 1 -- 172.21.15.66:0/2379265985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 msgr2=0x55a1e2fa0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98d37fe640 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2fa0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98d37fe640 1 -- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9b290 msgr2=0x55a1e2fa1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98d37fe640 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98d37fe640 1 -- 172.21.15.66:0/2379265985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e2d39f70 con 0x55a1e2fa3e10 2024-04-21T09:14:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98d37fe640 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa1880 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f98c4014e00 tx=0x7f98c4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856bf23ace2a48cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98b37fe640 1 -- 172.21.15.66:0/2379265985 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4014110 con 0x55a1e2fa3e10 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98b37fe640 1 -- 172.21.15.66:0/2379265985 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f98c4012450 con 0x55a1e2fa3e10 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c802cc90 con 0x55a1e2fa3e10 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98b37fe640 1 -- 172.21.15.66:0/2379265985 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f98c40142b0 con 0x55a1e2fa3e10 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98dd224740 10 client.17397 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1e2bd1170 con 0x55a1e2fa3e10 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.064+0000 7f98b37fe640 1 -- 172.21.15.66:0/2379265985 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c802cc90 con 0x55a1e2fa3e10 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98b37fe640 1 -- 172.21.15.66:0/2379265985 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f98c4020670 con 0x55a1e2fa3e10 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98dd224740 10 client.17397 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98dd224740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98dd224740 20 client.17397 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:46.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98dd224740 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e2fb43b0 0x55a1e2fb67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98dd224740 4 client.17397 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1e2f0c920 con 0x55a1e2fb43b0 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98d3fff640 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e2fb43b0 0x55a1e2fb67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98b37fe640 10 client.17397 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.065+0000 7f98d3fff640 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e2fb43b0 0x55a1e2fb67a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a101c4907234d8d server_cookie=23f196ff041acae1 in_seq=0 out_seq=0 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98b37fe640 1 -- 172.21.15.66:0/2379265985 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1e2f0c920 con 0x55a1e2fb43b0 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98b37fe640 10 client.17397 handle_command_reply: tid=0 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 client.17397 shutdown 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 msgr2=0x55a1e2fa1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa1880 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f98c4014e00 tx=0x7f98c4016040 comp rx=0 tx=0).stop 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 shutdown_connections 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e2fb43b0 0x55a1e2fb67a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2fa1880 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9b290 0x55a1e2fa1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98b37fe640 0 client.17397 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 --2- 172.21.15.66:0/2379265985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2fa0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 >> 172.21.15.66:0/2379265985 conn(0x55a1e2f94fb0 msgr2=0x55a1e2f96d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 shutdown_connections 2024-04-21T09:14:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 1 -- 172.21.15.66:0/2379265985 wait complete. 2024-04-21T09:14:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.066+0000 7f98dd224740 20 client.17397 trim_cache size 0 max 16384 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 Processor -- start 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 -- start start 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2fa3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fafbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f96b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41b1ad0 con 0x55a1e2f9b290 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41af3a0 con 0x55a1e2fa3e10 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98dd224740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41ad1d0 con 0x55a1e2f9a8e0 2024-04-21T09:14:46.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fafbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fafbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60602/0 (socket says 172.21.15.66:60602) 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 -- 172.21.15.66:0/3591446185 learned_addr learned my addr 172.21.15.66:0/3591446185 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f96b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 -- 172.21.15.66:0/3591446185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 msgr2=0x55a1e2fa3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2fa3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 -- 172.21.15.66:0/3591446185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2fa3e10 msgr2=0x55a1e2f96b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f96b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d3fff640 1 -- 172.21.15.66:0/3591446185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e2f99ba0 con 0x55a1e2f9b290 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.084+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f96b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98d3fff640 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fafbe0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009900 tx=0x7f98c80053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6177c534771ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98b9ffb640 1 -- 172.21.15.66:0/3591446185 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8044040 con 0x55a1e2f9b290 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98b9ffb640 1 -- 172.21.15.66:0/3591446185 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c8005440 con 0x55a1e2f9b290 2024-04-21T09:14:46.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98b9ffb640 1 -- 172.21.15.66:0/3591446185 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8003970 con 0x55a1e2f9b290 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 -- 172.21.15.66:0/3591446185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 msgr2=0x55a1e2fafbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fafbe0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009900 tx=0x7f98c80053d0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 -- 172.21.15.66:0/3591446185 shutdown_connections 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f96b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2f9b290 0x55a1e2fafbe0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 --2- 172.21.15.66:0/3591446185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2fa3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 -- 172.21.15.66:0/3591446185 >> 172.21.15.66:0/3591446185 conn(0x55a1e2f94fb0 msgr2=0x55a1e2f96400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 -- 172.21.15.66:0/3591446185 shutdown_connections 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.085+0000 7f98dd224740 1 -- 172.21.15.66:0/3591446185 wait complete. 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 -- start start 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 Processor -- start 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9b290 0x55a1e2f9e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41ad860 con 0x55a1e2fa3e10 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41aecd0 con 0x55a1e2f9a8e0 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41adf30 con 0x55a1e2f9b290 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98dd224740 10 client.0 _ll_register_callbacks cb 0x7f98c400c3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f9e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60612/0 (socket says 172.21.15.66:60612) 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98d37fe640 1 -- 172.21.15.66:0/3641668505 learned_addr learned my addr 172.21.15.66:0/3641668505 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.086+0000 7f98d3fff640 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9b290 0x55a1e2f9e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98d37fe640 1 -- 172.21.15.66:0/3641668505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9b290 msgr2=0x55a1e2f9e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9b290 0x55a1e2f9e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98d37fe640 1 -- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 msgr2=0x55a1e2f9db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98d37fe640 1 -- 172.21.15.66:0/3641668505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e301dda0 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98d37fe640 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f9e5e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c840 tx=0x7f98c4005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c35cb2cae852099 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98b37fe640 1 -- 172.21.15.66:0/3641668505 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c400c8d0 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98b37fe640 1 -- 172.21.15.66:0/3641668505 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f98c4005070 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c8005440 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98b37fe640 1 -- 172.21.15.66:0/3641668505 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f98c4005270 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98dd224740 10 client.17403 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.087+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1e301f0a0 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98b37fe640 1 -- 172.21.15.66:0/3641668505 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c8005440 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98b37fe640 1 -- 172.21.15.66:0/3641668505 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f98c403a070 con 0x55a1e2fa3e10 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98dd224740 10 client.17403 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98dd224740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:46.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98dd224740 20 client.17403 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:46.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98dd224740 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e301f790 0x55a1e3021bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98dd224740 4 client.17403 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:14:46.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1e2f06790 con 0x55a1e301f790 2024-04-21T09:14:46.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98d3fff640 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e301f790 0x55a1e3021bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98b37fe640 10 client.17403 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.088+0000 7f98d3fff640 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e301f790 0x55a1e3021bb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7fe14d4d433ecb server_cookie=5565c42cc2cf1a34 in_seq=0 out_seq=0 2024-04-21T09:14:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.089+0000 7f98b37fe640 1 -- 172.21.15.66:0/3641668505 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1e2f06790 con 0x55a1e301f790 2024-04-21T09:14:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.089+0000 7f98b37fe640 10 client.17403 handle_command_reply: tid=0 2024-04-21T09:14:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.089+0000 7f98dd224740 1 client.17403 shutdown 2024-04-21T09:14:46.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 msgr2=0x55a1e2f9e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f9e5e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c840 tx=0x7f98c4005990 comp rx=0 tx=0).stop 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 shutdown_connections 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55a1e301f790 0x55a1e3021bb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1e2fa3e10 0x55a1e2f9e5e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1e2f9b290 0x55a1e2f9e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 --2- 172.21.15.66:0/3641668505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e2f9a8e0 0x55a1e2f9db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98b37fe640 0 client.17403 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 >> 172.21.15.66:0/3641668505 conn(0x55a1e2f94fb0 msgr2=0x55a1e2f96c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 shutdown_connections 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 1 -- 172.21.15.66:0/3641668505 wait complete. 2024-04-21T09:14:46.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98dd224740 20 client.17403 trim_cache size 0 max 16384 2024-04-21T09:14:46.097 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:14:46.097 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:14:46.097 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "4c8c6a03-0158-41ba-9e14-e0efa9a478bb", 2024-04-21T09:14:46.097 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:14:46.097 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3d20 msgr2=0x7f98bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98da552640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3d20 0x7f98bc0a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0989f0 tx=0x7f98cc06f080 comp rx=0 tx=0).stop 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d419cfd0 msgr2=0x7f98d419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.090+0000 7f98da552640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d419cfd0 0x7f98d419d480 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009fd0 tx=0x7f98cc00b260 comp rx=0 tx=0).stop 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 shutdown_connections 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f98bc0a3d20 0x7f98bc0a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98d419cfd0 0x7f98d419d480 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98d4107270 0x7f98d4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 --2- 172.21.15.66:0/3449712843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98d406a840 0x7f98d4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 >> 172.21.15.66:0/3449712843 conn(0x7f98d4077eb0 msgr2=0x7f98d4108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 shutdown_connections 2024-04-21T09:14:46.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.091+0000 7f98da552640 1 -- 172.21.15.66:0/3449712843 wait complete. 2024-04-21T09:14:46.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:14:46.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:14:46.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 -- 172.21.15.66:0/2437069428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 msgr2=0x7f368806aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 --2- 172.21.15.66:0/2437069428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368806aa30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3670009930 tx=0x7f367002bc30 comp rx=0 tx=0).stop 2024-04-21T09:14:46.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 -- 172.21.15.66:0/2437069428 shutdown_connections 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 --2- 172.21.15.66:0/2437069428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f368806af70 0x7f368810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 --2- 172.21.15.66:0/2437069428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368806aa30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 --2- 172.21.15.66:0/2437069428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3688108060 0x7f3688108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 -- 172.21.15.66:0/2437069428 >> 172.21.15.66:0/2437069428 conn(0x7f3688077c40 msgr2=0x7f3688078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 -- 172.21.15.66:0/2437069428 shutdown_connections 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.207+0000 7f368d73d640 1 -- 172.21.15.66:0/2437069428 wait complete. 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 Processor -- start 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 -- start start 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f368806af70 0x7f368819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3688108060 0x7f36881985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3688112200 con 0x7f368806a5b0 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3688112080 con 0x7f3688108060 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f368d73d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3688112380 con 0x7f368806af70 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368819e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60634/0 (socket says 172.21.15.66:60634) 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f3686ffd640 1 -- 172.21.15.66:0/3912658081 learned_addr learned my addr 172.21.15.66:0/3912658081 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.208+0000 7f36877fe640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3688108060 0x7f36881985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3686ffd640 1 -- 172.21.15.66:0/3912658081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f368806af70 msgr2=0x7f368819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3686ffd640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f368806af70 0x7f368819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3686ffd640 1 -- 172.21.15.66:0/3912658081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3688108060 msgr2=0x7f36881985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3686ffd640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3688108060 0x7f36881985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3686ffd640 1 -- 172.21.15.66:0/3912658081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3688198d60 con 0x7f368806a5b0 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f36877fe640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3688108060 0x7f36881985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3686ffd640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368819e430 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f367800ea60 tx=0x7f367800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3684ff9640 1 -- 172.21.15.66:0/3912658081 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367800cde0 con 0x7f368806a5b0 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3688199050 con 0x7f368806a5b0 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3684ff9640 1 -- 172.21.15.66:0/3912658081 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36780035f0 con 0x7f368806a5b0 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f3684ff9640 1 -- 172.21.15.66:0/3912658081 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678010690 con 0x7f368806a5b0 2024-04-21T09:14:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.209+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36881a52f0 con 0x7f368806a5b0 2024-04-21T09:14:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.211+0000 7f3684ff9640 1 -- 172.21.15.66:0/3912658081 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f36780037a0 con 0x7f368806a5b0 2024-04-21T09:14:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 Processor -- start 2024-04-21T09:14:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 -- start start 2024-04-21T09:14:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30babd0 0x55dff30c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dff301f1e0 con 0x55dff30ba2b0 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dff2fbc390 con 0x55dff30babd0 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3690407740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dff301b1a0 con 0x55dff30c7e90 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f36877fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3684ff9640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f36740a3d20 0x7f36740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f36877fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30ba690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60646/0 (socket says 172.21.15.66:60646) 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f36877fe640 1 -- 172.21.15.66:0/657569511 learned_addr learned my addr 172.21.15.66:0/657569511 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f3686ffd640 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30babd0 0x55dff30c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.212+0000 7f36877fe640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f36740a3d20 0x7f36740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f3684ff9640 1 -- 172.21.15.66:0/3912658081 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f36780d0f70 con 0x7f368806a5b0 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f367ffff640 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f3686ffd640 1 -- 172.21.15.66:0/657569511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 msgr2=0x55dff30ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f3686ffd640 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f3686ffd640 1 -- 172.21.15.66:0/657569511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 msgr2=0x55dff30ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f3686ffd640 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30ba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f3686ffd640 1 -- 172.21.15.66:0/657569511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff2dfb7d0 con 0x55dff30babd0 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f36877fe640 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f36877fe640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f36740a3d20 0x7f36740a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f368000edc0 tx=0x7f3680009b10 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f3686ffd640 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30babd0 0x55dff30c7950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36780ce350 tx=0x7f367801b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f722e195c988c6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f367d7fa640 1 -- 172.21.15.66:0/657569511 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36780071c0 con 0x55dff30babd0 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f367d7fa640 1 -- 172.21.15.66:0/657569511 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367809cba0 con 0x55dff30babd0 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.213+0000 7f367d7fa640 1 -- 172.21.15.66:0/657569511 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367800ad40 con 0x55dff30babd0 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 -- 172.21.15.66:0/657569511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30babd0 msgr2=0x55dff30c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30babd0 0x55dff30c7950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36780ce350 tx=0x7f367801b6d0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 -- 172.21.15.66:0/657569511 shutdown_connections 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30babd0 0x55dff30c7950 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 --2- 172.21.15.66:0/657569511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 -- 172.21.15.66:0/657569511 >> 172.21.15.66:0/657569511 conn(0x55dff30b4cb0 msgr2=0x55dff30b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 -- 172.21.15.66:0/657569511 shutdown_connections 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 -- 172.21.15.66:0/657569511 wait complete. 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 -- start start 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.214+0000 7f3690407740 1 Processor -- start 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dff30cabf0 con 0x55dff30ba2b0 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dff2fd82f0 con 0x55dff30c7e90 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dff30cad70 con 0x55dff30babd0 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 10 client.0 _ll_register_callbacks cb 0x55dff2fe5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60656/0 (socket says 172.21.15.66:60656) 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 -- 172.21.15.66:0/911257947 learned_addr learned my addr 172.21.15.66:0/911257947 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f36877fe640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f367ffff640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 -- 172.21.15.66:0/911257947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 msgr2=0x55dff30c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 -- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 msgr2=0x55dff30c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 -- 172.21.15.66:0/911257947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff2d313f0 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f367ffff640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3686ffd640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30c4ed0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f367800c760 tx=0x7f367800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982164801016f497 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f366e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f366e7fc640 1 -- 172.21.15.66:0/911257947 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36780b6060 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f366e7fc640 1 -- 172.21.15.66:0/911257947 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f36780bd070 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.215+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367809cba0 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f366e7fc640 1 -- 172.21.15.66:0/911257947 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36780a5810 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 10 client.17415 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dff2cf10f0 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f366e7fc640 1 -- 172.21.15.66:0/911257947 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367809cba0 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f366e7fc640 1 -- 172.21.15.66:0/911257947 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36780988f0 con 0x55dff30ba2b0 2024-04-21T09:14:46.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 10 client.17415 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 20 client.17415 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff30d7ce0 0x55dff30da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 4 client.17415 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff302cc00 con 0x55dff30d7ce0 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.216+0000 7f367ffff640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff30d7ce0 0x55dff30da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f366e7fc640 10 client.17415 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f367ffff640 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff30d7ce0 0x55dff30da0d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8e7f38e04ba488 server_cookie=e132bdc38fc99e25 in_seq=0 out_seq=0 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f366e7fc640 1 -- 172.21.15.66:0/911257947 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dff302cc00 con 0x55dff30d7ce0 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f366e7fc640 10 client.17415 handle_command_reply: tid=0 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 client.17415 shutdown 2024-04-21T09:14:46.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 msgr2=0x55dff30c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30c4ed0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f367800c760 tx=0x7f367800ef90 comp rx=0 tx=0).stop 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 shutdown_connections 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff30d7ce0 0x55dff30da0d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 --2- 172.21.15.66:0/911257947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30c4ed0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f366e7fc640 0 client.17415 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.217+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 >> 172.21.15.66:0/911257947 conn(0x55dff30b4cb0 msgr2=0x55dff30b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.218+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 shutdown_connections 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.218+0000 7f3690407740 1 -- 172.21.15.66:0/911257947 wait complete. 2024-04-21T09:14:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.218+0000 7f3690407740 20 client.17415 trim_cache size 0 max 16384 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 Processor -- start 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 -- start start 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30ba2b0 0x55dff30c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30babd0 0x55dff30d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36881afb40 con 0x55dff30babd0 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ad410 con 0x55dff30ba2b0 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3690407740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ab240 con 0x55dff30c7e90 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f367ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30babd0 0x55dff30d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f367ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30babd0 0x55dff30d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60664/0 (socket says 172.21.15.66:60664) 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f367ffff640 1 -- 172.21.15.66:0/3857893159 learned_addr learned my addr 172.21.15.66:0/3857893159 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f36877fe640 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.235+0000 7f3686ffd640 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30ba2b0 0x55dff30c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367ffff640 1 -- 172.21.15.66:0/3857893159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 msgr2=0x55dff30dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367ffff640 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30dd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367ffff640 1 -- 172.21.15.66:0/3857893159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30ba2b0 msgr2=0x55dff30c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367ffff640 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30ba2b0 0x55dff30c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367ffff640 1 -- 172.21.15.66:0/3857893159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff30c3420 con 0x55dff30babd0 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f36877fe640 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367ffff640 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30babd0 0x55dff30d3710 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3670009230 tx=0x7f3670000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95387361855ea34a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367cff9640 1 -- 172.21.15.66:0/3857893159 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367004a070 con 0x55dff30babd0 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367cff9640 1 -- 172.21.15.66:0/3857893159 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367002cc90 con 0x55dff30babd0 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.236+0000 7f367cff9640 1 -- 172.21.15.66:0/3857893159 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36700046c0 con 0x55dff30babd0 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- 172.21.15.66:0/3857893159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30babd0 msgr2=0x55dff30d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30babd0 0x55dff30d3710 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3670009230 tx=0x7f3670000bc0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- 172.21.15.66:0/3857893159 shutdown_connections 2024-04-21T09:14:46.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30c7e90 0x55dff30dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30babd0 0x55dff30d3710 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 --2- 172.21.15.66:0/3857893159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30ba2b0 0x55dff30c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- 172.21.15.66:0/3857893159 >> 172.21.15.66:0/3857893159 conn(0x55dff30b4cb0 msgr2=0x55dff30d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- 172.21.15.66:0/3857893159 shutdown_connections 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- 172.21.15.66:0/3857893159 wait complete. 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- start start 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 Processor -- start 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ab8d0 con 0x55dff30ba2b0 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f36881acd40 con 0x55dff30c7e90 2024-04-21T09:14:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f36881abfa0 con 0x55dff30babd0 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3690407740 10 client.0 _ll_register_callbacks cb 0x7f368000e440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30daad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60674/0 (socket says 172.21.15.66:60674) 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f3686ffd640 1 -- 172.21.15.66:0/1766017195 learned_addr learned my addr 172.21.15.66:0/1766017195 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.237+0000 7f36877fe640 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3686ffd640 1 -- 172.21.15.66:0/1766017195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 msgr2=0x55dff30db010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3686ffd640 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30db010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3686ffd640 1 -- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 msgr2=0x55dff30db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3686ffd640 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3686ffd640 1 -- 172.21.15.66:0/1766017195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff313cec0 con 0x55dff30ba2b0 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f36877fe640 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:14:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3686ffd640 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30daad0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3678009230 tx=0x7f3678004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a812f54b49030245 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f366e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f366e7fc640 1 -- 172.21.15.66:0/1766017195 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367809c500 con 0x55dff30ba2b0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f366e7fc640 1 -- 172.21.15.66:0/1766017195 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f3678097330 con 0x55dff30ba2b0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367002cc90 con 0x55dff30ba2b0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f366e7fc640 1 -- 172.21.15.66:0/1766017195 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3678098e40 con 0x55dff30ba2b0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 10 client.17421 fetch_fsmap learned FSMap version 25 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dff313e150 con 0x55dff30ba2b0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f366e7fc640 1 -- 172.21.15.66:0/1766017195 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367002cc90 con 0x55dff30ba2b0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f366e7fc640 1 -- 172.21.15.66:0/1766017195 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3678099210 con 0x55dff30ba2b0 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 10 client.17421 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 20 client.17421 populate_metadata read hostname 'smithi066' 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff313e740 0x55dff3140b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:14:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 4 client.17421 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.238+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dff3026a70 con 0x55dff313e740 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f367ffff640 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff313e740 0x55dff3140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f366e7fc640 10 client.17421 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f367ffff640 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff313e740 0x55dff3140b80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8552359d9b61049 server_cookie=705e205b25c5a6cf in_seq=0 out_seq=0 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f366e7fc640 1 -- 172.21.15.66:0/1766017195 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dff3026a70 con 0x55dff313e740 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f366e7fc640 10 client.17421 handle_command_reply: tid=0 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 client.17421 shutdown 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 msgr2=0x55dff30daad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30daad0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3678009230 tx=0x7f3678004210 comp rx=0 tx=0).stop 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 shutdown_connections 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dff313e740 0x55dff3140b80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dff30c7e90 0x55dff30db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dff30babd0 0x55dff30db010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 --2- 172.21.15.66:0/1766017195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff30ba2b0 0x55dff30daad0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f366e7fc640 0 client.17421 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:14:46.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 >> 172.21.15.66:0/1766017195 conn(0x55dff30b4cb0 msgr2=0x55dff30b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 shutdown_connections 2024-04-21T09:14:46.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 1 -- 172.21.15.66:0/1766017195 wait complete. 2024-04-21T09:14:46.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.239+0000 7f3690407740 20 client.17421 trim_cache size 0 max 16384 2024-04-21T09:14:46.247 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:14:46.247 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout: "4c8c6a03-0158-41ba-9e14-e0efa9a478bb": { 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4c8c6a03-0158-41ba-9e14-e0efa9a478bb", 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:14:46.248 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f36740a3d20 msgr2=0x7f36740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f36740a3d20 0x7f36740a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f368000edc0 tx=0x7f3680009b10 comp rx=0 tx=0).stop 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 msgr2=0x7f368819e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368819e430 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f367800ea60 tx=0x7f367800ef20 comp rx=0 tx=0).stop 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 shutdown_connections 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f36740a3d20 0x7f36740a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3688108060 0x7f36881985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f368806af70 0x7f368819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 --2- 172.21.15.66:0/3912658081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806a5b0 0x7f368819e430 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:14:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 >> 172.21.15.66:0/3912658081 conn(0x7f3688077c40 msgr2=0x7f36880ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:14:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 shutdown_connections 2024-04-21T09:14:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:14:46.240+0000 7f368d73d640 1 -- 172.21.15.66:0/3912658081 wait complete. 2024-04-21T09:14:46.251 INFO:tasks.cephfs.filesystem:scrub status for tag:4c8c6a03-0158-41ba-9e14-e0efa9a478bb - {'path': '/', 'tag': '4c8c6a03-0158-41ba-9e14-e0efa9a478bb', 'options': 'recursive,force'} 2024-04-21T09:14:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:47 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:14:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:47 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:47 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:14:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:47 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:47 smithi186 ceph-mon[31505]: pgmap v200: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:47.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:47 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:14:47.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:47 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:47.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:47 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:14:47.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:47 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:47.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:47 smithi083 ceph-mon[31413]: pgmap v200: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:47 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:14:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:47 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:47 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:14:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:47 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:14:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:47 smithi066 ceph-mon[27953]: pgmap v200: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:14:49.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:49 smithi186 ceph-mon[31505]: pgmap v201: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:49.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:49 smithi083 ceph-mon[31413]: pgmap v201: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:49 smithi066 ceph-mon[27953]: pgmap v201: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:51 smithi186 ceph-mon[31505]: pgmap v202: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:51 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:14:51.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:51 smithi083 ceph-mon[31413]: pgmap v202: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:51.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:51 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:14:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:51 smithi066 ceph-mon[27953]: pgmap v202: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:51 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:14:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:53 smithi186 ceph-mon[31505]: pgmap v203: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:53.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:53 smithi083 ceph-mon[31413]: pgmap v203: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:53 smithi066 ceph-mon[27953]: pgmap v203: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:55 smithi186 ceph-mon[31505]: pgmap v204: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:55 smithi083 ceph-mon[31413]: pgmap v204: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:55.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:55 smithi066 ceph-mon[27953]: pgmap v204: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:57 smithi186 ceph-mon[31505]: pgmap v205: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:57.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:57 smithi083 ceph-mon[31413]: pgmap v205: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:57.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:57 smithi066 ceph-mon[27953]: pgmap v205: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:14:59.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:14:59 smithi186 ceph-mon[31505]: pgmap v206: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:59.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:14:59 smithi083 ceph-mon[31413]: pgmap v206: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:14:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:14:59 smithi066 ceph-mon[27953]: pgmap v206: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:01.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:01 smithi186 ceph-mon[31505]: pgmap v207: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:01.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:01 smithi083 ceph-mon[31413]: pgmap v207: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:01 smithi066 ceph-mon[27953]: pgmap v207: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:03 smithi083 ceph-mon[31413]: pgmap v208: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:03 smithi066 ceph-mon[27953]: pgmap v208: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:03.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:03 smithi186 ceph-mon[31505]: pgmap v208: 129 pgs: 129 active+clean; 259 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:05.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:05 smithi083 ceph-mon[31413]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:15:05.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:05 smithi066 ceph-mon[27953]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:15:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:05 smithi186 ceph-mon[31505]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:15:07.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:07 smithi083 ceph-mon[31413]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:15:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:07 smithi066 ceph-mon[27953]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:15:07.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:07 smithi186 ceph-mon[31505]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:15:09.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:09 smithi083 ceph-mon[31413]: pgmap v211: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:15:09.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:09 smithi066 ceph-mon[27953]: pgmap v211: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:15:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:09 smithi186 ceph-mon[31505]: pgmap v211: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:15:11.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:11 smithi083 ceph-mon[31413]: pgmap v212: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:15:11.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:11 smithi066 ceph-mon[27953]: pgmap v212: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:15:11.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:11 smithi186 ceph-mon[31505]: pgmap v212: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:15:12.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:12 smithi066 ceph-mon[27953]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T09:15:12.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:12 smithi186 ceph-mon[31505]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T09:15:12.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:12 smithi083 ceph-mon[31413]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T09:15:13.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:13 smithi066 ceph-mon[27953]: pgmap v214: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:15:13.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:13 smithi186 ceph-mon[31505]: pgmap v214: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:15:13.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:13 smithi083 ceph-mon[31413]: pgmap v214: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:15:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:15 smithi186 ceph-mon[31505]: pgmap v215: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T09:15:15.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:15 smithi186 ceph-mon[31505]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T09:15:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:15 smithi083 ceph-mon[31413]: pgmap v215: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T09:15:15.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:15 smithi083 ceph-mon[31413]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T09:15:15.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:15 smithi066 ceph-mon[27953]: pgmap v215: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T09:15:15.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:15 smithi066 ceph-mon[27953]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T09:15:16.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:15:16.252 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:15:16.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 -- 172.21.15.66:0/1157428222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8106150 msgr2=0x7f78d810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 --2- 172.21.15.66:0/1157428222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8106150 0x7f78d810dd20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009ca0 tx=0x7f78d002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 -- 172.21.15.66:0/1157428222 shutdown_connections 2024-04-21T09:15:16.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 --2- 172.21.15.66:0/1157428222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8106150 0x7f78d810dd20 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 --2- 172.21.15.66:0/1157428222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78d8101a30 0x7f78d8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 --2- 172.21.15.66:0/1157428222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d8101080 0x7f78d8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 -- 172.21.15.66:0/1157428222 >> 172.21.15.66:0/1157428222 conn(0x7f78d8078000 msgr2=0x7f78d8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 -- 172.21.15.66:0/1157428222 shutdown_connections 2024-04-21T09:15:16.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 -- 172.21.15.66:0/1157428222 wait complete. 2024-04-21T09:15:16.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.369+0000 7f78de196640 1 Processor -- start 2024-04-21T09:15:16.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78de196640 1 -- start start 2024-04-21T09:15:16.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78de196640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8101080 0x7f78d8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78de196640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d8106150 0x7f78d8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78de196640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78d819bec0 0x7f78d819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78de196640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81110c0 con 0x7f78d8101080 2024-04-21T09:15:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78de196640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8110f40 con 0x7f78d8106150 2024-04-21T09:15:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78de196640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8111240 con 0x7f78d819bec0 2024-04-21T09:15:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8101080 0x7f78d8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8101080 0x7f78d8197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43528/0 (socket says 172.21.15.66:43528) 2024-04-21T09:15:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78d77fe640 1 -- 172.21.15.66:0/1817477959 learned_addr learned my addr 172.21.15.66:0/1817477959 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d8106150 0x7f78d8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.370+0000 7f78d7fff640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78d819bec0 0x7f78d819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d77fe640 1 -- 172.21.15.66:0/1817477959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78d819bec0 msgr2=0x7f78d819c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d77fe640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78d819bec0 0x7f78d819c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d77fe640 1 -- 172.21.15.66:0/1817477959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d8106150 msgr2=0x7f78d8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d77fe640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d8106150 0x7f78d8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d77fe640 1 -- 172.21.15.66:0/1817477959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d819c9b0 con 0x7f78d8101080 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d77fe640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8101080 0x7f78d8197600 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d950 tx=0x7f78c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d4ff9640 1 -- 172.21.15.66:0/1817477959 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8017030 con 0x7f78d8101080 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d4ff9640 1 -- 172.21.15.66:0/1817477959 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c8004020 con 0x7f78d8101080 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d819cca0 con 0x7f78d8101080 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78d4ff9640 1 -- 172.21.15.66:0/1817477959 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c80047a0 con 0x7f78d8101080 2024-04-21T09:15:16.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.371+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81a4490 con 0x7f78d8101080 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.374+0000 7f78e0d87740 1 Processor -- start 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.374+0000 7f78e0d87740 1 -- start start 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4a580 0x55847bd4a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4aea0 0x55847bd53540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd55e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78e0d87740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55847bcaf1e0 con 0x55847bd4a580 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78e0d87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55847bc4c390 con 0x55847bd53a80 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78e0d87740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55847bcab1a0 con 0x55847bd4aea0 2024-04-21T09:15:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4a580 0x55847bd4a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4a580 0x55847bd4a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43538/0 (socket says 172.21.15.66:43538) 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 -- 172.21.15.66:0/3342184916 learned_addr learned my addr 172.21.15.66:0/3342184916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd55e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d77fe640 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4aea0 0x55847bd53540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 -- 172.21.15.66:0/3342184916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4aea0 msgr2=0x55847bd53540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4aea0 0x55847bd53540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 -- 172.21.15.66:0/3342184916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 msgr2=0x55847bd55e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd55e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.375+0000 7f78d7fff640 1 -- 172.21.15.66:0/3342184916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847ba8b7d0 con 0x55847bd4a580 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.376+0000 7f78d7fff640 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4a580 0x55847bd4a960 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009ca0 tx=0x7f78d00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4ee62978998a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.376+0000 7f78bdffb640 1 -- 172.21.15.66:0/3342184916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0043020 con 0x55847bd4a580 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.376+0000 7f78bdffb640 1 -- 172.21.15.66:0/3342184916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d0032970 con 0x55847bd4a580 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.376+0000 7f78bdffb640 1 -- 172.21.15.66:0/3342184916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0039040 con 0x55847bd4a580 2024-04-21T09:15:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.376+0000 7f78e0d87740 1 -- 172.21.15.66:0/3342184916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4a580 msgr2=0x55847bd4a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.376+0000 7f78e0d87740 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4a580 0x55847bd4a960 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009ca0 tx=0x7f78d00093f0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 -- 172.21.15.66:0/3342184916 shutdown_connections 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd55e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4aea0 0x55847bd53540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 --2- 172.21.15.66:0/3342184916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4a580 0x55847bd4a960 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 -- 172.21.15.66:0/3342184916 >> 172.21.15.66:0/3342184916 conn(0x55847bd44cb0 msgr2=0x55847bd45090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78d4ff9640 1 -- 172.21.15.66:0/1817477959 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f78c8009a30 con 0x7f78d8101080 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 -- 172.21.15.66:0/3342184916 shutdown_connections 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 -- 172.21.15.66:0/3342184916 wait complete. 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.377+0000 7f78e0d87740 1 -- start start 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 1 Processor -- start 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d4ff9640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f78c00a3c30 0x7f78c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd56c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4aea0 0x55847bd571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd57700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55847bd56810 con 0x55847bd4aea0 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55847bc682f0 con 0x55847bd53a80 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55847bd56990 con 0x55847bd4a580 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78e0d87740 10 client.0 _ll_register_callbacks cb 0x55847bc75580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d7fff640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f78c00a3c30 0x7f78c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d4ff9640 1 -- 172.21.15.66:0/1817477959 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f78c80d0c80 con 0x7f78d8101080 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4aea0 0x55847bd571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4aea0 0x55847bd571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43550/0 (socket says 172.21.15.66:43550) 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d6ffd640 1 -- 172.21.15.66:0/2721159801 learned_addr learned my addr 172.21.15.66:0/2721159801 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d7fff640 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd57700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.378+0000 7f78d77fe640 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd56c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78d6ffd640 1 -- 172.21.15.66:0/2721159801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 msgr2=0x55847bd56c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd56c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78d6ffd640 1 -- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 msgr2=0x55847bd57700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd57700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78d6ffd640 1 -- 172.21.15.66:0/2721159801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847b9c13f0 con 0x55847bd4aea0 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78d7fff640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f78c00a3c30 0x7f78c00a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009a60 tx=0x7f78d0032fc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4aea0 0x55847bd571c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00b920 tx=0x7f78cc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738f1d4123587691 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2721159801 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc005020 con 0x55847bd4aea0 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2721159801 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f78cc0051c0 con 0x55847bd4aea0 2024-04-21T09:15:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d0032970 con 0x55847bd4aea0 2024-04-21T09:15:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2721159801 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78cc024050 con 0x55847bd4aea0 2024-04-21T09:15:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78e0d87740 10 client.17433 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55847b9810f0 con 0x55847bd4aea0 2024-04-21T09:15:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.379+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2721159801 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d0032970 con 0x55847bd4aea0 2024-04-21T09:15:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2721159801 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78cc01b470 con 0x55847bd4aea0 2024-04-21T09:15:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78e0d87740 10 client.17433 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78e0d87740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78e0d87740 20 client.17433 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd65cf0 0x55847bd68130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78e0d87740 4 client.17433 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55847bcbcc00 con 0x55847bd65cf0 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78d77fe640 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd65cf0 0x55847bd68130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78b6ffd640 10 client.17433 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.380+0000 7f78d77fe640 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd65cf0 0x55847bd68130 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd127137d747d571 server_cookie=454697d5261b303a in_seq=0 out_seq=0 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2721159801 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55847bcbcc00 con 0x55847bd65cf0 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78b6ffd640 10 client.17433 handle_command_reply: tid=0 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 client.17433 shutdown 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4aea0 msgr2=0x55847bd571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4aea0 0x55847bd571c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00b920 tx=0x7f78cc00bde0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 shutdown_connections 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd65cf0 0x55847bd68130 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd53a80 0x55847bd57700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd4aea0 0x55847bd571c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78b6ffd640 0 client.17433 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2721159801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd56c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 >> 172.21.15.66:0/2721159801 conn(0x55847bd44cb0 msgr2=0x55847bd46a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.381+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 shutdown_connections 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.382+0000 7f78e0d87740 1 -- 172.21.15.66:0/2721159801 wait complete. 2024-04-21T09:15:16.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.382+0000 7f78e0d87740 20 client.17433 trim_cache size 0 max 16384 2024-04-21T09:15:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 Processor -- start 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 -- start start 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bd6da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81afac0 con 0x55847bd44cb0 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81ad390 con 0x55847bd4aea0 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.399+0000 7f78e0d87740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81ab200 con 0x55847bd4a580 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd45090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43560/0 (socket says 172.21.15.66:43560) 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 -- 172.21.15.66:0/4057756779 learned_addr learned my addr 172.21.15.66:0/4057756779 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d7fff640 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bd6da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 -- 172.21.15.66:0/4057756779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 msgr2=0x55847bd57310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd57310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 -- 172.21.15.66:0/4057756779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 msgr2=0x55847bd6da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bd6da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 -- 172.21.15.66:0/4057756779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847bd49900 con 0x55847bd44cb0 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d7fff640 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bd6da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78d77fe640 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd45090 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f78c809c7e0 tx=0x7f78c80984e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eac4926a73c8bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78bd7fa640 1 -- 172.21.15.66:0/4057756779 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c809cf50 con 0x55847bd44cb0 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78bd7fa640 1 -- 172.21.15.66:0/4057756779 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c806df70 con 0x55847bd44cb0 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.400+0000 7f78bd7fa640 1 -- 172.21.15.66:0/4057756779 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c80041d0 con 0x55847bd44cb0 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 -- 172.21.15.66:0/4057756779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 msgr2=0x55847bd45090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd45090 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f78c809c7e0 tx=0x7f78c80984e0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 -- 172.21.15.66:0/4057756779 shutdown_connections 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bd6da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 --2- 172.21.15.66:0/4057756779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd45090 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 -- 172.21.15.66:0/4057756779 >> 172.21.15.66:0/4057756779 conn(0x55847bd4c480 msgr2=0x55847bd56c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 -- 172.21.15.66:0/4057756779 shutdown_connections 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 -- 172.21.15.66:0/4057756779 wait complete. 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.401+0000 7f78e0d87740 1 -- start start 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 1 Processor -- start 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd5aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd5b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bdc5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81ab850 con 0x55847bd44cb0 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81accc0 con 0x55847bd4aea0 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81abf20 con 0x55847bd4a580 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78e0d87740 10 client.0 _ll_register_callbacks cb 0x7f78cc004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd5aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd5aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43568/0 (socket says 172.21.15.66:43568) 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 -- 172.21.15.66:0/2926921319 learned_addr learned my addr 172.21.15.66:0/2926921319 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd5b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 -- 172.21.15.66:0/2926921319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 msgr2=0x55847bd5b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd5b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d7fff640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bdc5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 -- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 msgr2=0x55847bdc5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bdc5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 -- 172.21.15.66:0/2926921319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847bdc5810 con 0x55847bd44cb0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd5b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78d77fe640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd5aef0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f78c80ce0b0 tx=0x7f78c80a0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7287bdb41151afc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.402+0000 7f78b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2926921319 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c809cf50 con 0x55847bd44cb0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78c806df70 con 0x55847bd44cb0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2926921319 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f78c80041d0 con 0x55847bd44cb0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2926921319 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78c80ce280 con 0x55847bd44cb0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 10 client.17439 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55847bdccb20 con 0x55847bd44cb0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2926921319 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c806df70 con 0x55847bd44cb0 2024-04-21T09:15:16.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2926921319 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78c80b2070 con 0x55847bd44cb0 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 10 client.17439 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 20 client.17439 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd53a80 0x55847bdcf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 4 client.17439 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55847bcb6a70 con 0x55847bd53a80 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.403+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd53a80 0x55847bdcf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78b6ffd640 10 client.17439 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78d6ffd640 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd53a80 0x55847bdcf180 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821486c41ae791a1 server_cookie=85e48395a365de99 in_seq=0 out_seq=0 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78b6ffd640 1 -- 172.21.15.66:0/2926921319 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55847bcb6a70 con 0x55847bd53a80 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78b6ffd640 10 client.17439 handle_command_reply: tid=0 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 client.17439 shutdown 2024-04-21T09:15:16.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 msgr2=0x55847bd5aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd5aef0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f78c80ce0b0 tx=0x7f78c80a0c00 comp rx=0 tx=0).stop 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 shutdown_connections 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55847bd53a80 0x55847bdcf180 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55847bd4aea0 0x55847bdc5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78b6ffd640 0 client.17439 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55847bd4a580 0x55847bd5b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 --2- 172.21.15.66:0/2926921319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55847bd44cb0 0x55847bd5aef0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 >> 172.21.15.66:0/2926921319 conn(0x55847bd4c480 msgr2=0x55847bd43d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 shutdown_connections 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 1 -- 172.21.15.66:0/2926921319 wait complete. 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.404+0000 7f78e0d87740 20 client.17439 trim_cache size 0 max 16384 2024-04-21T09:15:16.411 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:15:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:15:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:15:16.412 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f78c00a3c30 msgr2=0x7f78c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f78c00a3c30 0x7f78c00a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009a60 tx=0x7f78d0032fc0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8101080 msgr2=0x7f78d8197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8101080 0x7f78d8197600 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d950 tx=0x7f78c800de10 comp rx=0 tx=0).stop 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 shutdown_connections 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f78c00a3c30 0x7f78c00a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78d819bec0 0x7f78d819c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d8106150 0x7f78d8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 --2- 172.21.15.66:0/1817477959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78d8101080 0x7f78d8197600 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 >> 172.21.15.66:0/1817477959 conn(0x7f78d8078000 msgr2=0x7f78d80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 shutdown_connections 2024-04-21T09:15:16.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.405+0000 7f78de196640 1 -- 172.21.15.66:0/1817477959 wait complete. 2024-04-21T09:15:16.415 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:15:16.416 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:15:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 -- 172.21.15.66:0/3412487299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102b40 msgr2=0x7f7e48106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 --2- 172.21.15.66:0/3412487299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102b40 0x7f7e48106b80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009900 tx=0x7f7e3802bc30 comp rx=0 tx=0).stop 2024-04-21T09:15:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 -- 172.21.15.66:0/3412487299 shutdown_connections 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 --2- 172.21.15.66:0/3412487299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e48107260 0x7f7e4810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 --2- 172.21.15.66:0/3412487299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102b40 0x7f7e48106b80 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 --2- 172.21.15.66:0/3412487299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e48102190 0x7f7e48102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 -- 172.21.15.66:0/3412487299 >> 172.21.15.66:0/3412487299 conn(0x7f7e480793e0 msgr2=0x7f7e480797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 -- 172.21.15.66:0/3412487299 shutdown_connections 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.518+0000 7f7e4e626640 1 -- 172.21.15.66:0/3412487299 wait complete. 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 Processor -- start 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 -- start start 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102190 0x7f7e48198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e48107260 0x7f7e48198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e4819d020 0x7f7e4819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481121a0 con 0x7f7e48102190 2024-04-21T09:15:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48112020 con 0x7f7e48107260 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e4e626640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48112320 con 0x7f7e4819d020 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e47fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102190 0x7f7e48198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e47fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102190 0x7f7e48198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43590/0 (socket says 172.21.15.66:43590) 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.519+0000 7f7e47fff640 1 -- 172.21.15.66:0/2201025570 learned_addr learned my addr 172.21.15.66:0/2201025570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e477fe640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e48107260 0x7f7e48198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e4cbc3640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e4819d020 0x7f7e4819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e47fff640 1 -- 172.21.15.66:0/2201025570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e4819d020 msgr2=0x7f7e4819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e47fff640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e4819d020 0x7f7e4819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e47fff640 1 -- 172.21.15.66:0/2201025570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e48107260 msgr2=0x7f7e48198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e47fff640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e48107260 0x7f7e48198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e47fff640 1 -- 172.21.15.66:0/2201025570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4819db10 con 0x7f7e48102190 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e4cbc3640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e4819d020 0x7f7e4819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e47fff640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102190 0x7f7e48198760 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000d770 tx=0x7f7e3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e457fa640 1 -- 172.21.15.66:0/2201025570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30017070 con 0x7f7e48102190 2024-04-21T09:15:16.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4819de00 con 0x7f7e48102190 2024-04-21T09:15:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e457fa640 1 -- 172.21.15.66:0/2201025570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e300054d0 con 0x7f7e48102190 2024-04-21T09:15:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e457fa640 1 -- 172.21.15.66:0/2201025570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30004c30 con 0x7f7e48102190 2024-04-21T09:15:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.520+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481a55c0 con 0x7f7e48102190 2024-04-21T09:15:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.521+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e48069230 con 0x7f7e48102190 2024-04-21T09:15:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.522+0000 7f7e457fa640 1 -- 172.21.15.66:0/2201025570 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7e30004dd0 con 0x7f7e48102190 2024-04-21T09:15:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.523+0000 7f7e457fa640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7e340a3d20 0x7f7e340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.523+0000 7f7e477fe640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7e340a3d20 0x7f7e340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.523+0000 7f7e457fa640 1 -- 172.21.15.66:0/2201025570 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f7e300d0ae0 con 0x7f7e48102190 2024-04-21T09:15:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.523+0000 7f7e477fe640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7e340a3d20 0x7f7e340a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009230 tx=0x7f7e38005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.527+0000 7f7e457fa640 1 -- 172.21.15.66:0/2201025570 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e3009cdb0 con 0x7f7e48102190 2024-04-21T09:15:16.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.754+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e4810a4c0 con 0x7f7e48102190 2024-04-21T09:15:16.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.755+0000 7f7e457fa640 1 -- 172.21.15.66:0/2201025570 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f7e3009c6b0 con 0x7f7e48102190 2024-04-21T09:15:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:15:16.755 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:15:16.755 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7e340a3d20 msgr2=0x7f7e340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7e340a3d20 0x7f7e340a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009230 tx=0x7f7e38005fd0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102190 msgr2=0x7f7e48198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102190 0x7f7e48198760 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000d770 tx=0x7f7e3000dc30 comp rx=0 tx=0).stop 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 shutdown_connections 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f7e340a3d20 0x7f7e340a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e4819d020 0x7f7e4819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e48107260 0x7f7e48198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 --2- 172.21.15.66:0/2201025570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e48102190 0x7f7e48198760 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 >> 172.21.15.66:0/2201025570 conn(0x7f7e480793e0 msgr2=0x7f7e480ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 shutdown_connections 2024-04-21T09:15:16.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.757+0000 7f7e4e626640 1 -- 172.21.15.66:0/2201025570 wait complete. 2024-04-21T09:15:16.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:15:16.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 -- 172.21.15.66:0/2154251835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d006a840 msgr2=0x7f76d006ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 --2- 172.21.15.66:0/2154251835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d006a840 0x7f76d006ac20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c002bc30 comp rx=0 tx=0).stop 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 -- 172.21.15.66:0/2154251835 shutdown_connections 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 --2- 172.21.15.66:0/2154251835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d0107270 0x7f76d010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 --2- 172.21.15.66:0/2154251835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76d006b1f0 0x7f76d0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 --2- 172.21.15.66:0/2154251835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d006a840 0x7f76d006ac20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 -- 172.21.15.66:0/2154251835 >> 172.21.15.66:0/2154251835 conn(0x7f76d0077eb0 msgr2=0x7f76d00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 -- 172.21.15.66:0/2154251835 shutdown_connections 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.872+0000 7f76d679d640 1 -- 172.21.15.66:0/2154251835 wait complete. 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 Processor -- start 2024-04-21T09:15:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 -- start start 2024-04-21T09:15:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d006a840 0x7f76d0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d0107270 0x7f76d0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76d019cfd0 0x7f76d019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01121b0 con 0x7f76d0107270 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0112030 con 0x7f76d019cfd0 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76d679d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0112330 con 0x7f76d006a840 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.873+0000 7f76cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d0107270 0x7f76d0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d0107270 0x7f76d0198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43610/0 (socket says 172.21.15.66:43610) 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 -- 172.21.15.66:0/683680071 learned_addr learned my addr 172.21.15.66:0/683680071 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cffff640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d006a840 0x7f76d0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76d019cfd0 0x7f76d019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 -- 172.21.15.66:0/683680071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d006a840 msgr2=0x7f76d0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d006a840 0x7f76d0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 -- 172.21.15.66:0/683680071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76d019cfd0 msgr2=0x7f76d019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76d019cfd0 0x7f76d019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 -- 172.21.15.66:0/683680071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d019dac0 con 0x7f76d0107270 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76d019cfd0 0x7f76d019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d0107270 0x7f76d0198c50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005fd0 tx=0x7f76b800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cd7fa640 1 -- 172.21.15.66:0/683680071 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8017070 con 0x7f76d0107270 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cd7fa640 1 -- 172.21.15.66:0/683680071 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b8009c30 con 0x7f76d0107270 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d019ddb0 con 0x7f76d0107270 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76cd7fa640 1 -- 172.21.15.66:0/683680071 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8003d30 con 0x7f76d0107270 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.874+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d01a55a0 con 0x7f76d0107270 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.876+0000 7f76cd7fa640 1 -- 172.21.15.66:0/683680071 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f76b8003ed0 con 0x7f76d0107270 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 Processor -- start 2024-04-21T09:15:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 -- start start 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0288790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0291890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e02922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b5e01ed1f0 con 0x55b5e0288d60 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5e018a4b0 con 0x55b5e02883b0 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d938e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b5e01e91b0 con 0x55b5e0291e60 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cd7fa640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76bc0a3d20 0x7f76bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0291890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0291890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43624/0 (socket says 172.21.15.66:43624) 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 -- 172.21.15.66:0/919436529 learned_addr learned my addr 172.21.15.66:0/919436529 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0288790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cd7fa640 1 -- 172.21.15.66:0/683680071 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f76b80d0880 con 0x7f76d0107270 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e02922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76bc0a3d20 0x7f76bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 -- 172.21.15.66:0/919436529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 msgr2=0x55b5e02922c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e02922c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 -- 172.21.15.66:0/919436529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 msgr2=0x55b5e0288790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0288790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cffff640 1 -- 172.21.15.66:0/919436529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e017c1b0 con 0x55b5e0288d60 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0288790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.877+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e02922c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76cffff640 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0291890 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9470b8c69760df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76bc0a3d20 0x7f76bc0a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76c8005c40 tx=0x7f76c8009200 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76c67fc640 1 -- 172.21.15.66:0/919436529 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0045020 con 0x55b5e0288d60 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76c67fc640 1 -- 172.21.15.66:0/919436529 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c002c7e0 con 0x55b5e0288d60 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76c67fc640 1 -- 172.21.15.66:0/919436529 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c00046c0 con 0x55b5e0288d60 2024-04-21T09:15:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d938e740 1 -- 172.21.15.66:0/919436529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 msgr2=0x55b5e0291890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d938e740 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0291890 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c002c6c0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d938e740 1 -- 172.21.15.66:0/919436529 shutdown_connections 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d938e740 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e02922c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d938e740 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0291890 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d938e740 1 --2- 172.21.15.66:0/919436529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0288790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.878+0000 7f76d938e740 1 -- 172.21.15.66:0/919436529 >> 172.21.15.66:0/919436529 conn(0x55b5e0282ba0 msgr2=0x55b5e0282f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 -- 172.21.15.66:0/919436529 shutdown_connections 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 -- 172.21.15.66:0/919436529 wait complete. 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 -- start start 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 Processor -- start 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0298210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0288d60 0x55b5e02955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0291e60 0x55b5e0295ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b5e0298d40 con 0x55b5e0291e60 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5e01a6500 con 0x55b5e02883b0 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b5e0298ec0 con 0x55b5e0288d60 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d938e740 10 client.0 _ll_register_callbacks cb 0x55b5e0174b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d4d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0291e60 0x55b5e0295ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d4d3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0291e60 0x55b5e0295ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43638/0 (socket says 172.21.15.66:43638) 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.879+0000 7f76d4d3a640 1 -- 172.21.15.66:0/379835719 learned_addr learned my addr 172.21.15.66:0/379835719 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76cffff640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0298210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0288d60 0x55b5e02955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d4d3a640 1 -- 172.21.15.66:0/379835719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0288d60 msgr2=0x55b5e02955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0288d60 0x55b5e02955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d4d3a640 1 -- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 msgr2=0x55b5e0298210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0298210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d4d3a640 1 -- 172.21.15.66:0/379835719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5dfeff3f0 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0288d60 0x55b5e02955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0291e60 0x55b5e0295ae0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f76c8011920 tx=0x7f76c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477e069527cd7600 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76af7fe640 1 -- 172.21.15.66:0/379835719 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8024070 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76af7fe640 1 -- 172.21.15.66:0/379835719 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f76c800fd80 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c002c7e0 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76af7fe640 1 -- 172.21.15.66:0/379835719 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76c8030050 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d938e740 10 client.17457 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b5dffc97d0 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.880+0000 7f76af7fe640 1 -- 172.21.15.66:0/379835719 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c002c7e0 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76af7fe640 1 -- 172.21.15.66:0/379835719 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76c801f070 con 0x55b5e0291e60 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76d938e740 10 client.17457 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76d938e740 10 client.17457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:15:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76d938e740 20 client.17457 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76d938e740 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e02a5f10 0x55b5e02a8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76d938e740 4 client.17457 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5e018c930 con 0x55b5e02a5f10 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76cffff640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e02a5f10 0x55b5e02a8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76af7fe640 10 client.17457 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.881+0000 7f76cffff640 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e02a5f10 0x55b5e02a8360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b35c41fa0e70c56f server_cookie=783204add6dc27d5 in_seq=0 out_seq=0 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76af7fe640 1 -- 172.21.15.66:0/379835719 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5e018c930 con 0x55b5e02a5f10 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76af7fe640 10 client.17457 handle_command_reply: tid=0 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 client.17457 shutdown 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0291e60 msgr2=0x55b5e0295ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0291e60 0x55b5e0295ae0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f76c8011920 tx=0x7f76c8004a40 comp rx=0 tx=0).stop 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 shutdown_connections 2024-04-21T09:15:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e02a5f10 0x55b5e02a8360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0291e60 0x55b5e0295ae0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76af7fe640 0 client.17457 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0288d60 0x55b5e02955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 --2- 172.21.15.66:0/379835719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e02883b0 0x55b5e0298210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 >> 172.21.15.66:0/379835719 conn(0x55b5e0282ba0 msgr2=0x55b5e02848f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 shutdown_connections 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 1 -- 172.21.15.66:0/379835719 wait complete. 2024-04-21T09:15:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.882+0000 7f76d938e740 20 client.17457 trim_cache size 0 max 16384 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 Processor -- start 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 -- start start 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e02883b0 0x55b5e0293250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0288d60 0x55b5e02a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e0297cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01af940 con 0x55b5e02883b0 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ad210 con 0x55b5e0288d60 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76d938e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ab080 con 0x55b5e0291e60 2024-04-21T09:15:16.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e02883b0 0x55b5e0293250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e02883b0 0x55b5e0293250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43640/0 (socket says 172.21.15.66:43640) 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 -- 172.21.15.66:0/173026586 learned_addr learned my addr 172.21.15.66:0/173026586 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0288d60 0x55b5e02a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 -- 172.21.15.66:0/173026586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 msgr2=0x55b5e0297cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e0297cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 -- 172.21.15.66:0/173026586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0288d60 msgr2=0x55b5e02a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0288d60 0x55b5e02a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.896+0000 7f76cffff640 1 -- 172.21.15.66:0/173026586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e02a4b10 con 0x55b5e02883b0 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76cffff640 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e02883b0 0x55b5e0293250 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f76c0005c40 tx=0x7f76c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7fa3c2e3b2cf49 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76c5ffb640 1 -- 172.21.15.66:0/173026586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0045040 con 0x55b5e02883b0 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76c5ffb640 1 -- 172.21.15.66:0/173026586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c002c7e0 con 0x55b5e02883b0 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76c5ffb640 1 -- 172.21.15.66:0/173026586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0054680 con 0x55b5e02883b0 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 -- 172.21.15.66:0/173026586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e02883b0 msgr2=0x55b5e0293250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e02883b0 0x55b5e0293250 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f76c0005c40 tx=0x7f76c0000bc0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 -- 172.21.15.66:0/173026586 shutdown_connections 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e0291e60 0x55b5e0297cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0288d60 0x55b5e02a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 --2- 172.21.15.66:0/173026586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e02883b0 0x55b5e0293250 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 -- 172.21.15.66:0/173026586 >> 172.21.15.66:0/173026586 conn(0x55b5e0282ba0 msgr2=0x55b5e0295560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 -- 172.21.15.66:0/173026586 shutdown_connections 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.897+0000 7f76d938e740 1 -- 172.21.15.66:0/173026586 wait complete. 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 -- start start 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 Processor -- start 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e02883b0 0x55b5e0297400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0297940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0291e60 0x55b5e0297e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ab6d0 con 0x55b5e0288d60 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01acb40 con 0x55b5e0291e60 2024-04-21T09:15:16.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01abda0 con 0x55b5e02883b0 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d938e740 10 client.0 _ll_register_callbacks cb 0x7f76c8011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0297940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0297940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43656/0 (socket says 172.21.15.66:43656) 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 -- 172.21.15.66:0/1614104505 learned_addr learned my addr 172.21.15.66:0/1614104505 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76d4d3a640 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0291e60 0x55b5e0297e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 -- 172.21.15.66:0/1614104505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e02883b0 msgr2=0x55b5e0297400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cffff640 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e02883b0 0x55b5e0297400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e02883b0 0x55b5e0297400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 -- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0291e60 msgr2=0x55b5e0297e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0291e60 0x55b5e0297e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.898+0000 7f76cf7fe640 1 -- 172.21.15.66:0/1614104505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e030b6f0 con 0x55b5e0288d60 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76cf7fe640 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0297940 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f76b80ce200 tx=0x7f76b809cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19db7194c48e703f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:16.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76af7fe640 1 -- 172.21.15.66:0/1614104505 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b80b5060 con 0x55b5e0288d60 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76af7fe640 1 -- 172.21.15.66:0/1614104505 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f76b8098ac0 con 0x55b5e0288d60 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c002c7e0 con 0x55b5e0288d60 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76af7fe640 1 -- 172.21.15.66:0/1614104505 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76b809c450 con 0x55b5e0288d60 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 10 client.17463 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b5e030ca50 con 0x55b5e0288d60 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76af7fe640 1 -- 172.21.15.66:0/1614104505 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c002c7e0 con 0x55b5e0288d60 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76af7fe640 1 -- 172.21.15.66:0/1614104505 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76b80bb250 con 0x55b5e0288d60 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 10 client.17463 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 10 client.17463 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 20 client.17463 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e030d1a0 0x55b5e030f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 4 client.17463 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.899+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5e01fac10 con 0x55b5e030d1a0 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76cffff640 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e030d1a0 0x55b5e030f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:16.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76af7fe640 10 client.17463 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76cffff640 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e030d1a0 0x55b5e030f5c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1333a72ffb7c3939 server_cookie=52924f13bcc37314 in_seq=0 out_seq=0 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76af7fe640 1 -- 172.21.15.66:0/1614104505 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5e01fac10 con 0x55b5e030d1a0 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76af7fe640 10 client.17463 handle_command_reply: tid=0 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 client.17463 shutdown 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 msgr2=0x55b5e0297940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0297940 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f76b80ce200 tx=0x7f76b809cb50 comp rx=0 tx=0).stop 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 shutdown_connections 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55b5e030d1a0 0x55b5e030f5c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e0291e60 0x55b5e0297e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b5e0288d60 0x55b5e0297940 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76af7fe640 0 client.17463 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 --2- 172.21.15.66:0/1614104505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5e02883b0 0x55b5e0297400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 >> 172.21.15.66:0/1614104505 conn(0x55b5e0282ba0 msgr2=0x55b5e0281370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 shutdown_connections 2024-04-21T09:15:16.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.900+0000 7f76d938e740 1 -- 172.21.15.66:0/1614104505 wait complete. 2024-04-21T09:15:16.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d938e740 20 client.17463 trim_cache size 0 max 16384 2024-04-21T09:15:16.907 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76bc0a3d20 msgr2=0x7f76bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76bc0a3d20 0x7f76bc0a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76c8005c40 tx=0x7f76c8009200 comp rx=0 tx=0).stop 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d0107270 msgr2=0x7f76d0198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d0107270 0x7f76d0198c50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005fd0 tx=0x7f76b800dde0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 shutdown_connections 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76bc0a3d20 0x7f76bc0a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76d019cfd0 0x7f76d019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d0107270 0x7f76d0198c50 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 --2- 172.21.15.66:0/683680071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d006a840 0x7f76d0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 >> 172.21.15.66:0/683680071 conn(0x7f76d0077eb0 msgr2=0x7f76d0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 shutdown_connections 2024-04-21T09:15:16.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:16.901+0000 7f76d679d640 1 -- 172.21.15.66:0/683680071 wait complete. 2024-04-21T09:15:16.911 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 -- 172.21.15.66:0/1765946660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa484107270 msgr2=0x7fa48410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/1765946660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa484107270 0x7fa48410ee40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009ca0 tx=0x7fa47c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 -- 172.21.15.66:0/1765946660 shutdown_connections 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/1765946660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa484107270 0x7fa48410ee40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/1765946660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa48406b1f0 0x7fa484106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/1765946660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48406a840 0x7fa48406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 -- 172.21.15.66:0/1765946660 >> 172.21.15.66:0/1765946660 conn(0x7fa484077eb0 msgr2=0x7fa4840782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 -- 172.21.15.66:0/1765946660 shutdown_connections 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.015+0000 7fa48bd0d640 1 -- 172.21.15.66:0/1765946660 wait complete. 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 Processor -- start 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 -- start start 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48406a840 0x7fa4841986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa484107270 0x7fa484198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa48419cf40 0x7fa48419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841121b0 con 0x7fa48406a840 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa484112030 con 0x7fa484107270 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa48bd0d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa484112330 con 0x7fa48419cf40 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa489aa9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48406a840 0x7fa4841986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa489aa9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48406a840 0x7fa4841986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43676/0 (socket says 172.21.15.66:43676) 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.016+0000 7fa489aa9640 1 -- 172.21.15.66:0/923564620 learned_addr learned my addr 172.21.15.66:0/923564620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa489aa9640 1 -- 172.21.15.66:0/923564620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa48419cf40 msgr2=0x7fa48419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa4892a8640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa484107270 0x7fa484198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa489aa9640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa48419cf40 0x7fa48419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa489aa9640 1 -- 172.21.15.66:0/923564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa484107270 msgr2=0x7fa484198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa489aa9640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa484107270 0x7fa484198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa489aa9640 1 -- 172.21.15.66:0/923564620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48419da30 con 0x7fa48406a840 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa489aa9640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48406a840 0x7fa4841986d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e9f0 tx=0x7fa48000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa476ffd640 1 -- 172.21.15.66:0/923564620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48000cd80 con 0x7fa48406a840 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa476ffd640 1 -- 172.21.15.66:0/923564620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4800108e0 con 0x7fa48406a840 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48419dd20 con 0x7fa48406a840 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa476ffd640 1 -- 172.21.15.66:0/923564620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480018980 con 0x7fa48406a840 2024-04-21T09:15:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.017+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a5560 con 0x7fa48406a840 2024-04-21T09:15:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.019+0000 7fa476ffd640 1 -- 172.21.15.66:0/923564620 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa480010430 con 0x7fa48406a840 2024-04-21T09:15:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 Processor -- start 2024-04-21T09:15:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 -- start start 2024-04-21T09:15:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aa4d0 0x55a6044aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aae80 0x55a6044b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a60440f1f0 con 0x55a6044b3a00 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a6043ac4b0 con 0x55a6044aa4d0 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a60440b1b0 con 0x55a6044aae80 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa476ffd640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa4700a3da0 0x7fa4700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa4892a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa4892a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43684/0 (socket says 172.21.15.66:43684) 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa4892a8640 1 -- 172.21.15.66:0/1483494405 learned_addr learned my addr 172.21.15.66:0/1483494405 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aa4d0 0x55a6044aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa4700a3da0 0x7fa4700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.020+0000 7fa489aa9640 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aae80 0x55a6044b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa476ffd640 1 -- 172.21.15.66:0/923564620 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fa4800d0fc0 con 0x7fa48406a840 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa4892a8640 1 -- 172.21.15.66:0/1483494405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aae80 msgr2=0x55a6044b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa4892a8640 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aae80 0x55a6044b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa4892a8640 1 -- 172.21.15.66:0/1483494405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aa4d0 msgr2=0x55a6044aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa4892a8640 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aa4d0 0x55a6044aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa4892a8640 1 -- 172.21.15.66:0/1483494405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a60439e1b0 con 0x55a6044b3a00 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aa4d0 0x55a6044aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa4892a8640 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b5e50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa478009b70 tx=0x7fa478010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b83190b69efde9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa4700a3da0 0x7fa4700a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005cc0 tx=0x7fa47c005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa46ffff640 1 -- 172.21.15.66:0/1483494405 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478009e60 con 0x55a6044b3a00 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa46ffff640 1 -- 172.21.15.66:0/1483494405 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4780035f0 con 0x55a6044b3a00 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.021+0000 7fa46ffff640 1 -- 172.21.15.66:0/1483494405 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478012690 con 0x55a6044b3a00 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 -- 172.21.15.66:0/1483494405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 msgr2=0x55a6044b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b5e50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa478009b70 tx=0x7fa478010740 comp rx=0 tx=0).stop 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 -- 172.21.15.66:0/1483494405 shutdown_connections 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b5e50 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aae80 0x55a6044b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/1483494405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aa4d0 0x55a6044aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 -- 172.21.15.66:0/1483494405 >> 172.21.15.66:0/1483494405 conn(0x55a6044a4ba0 msgr2=0x55a6044a4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 -- 172.21.15.66:0/1483494405 shutdown_connections 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 -- 172.21.15.66:0/1483494405 wait complete. 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 -- start start 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.022+0000 7fa48e9ce740 1 Processor -- start 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aae80 0x55a6044b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044b3a00 0x55a6044b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a6044b6970 con 0x55a6044aae80 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a6043c8500 con 0x55a6044b3a00 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a6044b6af0 con 0x55a6044aa4d0 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 10 client.0 _ll_register_callbacks cb 0x55a604396b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aae80 0x55a6044b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aae80 0x55a6044b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43694/0 (socket says 172.21.15.66:43694) 2024-04-21T09:15:17.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 -- 172.21.15.66:0/3431116549 learned_addr learned my addr 172.21.15.66:0/3431116549 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa489aa9640 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044b3a00 0x55a6044b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 -- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 msgr2=0x55a6044b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 -- 172.21.15.66:0/3431116549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044b3a00 msgr2=0x55a6044b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044b3a00 0x55a6044b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 -- 172.21.15.66:0/3431116549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6041213f0 con 0x55a6044aae80 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa4892a8640 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aae80 0x55a6044b0640 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa478003e50 tx=0x7fa4780039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483b55373dc31796 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa46d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa46d7fa640 1 -- 172.21.15.66:0/3431116549 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478009740 con 0x55a6044aae80 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa46d7fa640 1 -- 172.21.15.66:0/3431116549 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fa478004210 con 0x55a6044aae80 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.023+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4780035f0 con 0x55a6044aae80 2024-04-21T09:15:17.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa46d7fa640 1 -- 172.21.15.66:0/3431116549 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa478004410 con 0x55a6044aae80 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 10 client.17475 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6041eb7d0 con 0x55a6044aae80 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa46d7fa640 1 -- 172.21.15.66:0/3431116549 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4780035f0 con 0x55a6044aae80 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa46d7fa640 1 -- 172.21.15.66:0/3431116549 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa47800f040 con 0x55a6044aae80 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 10 client.17475 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 10 client.17475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 20 client.17475 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a6044c3af0 0x55a6044c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 4 client.17475 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6043ae930 con 0x55a6044c3af0 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.024+0000 7fa489aa9640 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a6044c3af0 0x55a6044c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.025+0000 7fa46d7fa640 10 client.17475 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.025+0000 7fa489aa9640 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a6044c3af0 0x55a6044c5ee0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=182684edb1289139 server_cookie=8a234854fc38c0cf in_seq=0 out_seq=0 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa46d7fa640 1 -- 172.21.15.66:0/3431116549 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6043ae930 con 0x55a6044c3af0 2024-04-21T09:15:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa46d7fa640 10 client.17475 handle_command_reply: tid=0 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 client.17475 shutdown 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aae80 msgr2=0x55a6044b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aae80 0x55a6044b0640 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa478003e50 tx=0x7fa4780039d0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 shutdown_connections 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044b3a00 0x55a6044b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a6044c3af0 0x55a6044c5ee0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aae80 0x55a6044b0640 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/3431116549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa46d7fa640 0 client.17475 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 >> 172.21.15.66:0/3431116549 conn(0x55a6044a4ba0 msgr2=0x55a6044a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 shutdown_connections 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 1 -- 172.21.15.66:0/3431116549 wait complete. 2024-04-21T09:15:17.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.026+0000 7fa48e9ce740 20 client.17475 trim_cache size 0 max 16384 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.039+0000 7fa48e9ce740 1 Processor -- start 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.039+0000 7fa48e9ce740 1 -- start start 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841b1cc0 con 0x55a6044b3a00 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841af590 con 0x55a6044aae80 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841ad400 con 0x55a6044aa4d0 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43698/0 (socket says 172.21.15.66:43698) 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 -- 172.21.15.66:0/395559984 learned_addr learned my addr 172.21.15.66:0/395559984 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa489aa9640 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 -- 172.21.15.66:0/395559984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 msgr2=0x55a6044bf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa4892a8640 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044bf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 -- 172.21.15.66:0/395559984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 msgr2=0x55a6044a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 -- 172.21.15.66:0/395559984 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6044b6160 con 0x55a6044b3a00 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa4892a8640 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b2d70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009ca0 tx=0x7fa47c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebb828bd603ced5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa46f7fe640 1 -- 172.21.15.66:0/395559984 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c039040 con 0x55a6044b3a00 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa46f7fe640 1 -- 172.21.15.66:0/395559984 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c046070 con 0x55a6044b3a00 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.040+0000 7fa46f7fe640 1 -- 172.21.15.66:0/395559984 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c004690 con 0x55a6044b3a00 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 -- 172.21.15.66:0/395559984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 msgr2=0x55a6044b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b2d70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009ca0 tx=0x7fa47c02d000 comp rx=0 tx=0).stop 2024-04-21T09:15:17.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 -- 172.21.15.66:0/395559984 shutdown_connections 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044b3a00 0x55a6044b2d70 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/395559984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044aa4d0 0x55a6044bf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 -- 172.21.15.66:0/395559984 >> 172.21.15.66:0/395559984 conn(0x55a6044a4ba0 msgr2=0x55a6044a3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 -- 172.21.15.66:0/395559984 shutdown_connections 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 -- 172.21.15.66:0/395559984 wait complete. 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 -- start start 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.041+0000 7fa48e9ce740 1 Processor -- start 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aa4d0 0x55a6044ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48e9ce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044b3a00 0x55a6044ade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841ada50 con 0x55a6044aa4d0 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841aeec0 con 0x55a6044aae80 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48e9ce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841ae120 con 0x55a6044b3a00 2024-04-21T09:15:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48e9ce740 10 client.0 _ll_register_callbacks cb 0x7fa478003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aa4d0 0x55a6044ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aa4d0 0x55a6044ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43710/0 (socket says 172.21.15.66:43710) 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 -- 172.21.15.66:0/2914853198 learned_addr learned my addr 172.21.15.66:0/2914853198 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa48a2aa640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044b3a00 0x55a6044ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa4892a8640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 -- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044b3a00 msgr2=0x55a6044ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044b3a00 0x55a6044ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 -- 172.21.15.66:0/2914853198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 msgr2=0x55a6044ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 -- 172.21.15.66:0/2914853198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a604528f90 con 0x55a6044aa4d0 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa4892a8640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa489aa9640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aa4d0 0x55a6044ad3e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c6a0 tx=0x7fa480092020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42bab78cd60efeb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa46d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:17.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.042+0000 7fa46d7fa640 1 -- 172.21.15.66:0/2914853198 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4800a5800 con 0x55a6044aa4d0 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa46d7fa640 1 -- 172.21.15.66:0/2914853198 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fa4800ce5c0 con 0x55a6044aa4d0 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47c046070 con 0x55a6044aa4d0 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa46d7fa640 1 -- 172.21.15.66:0/2914853198 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa4800b6080 con 0x55a6044aa4d0 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 10 client.17481 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a60452a260 con 0x55a6044aa4d0 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa46d7fa640 1 -- 172.21.15.66:0/2914853198 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c046070 con 0x55a6044aa4d0 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa46d7fa640 1 -- 172.21.15.66:0/2914853198 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa4800bd040 con 0x55a6044aa4d0 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 10 client.17481 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 10 client.17481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 20 client.17481 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a60452a960 0x55a60452cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 4 client.17481 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a60441cc10 con 0x55a60452a960 2024-04-21T09:15:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.043+0000 7fa4892a8640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a60452a960 0x55a60452cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa46d7fa640 10 client.17481 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa4892a8640 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a60452a960 0x55a60452cd80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19e0af9974ceab11 server_cookie=9c27f83d9797e7a5 in_seq=0 out_seq=0 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa46d7fa640 1 -- 172.21.15.66:0/2914853198 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a60441cc10 con 0x55a60452a960 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa46d7fa640 10 client.17481 handle_command_reply: tid=0 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 client.17481 shutdown 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aa4d0 msgr2=0x55a6044ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aa4d0 0x55a6044ad3e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c6a0 tx=0x7fa480092020 comp rx=0 tx=0).stop 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 shutdown_connections 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6044b3a00 0x55a6044ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6044aae80 0x55a6044ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a60452a960 0x55a60452cd80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 --2- 172.21.15.66:0/2914853198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6044aa4d0 0x55a6044ad3e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa46d7fa640 0 client.17481 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 >> 172.21.15.66:0/2914853198 conn(0x55a6044a4ba0 msgr2=0x55a6044abaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 shutdown_connections 2024-04-21T09:15:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 1 -- 172.21.15.66:0/2914853198 wait complete. 2024-04-21T09:15:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.044+0000 7fa48e9ce740 20 client.17481 trim_cache size 0 max 16384 2024-04-21T09:15:17.051 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa4700a3da0 msgr2=0x7fa4700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa4700a3da0 0x7fa4700a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005cc0 tx=0x7fa47c005c50 comp rx=0 tx=0).stop 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48406a840 msgr2=0x7fa4841986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48406a840 0x7fa4841986d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e9f0 tx=0x7fa48000eeb0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 shutdown_connections 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa4700a3da0 0x7fa4700a6260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa48419cf40 0x7fa48419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa484107270 0x7fa484198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 --2- 172.21.15.66:0/923564620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48406a840 0x7fa4841986d0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 >> 172.21.15.66:0/923564620 conn(0x7fa484077eb0 msgr2=0x7fa4840fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 shutdown_connections 2024-04-21T09:15:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.045+0000 7fa48bd0d640 1 -- 172.21.15.66:0/923564620 wait complete. 2024-04-21T09:15:17.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 -- 172.21.15.66:0/1947126928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc908101a50 msgr2=0x7fc908105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 --2- 172.21.15.66:0/1947126928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc908101a50 0x7fc908105a90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009900 tx=0x7fc8f802bc30 comp rx=0 tx=0).stop 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 -- 172.21.15.66:0/1947126928 shutdown_connections 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 --2- 172.21.15.66:0/1947126928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc908106170 0x7fc90810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 --2- 172.21.15.66:0/1947126928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc908101a50 0x7fc908105a90 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 --2- 172.21.15.66:0/1947126928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081010a0 0x7fc908101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 -- 172.21.15.66:0/1947126928 >> 172.21.15.66:0/1947126928 conn(0x7fc908077eb0 msgr2=0x7fc9080782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 -- 172.21.15.66:0/1947126928 shutdown_connections 2024-04-21T09:15:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.159+0000 7fc90cbab640 1 -- 172.21.15.66:0/1947126928 wait complete. 2024-04-21T09:15:17.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 Processor -- start 2024-04-21T09:15:17.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 -- start start 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9081010a0 0x7fc908102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc908106170 0x7fc908103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9081035b0 0x7fc9081a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc908111090 con 0x7fc9081010a0 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc908110f10 con 0x7fc908106170 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90cbab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc908111210 con 0x7fc9081035b0 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9081010a0 0x7fc908102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9081010a0 0x7fc908102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43736/0 (socket says 172.21.15.66:43736) 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc90659c640 1 -- 172.21.15.66:0/3137396544 learned_addr learned my addr 172.21.15.66:0/3137396544 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc905d9b640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc908106170 0x7fc908103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.160+0000 7fc906d9d640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9081035b0 0x7fc9081a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90659c640 1 -- 172.21.15.66:0/3137396544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9081035b0 msgr2=0x7fc9081a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90659c640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9081035b0 0x7fc9081a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90659c640 1 -- 172.21.15.66:0/3137396544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc908106170 msgr2=0x7fc908103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90659c640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc908106170 0x7fc908103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90659c640 1 -- 172.21.15.66:0/3137396544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9081a6fd0 con 0x7fc9081010a0 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90659c640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9081010a0 0x7fc908102b30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000ea60 tx=0x7fc8f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc8ff7fe640 1 -- 172.21.15.66:0/3137396544 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000cde0 con 0x7fc9081010a0 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc8ff7fe640 1 -- 172.21.15.66:0/3137396544 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f00035f0 con 0x7fc9081010a0 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9081a72c0 con 0x7fc9081010a0 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc8ff7fe640 1 -- 172.21.15.66:0/3137396544 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0010690 con 0x7fc9081010a0 2024-04-21T09:15:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.161+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9081a77e0 con 0x7fc9081010a0 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.163+0000 7fc8ff7fe640 1 -- 172.21.15.66:0/3137396544 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc8f00037a0 con 0x7fc9081010a0 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.163+0000 7fc90f79d740 1 Processor -- start 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90f79d740 1 -- start start 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae64e80 0x562ccae6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90f79d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562ccadc91f0 con 0x562ccae644d0 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90f79d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562ccad664b0 con 0x562ccae64e80 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90f79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562ccadc51b0 con 0x562ccae6da00 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43750/0 (socket says 172.21.15.66:43750) 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 -- 172.21.15.66:0/2931424589 learned_addr learned my addr 172.21.15.66:0/2931424589 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc90659c640 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae64e80 0x562ccae6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc8ff7fe640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc905d9b640 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 -- 172.21.15.66:0/2931424589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 msgr2=0x562ccae6fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 -- 172.21.15.66:0/2931424589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae64e80 msgr2=0x562ccae6d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae64e80 0x562ccae6d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 -- 172.21.15.66:0/2931424589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ccad581b0 con 0x562ccae644d0 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc8ff7fe640 1 -- 172.21.15.66:0/3137396544 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fc8f00d0f40 con 0x7fc9081010a0 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.164+0000 7fc906d9d640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.165+0000 7fc906d9d640 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae648b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc900010520 tx=0x7fc9000109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5a3ff516d281c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.165+0000 7fc8fcff9640 1 -- 172.21.15.66:0/2931424589 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900019070 con 0x562ccae644d0 2024-04-21T09:15:17.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.165+0000 7fc8fcff9640 1 -- 172.21.15.66:0/2931424589 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc900003e10 con 0x562ccae644d0 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.165+0000 7fc8fcff9640 1 -- 172.21.15.66:0/2931424589 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900011660 con 0x562ccae644d0 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- 172.21.15.66:0/2931424589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 msgr2=0x562ccae648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae648b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc900010520 tx=0x7fc9000109e0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc906d9d640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc900021000 tx=0x7fc90001e000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- 172.21.15.66:0/2931424589 shutdown_connections 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae64e80 0x562ccae6d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 --2- 172.21.15.66:0/2931424589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae648b0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- 172.21.15.66:0/2931424589 >> 172.21.15.66:0/2931424589 conn(0x562ccae5eba0 msgr2=0x562ccae5ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- 172.21.15.66:0/2931424589 shutdown_connections 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- 172.21.15.66:0/2931424589 wait complete. 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- start start 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 Processor -- start 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae6d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae6a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562ccae70970 con 0x562ccae64e80 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562ccad82500 con 0x562ccae644d0 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562ccae70af0 con 0x562ccae6da00 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc90f79d740 10 client.0 _ll_register_callbacks cb 0x562ccad50b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.166+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae6a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae6a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43758/0 (socket says 172.21.15.66:43758) 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 -- 172.21.15.66:0/3372465733 learned_addr learned my addr 172.21.15.66:0/3372465733 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc90659c640 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae6d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc906d9d640 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 -- 172.21.15.66:0/3372465733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 msgr2=0x562ccae6abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 -- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 msgr2=0x562ccae6d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae6d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 -- 172.21.15.66:0/3372465733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccaadb3f0 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc905d9b640 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae6a670 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9084571b7f033f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc8edffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc8edffb640 1 -- 172.21.15.66:0/3372465733 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f802c610 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc8edffb640 1 -- 172.21.15.66:0/3372465733 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fc8f8005030 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc900003e10 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc8edffb640 1 -- 172.21.15.66:0/3372465733 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8f8005230 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc90f79d740 10 client.17493 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.167+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562ccaba57d0 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc8edffb640 1 -- 172.21.15.66:0/3372465733 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc900003e10 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc8edffb640 1 -- 172.21.15.66:0/3372465733 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8f80035f0 con 0x562ccae64e80 2024-04-21T09:15:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90f79d740 10 client.17493 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90f79d740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90f79d740 20 client.17493 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccae80040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90f79d740 4 client.17493 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ccad68930 con 0x562ccae7dc20 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90659c640 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccae80040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc8edffb640 10 client.17493 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.168+0000 7fc90659c640 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccae80040 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd0b0465c8759c1 server_cookie=b3bb9a05740e1ba4 in_seq=0 out_seq=0 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc8edffb640 1 -- 172.21.15.66:0/3372465733 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ccad68930 con 0x562ccae7dc20 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc8edffb640 10 client.17493 handle_command_reply: tid=0 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 client.17493 shutdown 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 msgr2=0x562ccae6a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae6a670 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f802ca10 comp rx=0 tx=0).stop 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 shutdown_connections 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccae80040 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae6a670 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc8edffb640 0 client.17493 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3372465733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae6d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 >> 172.21.15.66:0/3372465733 conn(0x562ccae5eba0 msgr2=0x562ccae608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 shutdown_connections 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 1 -- 172.21.15.66:0/3372465733 wait complete. 2024-04-21T09:15:17.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.169+0000 7fc90f79d740 20 client.17493 trim_cache size 0 max 16384 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 Processor -- start 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 -- start start 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae79830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae65ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081b3ec0 con 0x562ccae64e80 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081b1790 con 0x562ccae644d0 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90f79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081af600 con 0x562ccae6da00 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae65ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae65ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43768/0 (socket says 172.21.15.66:43768) 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 -- 172.21.15.66:0/3191024656 learned_addr learned my addr 172.21.15.66:0/3191024656 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc90659c640 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae79830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc906d9d640 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 -- 172.21.15.66:0/3191024656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 msgr2=0x562ccae6cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 -- 172.21.15.66:0/3191024656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 msgr2=0x562ccae79830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae79830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 -- 172.21.15.66:0/3191024656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ccae63790 con 0x562ccae64e80 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.183+0000 7fc905d9b640 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae65ae0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc8f802c170 tx=0x7fc8f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae478040f77172b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc8effff640 1 -- 172.21.15.66:0/3191024656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f802c610 con 0x562ccae64e80 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc8effff640 1 -- 172.21.15.66:0/3191024656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f805a070 con 0x562ccae64e80 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc8effff640 1 -- 172.21.15.66:0/3191024656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8005d50 con 0x562ccae64e80 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 -- 172.21.15.66:0/3191024656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 msgr2=0x562ccae65ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae65ae0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc8f802c170 tx=0x7fc8f8009780 comp rx=0 tx=0).stop 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 -- 172.21.15.66:0/3191024656 shutdown_connections 2024-04-21T09:15:17.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae6da00 0x562ccae6cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae64e80 0x562ccae65ae0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 --2- 172.21.15.66:0/3191024656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae644d0 0x562ccae79830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 -- 172.21.15.66:0/3191024656 >> 172.21.15.66:0/3191024656 conn(0x562ccae5eba0 msgr2=0x562ccae653d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 -- 172.21.15.66:0/3191024656 shutdown_connections 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.184+0000 7fc90f79d740 1 -- 172.21.15.66:0/3191024656 wait complete. 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 -- start start 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 Processor -- start 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae67690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae64e80 0x562ccae67bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae6da00 0x562ccaee3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081afc50 con 0x562ccae644d0 2024-04-21T09:15:17.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081b10c0 con 0x562ccae6da00 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081b0320 con 0x562ccae64e80 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90f79d740 10 client.0 _ll_register_callbacks cb 0x7fc8f8004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae67690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae67690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43774/0 (socket says 172.21.15.66:43774) 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 -- 172.21.15.66:0/610599384 learned_addr learned my addr 172.21.15.66:0/610599384 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc905d9b640 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae64e80 0x562ccae67bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc906d9d640 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae6da00 0x562ccaee3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 -- 172.21.15.66:0/610599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae64e80 msgr2=0x562ccae67bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae64e80 0x562ccae67bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 -- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae6da00 msgr2=0x562ccaee3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae6da00 0x562ccaee3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 -- 172.21.15.66:0/610599384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccae684a0 con 0x562ccae644d0 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc90659c640 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae67690 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c760 tx=0x7fc8f00d4630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ca692098515c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:17.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.185+0000 7fc8edffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc8edffb640 1 -- 172.21.15.66:0/610599384 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f009caf0 con 0x562ccae644d0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc8edffb640 1 -- 172.21.15.66:0/610599384 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fc8f0003ac0 con 0x562ccae644d0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f805a070 con 0x562ccae644d0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc8edffb640 1 -- 172.21.15.66:0/610599384 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8f00a0ea0 con 0x562ccae644d0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 10 client.17499 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562ccaee4790 con 0x562ccae644d0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc8edffb640 1 -- 172.21.15.66:0/610599384 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f805a070 con 0x562ccae644d0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc8edffb640 1 -- 172.21.15.66:0/610599384 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8f00ce520 con 0x562ccae644d0 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 10 client.17499 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 20 client.17499 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccaee6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 4 client.17499 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ccadd6c10 con 0x562ccae7dc20 2024-04-21T09:15:17.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.186+0000 7fc905d9b640 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccaee6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc8edffb640 10 client.17499 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc905d9b640 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccaee6ea0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914a933528e50ce5 server_cookie=da1d1079ebd8a17c in_seq=0 out_seq=0 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc8edffb640 1 -- 172.21.15.66:0/610599384 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ccadd6c10 con 0x562ccae7dc20 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc8edffb640 10 client.17499 handle_command_reply: tid=0 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 client.17499 shutdown 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 msgr2=0x562ccae67690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae67690 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c760 tx=0x7fc8f00d4630 comp rx=0 tx=0).stop 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 shutdown_connections 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x562ccae7dc20 0x562ccaee6ea0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ccae6da00 0x562ccaee3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc8edffb640 0 client.17499 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562ccae64e80 0x562ccae67bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 --2- 172.21.15.66:0/610599384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ccae644d0 0x562ccae67690 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 >> 172.21.15.66:0/610599384 conn(0x562ccae5eba0 msgr2=0x562ccae60850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 shutdown_connections 2024-04-21T09:15:17.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 1 -- 172.21.15.66:0/610599384 wait complete. 2024-04-21T09:15:17.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.187+0000 7fc90f79d740 20 client.17499 trim_cache size 0 max 16384 2024-04-21T09:15:17.194 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8f40a3d20 msgr2=0x7fc8f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc900021000 tx=0x7fc90001e000 comp rx=0 tx=0).stop 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9081010a0 msgr2=0x7fc908102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9081010a0 0x7fc908102b30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000ea60 tx=0x7fc8f000ef20 comp rx=0 tx=0).stop 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 shutdown_connections 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9081035b0 0x7fc9081a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc908106170 0x7fc908103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 --2- 172.21.15.66:0/3137396544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9081010a0 0x7fc908102b30 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 >> 172.21.15.66:0/3137396544 conn(0x7fc908077eb0 msgr2=0x7fc90806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 shutdown_connections 2024-04-21T09:15:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:17.188+0000 7fc90cbab640 1 -- 172.21.15.66:0/3137396544 wait complete. 2024-04-21T09:15:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:17 smithi186 ceph-mon[31505]: pgmap v217: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:15:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2201025570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:15:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:17 smithi083 ceph-mon[31413]: pgmap v217: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:15:17.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2201025570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:15:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:17 smithi066 ceph-mon[27953]: pgmap v217: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:15:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2201025570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:15:18.199 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:15:18.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:15:18.199 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:15:18.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 -- 172.21.15.66:0/1698030572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d38107260 msgr2=0x7f0d3810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1698030572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d38107260 0x7f0d3810ee30 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009ca0 tx=0x7f0d3002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 -- 172.21.15.66:0/1698030572 shutdown_connections 2024-04-21T09:15:18.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1698030572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d38107260 0x7f0d3810ee30 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1698030572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38102b40 0x7f0d38106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1698030572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d38102190 0x7f0d38102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 -- 172.21.15.66:0/1698030572 >> 172.21.15.66:0/1698030572 conn(0x7f0d380fd990 msgr2=0x7f0d380ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 -- 172.21.15.66:0/1698030572 shutdown_connections 2024-04-21T09:15:18.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.319+0000 7f0d3e304640 1 -- 172.21.15.66:0/1698030572 wait complete. 2024-04-21T09:15:18.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 Processor -- start 2024-04-21T09:15:18.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 -- start start 2024-04-21T09:15:18.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38102190 0x7f0d38198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d38107260 0x7f0d38198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3819cfd0 0x7f0d3819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381121d0 con 0x7f0d3819cfd0 2024-04-21T09:15:18.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38112050 con 0x7f0d38107260 2024-04-21T09:15:18.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3e304640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38112350 con 0x7f0d38102190 2024-04-21T09:15:18.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3c8a1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3819cfd0 0x7f0d3819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3c8a1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3819cfd0 0x7f0d3819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43790/0 (socket says 172.21.15.66:43790) 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.320+0000 7f0d3c8a1640 1 -- 172.21.15.66:0/1926342978 learned_addr learned my addr 172.21.15.66:0/1926342978 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d377fe640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d38107260 0x7f0d38198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d37fff640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38102190 0x7f0d38198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3c8a1640 1 -- 172.21.15.66:0/1926342978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38102190 msgr2=0x7f0d38198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3c8a1640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38102190 0x7f0d38198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3c8a1640 1 -- 172.21.15.66:0/1926342978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d38107260 msgr2=0x7f0d38198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3c8a1640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d38107260 0x7f0d38198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3c8a1640 1 -- 172.21.15.66:0/1926342978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3819dac0 con 0x7f0d3819cfd0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d377fe640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d38107260 0x7f0d38198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3c8a1640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3819cfd0 0x7f0d3819d480 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d30005e30 tx=0x7f0d30032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d357fa640 1 -- 172.21.15.66:0/1926342978 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30041020 con 0x7f0d3819cfd0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d357fa640 1 -- 172.21.15.66:0/1926342978 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30032970 con 0x7f0d3819cfd0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3819dd50 con 0x7f0d3819cfd0 2024-04-21T09:15:18.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d357fa640 1 -- 172.21.15.66:0/1926342978 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30039040 con 0x7f0d3819cfd0 2024-04-21T09:15:18.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.321+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d381a5570 con 0x7f0d3819cfd0 2024-04-21T09:15:18.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.323+0000 7f0d357fa640 1 -- 172.21.15.66:0/1926342978 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0d30044020 con 0x7f0d3819cfd0 2024-04-21T09:15:18.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 Processor -- start 2024-04-21T09:15:18.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 -- start start 2024-04-21T09:15:18.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55690e93df00 con 0x55690e9e3270 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55690e8db590 con 0x55690e9da170 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55690e939ec0 con 0x55690e9d97c0 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d357fa640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d200a3da0 0x7f0d200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43798/0 (socket says 172.21.15.66:43798) 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d377fe640 1 -- 172.21.15.66:0/1457278150 learned_addr learned my addr 172.21.15.66:0/1457278150 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d3c8a1640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d200a3da0 0x7f0d200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d37fff640 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.324+0000 7f0d377fe640 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d357fa640 1 -- 172.21.15.66:0/1926342978 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f0d300f2880 con 0x7f0d3819cfd0 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d37fff640 1 -- 172.21.15.66:0/1457278150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 msgr2=0x55690e9d9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d37fff640 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9d9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d37fff640 1 -- 172.21.15.66:0/1457278150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 msgr2=0x55690e9e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d37fff640 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d37fff640 1 -- 172.21.15.66:0/1457278150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55690e7187e0 con 0x55690e9da170 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d377fe640 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d3c8a1640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d200a3da0 0x7f0d200a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d3003f000 tx=0x7f0d30091050 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d37fff640 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e2ca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009ab0 tx=0x7f0d28009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae29c241b67c6b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d1e7fc640 1 -- 172.21.15.66:0/1457278150 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28016070 con 0x55690e9da170 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d1e7fc640 1 -- 172.21.15.66:0/1457278150 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d280054d0 con 0x55690e9da170 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.325+0000 7f0d1e7fc640 1 -- 172.21.15.66:0/1457278150 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28004d90 con 0x55690e9da170 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 -- 172.21.15.66:0/1457278150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 msgr2=0x55690e9e2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e2ca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009ab0 tx=0x7f0d28009f70 comp rx=0 tx=0).stop 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 -- 172.21.15.66:0/1457278150 shutdown_connections 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e2ca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/1457278150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9d9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 -- 172.21.15.66:0/1457278150 >> 172.21.15.66:0/1457278150 conn(0x55690e9d3fb0 msgr2=0x55690e9d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 -- 172.21.15.66:0/1457278150 shutdown_connections 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 -- 172.21.15.66:0/1457278150 wait complete. 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.326+0000 7f0d40fd3740 1 -- start start 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 1 Processor -- start 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55690e9ea1b0 con 0x55690e9e3270 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55690e8f7580 con 0x55690e9da170 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55690e9ea330 con 0x55690e9d97c0 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d40fd3740 10 client.0 _ll_register_callbacks cb 0x55690e8fd650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:18.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d37fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43812/0 (socket says 172.21.15.66:43812) 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 -- 172.21.15.66:0/3178399184 learned_addr learned my addr 172.21.15.66:0/3178399184 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 -- 172.21.15.66:0/3178399184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 msgr2=0x55690e9e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 -- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 msgr2=0x55690e9e71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 -- 172.21.15.66:0/3178399184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55690e778f70 con 0x55690e9e3270 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d37fff640 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d377fe640 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e7730 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c014e50 tx=0x7f0d2c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664365b08c8667a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.327+0000 7f0d177fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d177fe640 1 -- 172.21.15.66:0/3178399184 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0140d0 con 0x55690e9e3270 2024-04-21T09:15:18.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d177fe640 1 -- 172.21.15.66:0/3178399184 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f0d2c009040 con 0x55690e9e3270 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d280054d0 con 0x55690e9e3270 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d177fe640 1 -- 172.21.15.66:0/3178399184 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d2c014270 con 0x55690e9e3270 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 10 client.17511 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55690e610170 con 0x55690e9e3270 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d177fe640 1 -- 172.21.15.66:0/3178399184 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d280054d0 con 0x55690e9e3270 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d177fe640 1 -- 172.21.15.66:0/3178399184 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d2c003b40 con 0x55690e9e3270 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 10 client.17511 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 20 client.17511 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690e9f7630 0x55690e9f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 4 client.17511 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.328+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55690e94b920 con 0x55690e9f7630 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.329+0000 7f0d37fff640 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690e9f7630 0x55690e9f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.329+0000 7f0d37fff640 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690e9f7630 0x55690e9f9a40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd26b6c6bcd4bc2 server_cookie=3950181c8d7e4bd3 in_seq=0 out_seq=0 2024-04-21T09:15:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.329+0000 7f0d177fe640 10 client.17511 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d177fe640 1 -- 172.21.15.66:0/3178399184 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55690e94b920 con 0x55690e9f7630 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d177fe640 10 client.17511 handle_command_reply: tid=0 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 client.17511 shutdown 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 msgr2=0x55690e9e7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e7730 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c014e50 tx=0x7f0d2c016040 comp rx=0 tx=0).stop 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 shutdown_connections 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690e9f7630 0x55690e9f9a40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e7730 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d177fe640 0 client.17511 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9da170 0x55690e9e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/3178399184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9d97c0 0x55690e9e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 >> 172.21.15.66:0/3178399184 conn(0x55690e9d3fb0 msgr2=0x55690e9d5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 shutdown_connections 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 1 -- 172.21.15.66:0/3178399184 wait complete. 2024-04-21T09:15:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.330+0000 7f0d40fd3740 20 client.17511 trim_cache size 0 max 16384 2024-04-21T09:15:18.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 Processor -- start 2024-04-21T09:15:18.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 -- start start 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381afae0 con 0x55690e9e3270 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381ad3b0 con 0x55690e9d97c0 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.348+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381ab220 con 0x55690e9da170 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43828/0 (socket says 172.21.15.66:43828) 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 -- 172.21.15.66:0/4106581845 learned_addr learned my addr 172.21.15.66:0/4106581845 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d37fff640 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 -- 172.21.15.66:0/4106581845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 msgr2=0x55690e9f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 -- 172.21.15.66:0/4106581845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 msgr2=0x55690e9e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 -- 172.21.15.66:0/4106581845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55690e9cf880 con 0x55690e9e3270 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d377fe640 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e65b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c005fd0 tx=0x7f0d2c023430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d81a07e5cafcf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d1dffb640 1 -- 172.21.15.66:0/4106581845 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0140d0 con 0x55690e9e3270 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d1dffb640 1 -- 172.21.15.66:0/4106581845 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2c005850 con 0x55690e9e3270 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.349+0000 7f0d1dffb640 1 -- 172.21.15.66:0/4106581845 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c023850 con 0x55690e9e3270 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 -- 172.21.15.66:0/4106581845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 msgr2=0x55690e9e65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e65b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c005fd0 tx=0x7f0d2c023430 comp rx=0 tx=0).stop 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 -- 172.21.15.66:0/4106581845 shutdown_connections 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9e65b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/4106581845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 -- 172.21.15.66:0/4106581845 >> 172.21.15.66:0/4106581845 conn(0x55690e9d3fb0 msgr2=0x55690e9d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 -- 172.21.15.66:0/4106581845 shutdown_connections 2024-04-21T09:15:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 -- 172.21.15.66:0/4106581845 wait complete. 2024-04-21T09:15:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.350+0000 7f0d40fd3740 1 -- start start 2024-04-21T09:15:18.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 1 Processor -- start 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9fe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9fee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9ff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381ab870 con 0x55690e9e3270 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381acce0 con 0x55690e9d97c0 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381abf40 con 0x55690e9da170 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d40fd3740 10 client.0 _ll_register_callbacks cb 0x7f0d2c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9ff380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43842/0 (socket says 172.21.15.66:43842) 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 -- 172.21.15.66:0/2338776582 learned_addr learned my addr 172.21.15.66:0/2338776582 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d37fff640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 -- 172.21.15.66:0/2338776582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 msgr2=0x55690e9fee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9fee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 -- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 msgr2=0x55690e9fe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9fe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 -- 172.21.15.66:0/2338776582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55690ea5cb80 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d37fff640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d377fe640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9ff380 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c023a50 tx=0x7f0d2c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668cfcba4a15ed33 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d177fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.351+0000 7f0d177fe640 1 -- 172.21.15.66:0/2338776582 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c023cd0 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d177fe640 1 -- 172.21.15.66:0/2338776582 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f0d2c003f90 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d2c005850 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d177fe640 1 -- 172.21.15.66:0/2338776582 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d2c0140b0 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 10 client.17517 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55690ea5de60 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d177fe640 1 -- 172.21.15.66:0/2338776582 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2c005850 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d177fe640 1 -- 172.21.15.66:0/2338776582 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d2c012450 con 0x55690e9e3270 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 10 client.17517 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:18.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 20 client.17517 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690ea5e470 0x55690ea60860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 4 client.17517 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55690e945790 con 0x55690ea5e470 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.352+0000 7f0d37fff640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690ea5e470 0x55690ea60860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.353+0000 7f0d177fe640 10 client.17517 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.353+0000 7f0d37fff640 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690ea5e470 0x55690ea60860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b666255afeb8127 server_cookie=9d7d12fa86e018ff in_seq=0 out_seq=0 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d177fe640 1 -- 172.21.15.66:0/2338776582 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55690e945790 con 0x55690ea5e470 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d177fe640 10 client.17517 handle_command_reply: tid=0 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 client.17517 shutdown 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 msgr2=0x55690e9ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9ff380 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c023a50 tx=0x7f0d2c009890 comp rx=0 tx=0).stop 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 shutdown_connections 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55690ea5e470 0x55690ea60860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55690e9e3270 0x55690e9ff380 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55690e9da170 0x55690e9fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 --2- 172.21.15.66:0/2338776582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55690e9d97c0 0x55690e9fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d177fe640 0 client.17517 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 >> 172.21.15.66:0/2338776582 conn(0x55690e9d3fb0 msgr2=0x55690e9e6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 shutdown_connections 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 1 -- 172.21.15.66:0/2338776582 wait complete. 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.354+0000 7f0d40fd3740 20 client.17517 trim_cache size 0 max 16384 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "f96c406a-6355-43d8-8619-30402cc1d4ba", 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:15:18.359 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:15:18.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d200a3da0 msgr2=0x7f0d200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d200a3da0 0x7f0d200a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d3003f000 tx=0x7f0d30091050 comp rx=0 tx=0).stop 2024-04-21T09:15:18.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3819cfd0 msgr2=0x7f0d3819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3819cfd0 0x7f0d3819d480 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0d30005e30 tx=0x7f0d30032420 comp rx=0 tx=0).stop 2024-04-21T09:15:18.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 shutdown_connections 2024-04-21T09:15:18.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0d200a3da0 0x7f0d200a6260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d3819cfd0 0x7f0d3819d480 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d38107260 0x7f0d38198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 --2- 172.21.15.66:0/1926342978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d38102190 0x7f0d38198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 >> 172.21.15.66:0/1926342978 conn(0x7f0d380fd990 msgr2=0x7f0d380fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 shutdown_connections 2024-04-21T09:15:18.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.355+0000 7f0d3e304640 1 -- 172.21.15.66:0/1926342978 wait complete. 2024-04-21T09:15:18.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:15:18.365 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:15:18.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 -- 172.21.15.66:0/1111542227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf0107270 msgr2=0x7f5bf010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/1111542227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf0107270 0x7f5bf010ee40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009ca0 tx=0x7f5be802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 -- 172.21.15.66:0/1111542227 shutdown_connections 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/1111542227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf0107270 0x7f5bf010ee40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/1111542227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0102b50 0x7f5bf0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/1111542227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf01021a0 0x7f5bf0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 -- 172.21.15.66:0/1111542227 >> 172.21.15.66:0/1111542227 conn(0x7f5bf00793e0 msgr2=0x7f5bf00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 -- 172.21.15.66:0/1111542227 shutdown_connections 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.471+0000 7f5bf4e06640 1 -- 172.21.15.66:0/1111542227 wait complete. 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 Processor -- start 2024-04-21T09:15:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 -- start start 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf01021a0 0x7f5bf0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0107270 0x7f5bf0198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf019cfe0 0x7f5bf019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf0112230 con 0x7f5bf019cfe0 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01120b0 con 0x7f5bf0107270 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.472+0000 7f5bf4e06640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01123b0 con 0x7f5bf01021a0 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf019cfe0 0x7f5bf019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf019cfe0 0x7f5bf019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43862/0 (socket says 172.21.15.66:43862) 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 -- 172.21.15.66:0/186871814 learned_addr learned my addr 172.21.15.66:0/186871814 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5bee59c640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf01021a0 0x7f5bf0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0107270 0x7f5bf0198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 -- 172.21.15.66:0/186871814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf01021a0 msgr2=0x7f5bf0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf01021a0 0x7f5bf0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 -- 172.21.15.66:0/186871814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0107270 msgr2=0x7f5bf0198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0107270 0x7f5bf0198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 -- 172.21.15.66:0/186871814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf019dad0 con 0x7f5bf019cfe0 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5beed9d640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf019cfe0 0x7f5bf019d490 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5be8005e30 tx=0x7f5be8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5bdf7fe640 1 -- 172.21.15.66:0/186871814 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8039040 con 0x7f5bf019cfe0 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5bdf7fe640 1 -- 172.21.15.66:0/186871814 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be8044070 con 0x7f5bf019cfe0 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf019dd60 con 0x7f5bf019cfe0 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5bdf7fe640 1 -- 172.21.15.66:0/186871814 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8003e60 con 0x7f5bf019cfe0 2024-04-21T09:15:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.473+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf01a5520 con 0x7f5bf019cfe0 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.475+0000 7f5bdf7fe640 1 -- 172.21.15.66:0/186871814 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5be80324c0 con 0x7f5bf019cfe0 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.476+0000 7f5bf79f9740 1 Processor -- start 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.476+0000 7f5bf79f9740 1 -- start start 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.476+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c5795f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.476+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 0x557c57968540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c578c41e0 con 0x557c57968a80 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c57861390 con 0x557c5795f580 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557c578c01a0 con 0x557c5795fea0 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bdf7fe640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5bd80a3d20 0x7f5bd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5beed9d640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5bd80a3d20 0x7f5bd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43874/0 (socket says 172.21.15.66:43874) 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 -- 172.21.15.66:0/3249743363 learned_addr learned my addr 172.21.15.66:0/3249743363 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bdf7fe640 1 -- 172.21.15.66:0/186871814 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f5be80f2eb0 con 0x7f5bf019cfe0 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c5795f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bee59c640 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 0x557c57968540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 -- 172.21.15.66:0/3249743363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 msgr2=0x557c57968540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 0x557c57968540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 -- 172.21.15.66:0/3249743363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 msgr2=0x557c5795f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c5795f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 -- 172.21.15.66:0/3249743363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c576a07d0 con 0x557c57968a80 2024-04-21T09:15:18.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796ae70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5be4014990 tx=0x7f5be400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4988c435bdf169fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5beed9d640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5bd80a3d20 0x7f5bd80a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5be80be830 tx=0x7f5be8090570 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.477+0000 7f5bdcff9640 1 -- 172.21.15.66:0/3249743363 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40142d0 con 0x557c57968a80 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bdcff9640 1 -- 172.21.15.66:0/3249743363 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be400fc10 con 0x557c57968a80 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bdcff9640 1 -- 172.21.15.66:0/3249743363 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4012a20 con 0x557c57968a80 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3249743363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 msgr2=0x557c5796ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796ae70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5be4014990 tx=0x7f5be400c490 comp rx=0 tx=0).stop 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3249743363 shutdown_connections 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796ae70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 0x557c57968540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3249743363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c5795f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3249743363 >> 172.21.15.66:0/3249743363 conn(0x557c57959cb0 msgr2=0x557c5795a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3249743363 shutdown_connections 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.478+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3249743363 wait complete. 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 -- start start 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 Processor -- start 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795f580 0x557c57965aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795fea0 0x557c57965fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c57968a80 0x557c57966520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c5796b870 con 0x557c5795f580 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c5787d2f0 con 0x557c5795fea0 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557c5796b9f0 con 0x557c57968a80 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bf79f9740 10 client.0 _ll_register_callbacks cb 0x557c5788a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795f580 0x557c57965aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795f580 0x557c57965aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43886/0 (socket says 172.21.15.66:43886) 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 -- 172.21.15.66:0/584784094 learned_addr learned my addr 172.21.15.66:0/584784094 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bee59c640 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795fea0 0x557c57965fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 -- 172.21.15.66:0/584784094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c57968a80 msgr2=0x557c57966520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c57968a80 0x557c57966520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 -- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795fea0 msgr2=0x557c57965fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795fea0 0x557c57965fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.479+0000 7f5bedd9b640 1 -- 172.21.15.66:0/584784094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c575d63f0 con 0x557c5795f580 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795f580 0x557c57965aa0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5be4014960 tx=0x7f5be400fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61299ad06040dc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/584784094 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40142d0 con 0x557c5795f580 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/584784094 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f5be400f760 con 0x557c5795f580 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be400fc10 con 0x557c5795f580 2024-04-21T09:15:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/584784094 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5be4032050 con 0x557c5795f580 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 10 client.17529 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c575960f0 con 0x557c5795f580 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/584784094 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be400fc10 con 0x557c5795f580 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/584784094 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5be4012c30 con 0x557c5795f580 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 10 client.17529 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 20 client.17529 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579789d0 0x557c5797adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 4 client.17529 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.480+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c578d1c00 con 0x557c579789d0 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.481+0000 7f5bee59c640 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579789d0 0x557c5797adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.481+0000 7f5bee59c640 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579789d0 0x557c5797adc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4aa7f7eb8c4ba3 server_cookie=d2598d03210eb41f in_seq=0 out_seq=0 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.481+0000 7f5bd5ffb640 10 client.17529 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/584784094 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c578d1c00 con 0x557c579789d0 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bd5ffb640 10 client.17529 handle_command_reply: tid=0 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 client.17529 shutdown 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795f580 msgr2=0x557c57965aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795f580 0x557c57965aa0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5be4014960 tx=0x7f5be400fe30 comp rx=0 tx=0).stop 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 shutdown_connections 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579789d0 0x557c5797adc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c57968a80 0x557c57966520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795fea0 0x557c57965fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bd5ffb640 0 client.17529 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/584784094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795f580 0x557c57965aa0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 >> 172.21.15.66:0/584784094 conn(0x557c57959cb0 msgr2=0x557c5795ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 shutdown_connections 2024-04-21T09:15:18.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 1 -- 172.21.15.66:0/584784094 wait complete. 2024-04-21T09:15:18.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.482+0000 7f5bf79f9740 20 client.17529 trim_cache size 0 max 16384 2024-04-21T09:15:18.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 Processor -- start 2024-04-21T09:15:18.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 -- start start 2024-04-21T09:15:18.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795f580 0x557c579681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795fea0 0x557c57974760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c57968a80 0x557c5795b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01b1b70 con 0x557c5795fea0 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01af440 con 0x557c57968a80 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01ad2b0 con 0x557c5795f580 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795fea0 0x557c57974760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795fea0 0x557c57974760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43902/0 (socket says 172.21.15.66:43902) 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 -- 172.21.15.66:0/3250404674 learned_addr learned my addr 172.21.15.66:0/3250404674 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c57968a80 0x557c5795b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 -- 172.21.15.66:0/3250404674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795f580 msgr2=0x557c579681d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795f580 0x557c579681d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 -- 172.21.15.66:0/3250404674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c57968a80 msgr2=0x557c5795b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c57968a80 0x557c5795b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.500+0000 7f5bee59c640 1 -- 172.21.15.66:0/3250404674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5795aee0 con 0x557c5795fea0 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bee59c640 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795fea0 0x557c57974760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5be000b9d0 tx=0x7f5be0016430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c80239eef4abc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bd7fff640 1 -- 172.21.15.66:0/3250404674 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0016bf0 con 0x557c5795fea0 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bd7fff640 1 -- 172.21.15.66:0/3250404674 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be0016d90 con 0x557c5795fea0 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bd7fff640 1 -- 172.21.15.66:0/3250404674 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00146a0 con 0x557c5795fea0 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3250404674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795fea0 msgr2=0x557c57974760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795fea0 0x557c57974760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5be000b9d0 tx=0x7f5be0016430 comp rx=0 tx=0).stop 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3250404674 shutdown_connections 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c57968a80 0x557c5795b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c5795fea0 0x557c57974760 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/3250404674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795f580 0x557c579681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3250404674 >> 172.21.15.66:0/3250404674 conn(0x557c57959cb0 msgr2=0x557c579584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3250404674 shutdown_connections 2024-04-21T09:15:18.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.501+0000 7f5bf79f9740 1 -- 172.21.15.66:0/3250404674 wait complete. 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 -- start start 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 Processor -- start 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c57970d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 0x557c5796c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01ad900 con 0x557c57968a80 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01aed70 con 0x557c5795f580 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01adfd0 con 0x557c5795fea0 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bf79f9740 10 client.0 _ll_register_callbacks cb 0x7f5be400ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43904/0 (socket says 172.21.15.66:43904) 2024-04-21T09:15:18.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 -- 172.21.15.66:0/1671049133 learned_addr learned my addr 172.21.15.66:0/1671049133 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c57970d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 -- 172.21.15.66:0/1671049133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 msgr2=0x557c5796c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 0x557c5796c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 -- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 msgr2=0x557c57970d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c57970d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 -- 172.21.15.66:0/1671049133 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5796cc90 con 0x557c57968a80 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bedd9b640 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796c750 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005780 tx=0x7f5be4005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4020377705b335c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.502+0000 7f5bd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/1671049133 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4024040 con 0x557c57968a80 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/1671049133 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f5be400cb30 con 0x557c57968a80 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be0016d90 con 0x557c57968a80 2024-04-21T09:15:18.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/1671049133 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5be400cd30 con 0x557c57968a80 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 10 client.17535 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c579dea00 con 0x557c57968a80 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/1671049133 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be0016d90 con 0x557c57968a80 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/1671049133 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5be4012c30 con 0x557c57968a80 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 10 client.17535 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 20 client.17535 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579df090 0x557c579e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 4 client.17535 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c578cba70 con 0x557c579df090 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.503+0000 7f5bee59c640 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579df090 0x557c579e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bee59c640 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579df090 0x557c579e14b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1163f8cf2f5b4f server_cookie=6852eda097938526 in_seq=0 out_seq=0 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bd5ffb640 10 client.17535 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bd5ffb640 1 -- 172.21.15.66:0/1671049133 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557c578cba70 con 0x557c579df090 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bd5ffb640 10 client.17535 handle_command_reply: tid=0 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 client.17535 shutdown 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 msgr2=0x557c5796c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796c750 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005780 tx=0x7f5be4005690 comp rx=0 tx=0).stop 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 shutdown_connections 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x557c579df090 0x557c579e14b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c57968a80 0x557c5796c750 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bd5ffb640 0 client.17535 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c5795fea0 0x557c5796c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 --2- 172.21.15.66:0/1671049133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c5795f580 0x557c57970d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 >> 172.21.15.66:0/1671049133 conn(0x557c57959cb0 msgr2=0x557c57965a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 shutdown_connections 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 1 -- 172.21.15.66:0/1671049133 wait complete. 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.504+0000 7f5bf79f9740 20 client.17535 trim_cache size 0 max 16384 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:15:18.512 INFO:teuthology.orchestra.run.smithi066.stdout: "f96c406a-6355-43d8-8619-30402cc1d4ba": { 2024-04-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "f96c406a-6355-43d8-8619-30402cc1d4ba", 2024-04-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5bd80a3d20 msgr2=0x7f5bd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5bd80a3d20 0x7f5bd80a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5be80be830 tx=0x7f5be8090570 comp rx=0 tx=0).stop 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf019cfe0 msgr2=0x7f5bf019d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf019cfe0 0x7f5bf019d490 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5be8005e30 tx=0x7f5be8034760 comp rx=0 tx=0).stop 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 shutdown_connections 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f5bd80a3d20 0x7f5bd80a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf019cfe0 0x7f5bf019d490 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0107270 0x7f5bf0198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 --2- 172.21.15.66:0/186871814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf01021a0 0x7f5bf0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:18.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 >> 172.21.15.66:0/186871814 conn(0x7f5bf00793e0 msgr2=0x7f5bf00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:18.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 shutdown_connections 2024-04-21T09:15:18.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:18.505+0000 7f5bf4e06640 1 -- 172.21.15.66:0/186871814 wait complete. 2024-04-21T09:15:18.516 INFO:tasks.cephfs.filesystem:scrub status for tag:f96c406a-6355-43d8-8619-30402cc1d4ba - {'path': '/', 'tag': 'f96c406a-6355-43d8-8619-30402cc1d4ba', 'options': 'recursive,force'} 2024-04-21T09:15:19.640 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:19 smithi186 ceph-mon[31505]: pgmap v218: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:15:19.640 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:19 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:15:19.640 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:19 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:19.640 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:19 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:15:19.640 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:19 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:19.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:19 smithi083 ceph-mon[31413]: pgmap v218: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:15:19.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:19 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:15:19.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:19 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:19.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:19 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:15:19.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:19 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:19.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:19 smithi066 ceph-mon[27953]: pgmap v218: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:15:19.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:19 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:15:19.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:19 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:19.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:19 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:15:19.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:19 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:21.662 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:21 smithi083 ceph-mon[31413]: pgmap v219: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 232 B/s wr, 6 op/s 2024-04-21T09:15:21.662 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:21 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:15:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:21 smithi186 ceph-mon[31505]: pgmap v219: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 232 B/s wr, 6 op/s 2024-04-21T09:15:21.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:21 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:15:21.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:21 smithi066 ceph-mon[27953]: pgmap v219: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 232 B/s wr, 6 op/s 2024-04-21T09:15:21.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:21 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:15:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:23 smithi186 ceph-mon[31505]: pgmap v220: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-21T09:15:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:23 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:15:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:23 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:15:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:23 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:15:23.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:23 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:15:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:23 smithi083 ceph-mon[31413]: pgmap v220: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-21T09:15:23.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:23 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:15:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:23 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:15:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:23 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:15:23.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:23 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:15:23.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:23 smithi066 ceph-mon[27953]: pgmap v220: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-21T09:15:23.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:23 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:15:23.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:23 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:15:23.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:23 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:15:23.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:23 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:15:25.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:25 smithi186 ceph-mon[31505]: pgmap v221: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:15:25.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:25 smithi083 ceph-mon[31413]: pgmap v221: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:15:25.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:25 smithi066 ceph-mon[27953]: pgmap v221: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:15:27.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:27 smithi186 ceph-mon[31505]: pgmap v222: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 89 B/s wr, 6 op/s 2024-04-21T09:15:27.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:27 smithi083 ceph-mon[31413]: pgmap v222: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 89 B/s wr, 6 op/s 2024-04-21T09:15:27.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:27 smithi066 ceph-mon[27953]: pgmap v222: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 89 B/s wr, 6 op/s 2024-04-21T09:15:29.672 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:29 smithi066 ceph-mon[27953]: pgmap v223: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:15:29.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:29 smithi186 ceph-mon[31505]: pgmap v223: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:15:29.867 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:29 smithi083 ceph-mon[31413]: pgmap v223: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:15:31.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:31 smithi186 ceph-mon[31505]: pgmap v224: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:31.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:31 smithi083 ceph-mon[31413]: pgmap v224: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:31.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:31 smithi066 ceph-mon[27953]: pgmap v224: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:33.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:33 smithi186 ceph-mon[31505]: pgmap v225: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:33.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:33 smithi083 ceph-mon[31413]: pgmap v225: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:33.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:33 smithi066 ceph-mon[27953]: pgmap v225: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:35.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:35 smithi186 ceph-mon[31505]: pgmap v226: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:35.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:35 smithi083 ceph-mon[31413]: pgmap v226: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:35.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:35 smithi066 ceph-mon[27953]: pgmap v226: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:37.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:37 smithi186 ceph-mon[31505]: pgmap v227: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:37.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:37 smithi083 ceph-mon[31413]: pgmap v227: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:37.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:37 smithi066 ceph-mon[27953]: pgmap v227: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:39.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:39 smithi186 ceph-mon[31505]: pgmap v228: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:39.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:39 smithi083 ceph-mon[31413]: pgmap v228: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:39.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:39 smithi066 ceph-mon[27953]: pgmap v228: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:41.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:41 smithi186 ceph-mon[31505]: pgmap v229: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:41.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:41 smithi083 ceph-mon[31413]: pgmap v229: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:41.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:41 smithi066 ceph-mon[27953]: pgmap v229: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:43.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:43 smithi186 ceph-mon[31505]: pgmap v230: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:43.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:43 smithi083 ceph-mon[31413]: pgmap v230: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:43.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:43 smithi066 ceph-mon[27953]: pgmap v230: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:45.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:45 smithi186 ceph-mon[31505]: pgmap v231: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:45.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:45 smithi083 ceph-mon[31413]: pgmap v231: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:45.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:45 smithi066 ceph-mon[27953]: pgmap v231: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:47.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:47 smithi186 ceph-mon[31505]: pgmap v232: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:47.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:47 smithi083 ceph-mon[31413]: pgmap v232: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:47.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:47 smithi066 ceph-mon[27953]: pgmap v232: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:48.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:15:48.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 -- 172.21.15.66:0/1198839874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c103ae0 msgr2=0x7f710c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 --2- 172.21.15.66:0/1198839874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c103ae0 0x7f710c10bfc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009900 tx=0x7f70f402bcd0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 -- 172.21.15.66:0/1198839874 shutdown_connections 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 --2- 172.21.15.66:0/1198839874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f710c10c590 0x7f710c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 --2- 172.21.15.66:0/1198839874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c103ae0 0x7f710c10bfc0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 --2- 172.21.15.66:0/1198839874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f710c1031c0 0x7f710c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 -- 172.21.15.66:0/1198839874 >> 172.21.15.66:0/1198839874 conn(0x7f710c077f70 msgr2=0x7f710c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 -- 172.21.15.66:0/1198839874 shutdown_connections 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 -- 172.21.15.66:0/1198839874 wait complete. 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 Processor -- start 2024-04-21T09:15:48.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.635+0000 7f7110c74640 1 -- start start 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7110c74640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f710c1031c0 0x7f710c100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7110c74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f710c103ae0 0x7f710c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7110c74640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c10c590 0x7f710c101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7110c74640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1121b0 con 0x7f710c10c590 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7110c74640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f710c112030 con 0x7f710c103ae0 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7110c74640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f710c112330 con 0x7f710c1031c0 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c10c590 0x7f710c101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c10c590 0x7f710c101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59934/0 (socket says 172.21.15.66:59934) 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 -- 172.21.15.66:0/1070343895 learned_addr learned my addr 172.21.15.66:0/1070343895 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710a59c640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f710c1031c0 0x7f710c100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7103fff640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f710c103ae0 0x7f710c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 -- 172.21.15.66:0/1070343895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f710c1031c0 msgr2=0x7f710c100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f710c1031c0 0x7f710c100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 -- 172.21.15.66:0/1070343895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f710c103ae0 msgr2=0x7f710c101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f710c103ae0 0x7f710c101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 -- 172.21.15.66:0/1070343895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c1a6c10 con 0x7f710c10c590 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f7103fff640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f710c103ae0 0x7f710c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.636+0000 7f710ad9d640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c10c590 0x7f710c101890 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9d0 tx=0x7f710400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.637+0000 7f71037fe640 1 -- 172.21.15.66:0/1070343895 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710400cd80 con 0x7f710c10c590 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.637+0000 7f71037fe640 1 -- 172.21.15.66:0/1070343895 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7104005020 con 0x7f710c10c590 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.637+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c1a6f00 con 0x7f710c10c590 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.637+0000 7f71037fe640 1 -- 172.21.15.66:0/1070343895 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040116d0 con 0x7f710c10c590 2024-04-21T09:15:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.637+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c1a73d0 con 0x7f710c10c590 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.639+0000 7f71037fe640 1 -- 172.21.15.66:0/1070343895 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f71040051d0 con 0x7f710c10c590 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.639+0000 7f711386c740 1 Processor -- start 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.639+0000 7f711386c740 1 -- start start 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f711386c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f711386c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f711386c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f711386c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55853112f1e0 con 0x5585311caea0 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f711386c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5585310cc390 con 0x5585311d3a80 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f711386c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55853112b1a0 con 0x5585311ca580 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311d3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59940/0 (socket says 172.21.15.66:59940) 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 -- 172.21.15.66:0/2117664047 learned_addr learned my addr 172.21.15.66:0/2117664047 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f7103fff640 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f71037fe640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f70f80a3d50 0x7f70f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f7103fff640 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 -- 172.21.15.66:0/2117664047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 msgr2=0x5585311ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710ad9d640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f70f80a3d50 0x7f70f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 -- 172.21.15.66:0/2117664047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 msgr2=0x5585311d5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311d5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 -- 172.21.15.66:0/2117664047 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558530f0b7d0 con 0x5585311caea0 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f7103fff640 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f71037fe640 1 -- 172.21.15.66:0/1070343895 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f71040d0dc0 con 0x7f710c10c590 2024-04-21T09:15:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.640+0000 7f710a59c640 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311d3540 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009cf0 tx=0x7f70fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28077a6765d0dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f7100ff9640 1 -- 172.21.15.66:0/2117664047 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004230 con 0x5585311caea0 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f710ad9d640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f70f80a3d50 0x7f70f80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f710409c8c0 tx=0x7f7104087040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f7100ff9640 1 -- 172.21.15.66:0/2117664047 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc0043d0 con 0x5585311caea0 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f7100ff9640 1 -- 172.21.15.66:0/2117664047 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc019880 con 0x5585311caea0 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f711386c740 1 -- 172.21.15.66:0/2117664047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 msgr2=0x5585311d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f711386c740 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311d3540 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009cf0 tx=0x7f70fc011810 comp rx=0 tx=0).stop 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f711386c740 1 -- 172.21.15.66:0/2117664047 shutdown_connections 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f711386c740 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f711386c740 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311d3540 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f711386c740 1 --2- 172.21.15.66:0/2117664047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.641+0000 7f711386c740 1 -- 172.21.15.66:0/2117664047 >> 172.21.15.66:0/2117664047 conn(0x5585311c4cb0 msgr2=0x5585311c5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 -- 172.21.15.66:0/2117664047 shutdown_connections 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 -- 172.21.15.66:0/2117664047 wait complete. 2024-04-21T09:15:48.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 -- start start 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 Processor -- start 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311d3a80 0x5585311d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5585311d6870 con 0x5585311d3a80 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5585310e82f0 con 0x5585311caea0 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5585311d69f0 con 0x5585311ca580 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f711386c740 10 client.0 _ll_register_callbacks cb 0x5585310f5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f7103fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311d3a80 0x5585311d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f7103fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311d3a80 0x5585311d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59948/0 (socket says 172.21.15.66:59948) 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.642+0000 7f7103fff640 1 -- 172.21.15.66:0/2577395186 learned_addr learned my addr 172.21.15.66:0/2577395186 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f7103fff640 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f710a59c640 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f7103fff640 1 -- 172.21.15.66:0/2577395186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 msgr2=0x5585311d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f7103fff640 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f7103fff640 1 -- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 msgr2=0x5585311d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f7103fff640 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f7103fff640 1 -- 172.21.15.66:0/2577395186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558530e413f0 con 0x5585311d3a80 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f7103fff640 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311d3a80 0x5585311d1520 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f70f402c210 tx=0x7f70f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba572a8877f8f149 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f70f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f70f1ffb640 1 -- 172.21.15.66:0/2577395186 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f402cd80 con 0x5585311d3a80 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f70f1ffb640 1 -- 172.21.15.66:0/2577395186 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f70f40035f0 con 0x5585311d3a80 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc0043d0 con 0x5585311d3a80 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f70f1ffb640 1 -- 172.21.15.66:0/2577395186 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70f4051050 con 0x5585311d3a80 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f711386c740 10 client.17547 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.643+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558530e010f0 con 0x5585311d3a80 2024-04-21T09:15:48.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f70f1ffb640 1 -- 172.21.15.66:0/2577395186 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc0043d0 con 0x5585311d3a80 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f70f1ffb640 1 -- 172.21.15.66:0/2577395186 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70f4043570 con 0x5585311d3a80 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f711386c740 10 client.17547 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f711386c740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f711386c740 20 client.17547 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f711386c740 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5585311e39c0 0x5585311e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f711386c740 4 client.17547 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55853113cc00 con 0x5585311e39c0 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.644+0000 7f710a59c640 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5585311e39c0 0x5585311e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f70f1ffb640 10 client.17547 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f710a59c640 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5585311e39c0 0x5585311e5dd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb7eceb8b69bcee server_cookie=b44dd8d0747623e8 in_seq=0 out_seq=0 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f70f1ffb640 1 -- 172.21.15.66:0/2577395186 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55853113cc00 con 0x5585311e39c0 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f70f1ffb640 10 client.17547 handle_command_reply: tid=0 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 client.17547 shutdown 2024-04-21T09:15:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311d3a80 msgr2=0x5585311d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311d3a80 0x5585311d1520 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f70f402c210 tx=0x7f70f4009260 comp rx=0 tx=0).stop 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 shutdown_connections 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5585311e39c0 0x5585311e5dd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311d3a80 0x5585311d1520 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 --2- 172.21.15.66:0/2577395186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f70f1ffb640 0 client.17547 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.645+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 >> 172.21.15.66:0/2577395186 conn(0x5585311c4cb0 msgr2=0x5585311c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.646+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 shutdown_connections 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.646+0000 7f711386c740 1 -- 172.21.15.66:0/2577395186 wait complete. 2024-04-21T09:15:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.646+0000 7f711386c740 20 client.17547 trim_cache size 0 max 16384 2024-04-21T09:15:48.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.663+0000 7f711386c740 1 Processor -- start 2024-04-21T09:15:48.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.663+0000 7f711386c740 1 -- start start 2024-04-21T09:15:48.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f711386c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f711386c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f711386c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f711386c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1b3f50 con 0x5585311caea0 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f711386c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1b1820 con 0x5585311d3a80 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f711386c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1af690 con 0x5585311ca580 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311df730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59958/0 (socket says 172.21.15.66:59958) 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 -- 172.21.15.66:0/3650313025 learned_addr learned my addr 172.21.15.66:0/3650313025 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f7103fff640 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 -- 172.21.15.66:0/3650313025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 msgr2=0x5585311d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 -- 172.21.15.66:0/3650313025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 msgr2=0x5585311cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 -- 172.21.15.66:0/3650313025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585311cacd0 con 0x5585311caea0 2024-04-21T09:15:48.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f710a59c640 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311df730 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009cf0 tx=0x7f70fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e389c539469d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f70f3fff640 1 -- 172.21.15.66:0/3650313025 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc031070 con 0x5585311caea0 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f70f3fff640 1 -- 172.21.15.66:0/3650313025 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc02b850 con 0x5585311caea0 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.664+0000 7f70f3fff640 1 -- 172.21.15.66:0/3650313025 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc005610 con 0x5585311caea0 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 -- 172.21.15.66:0/3650313025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 msgr2=0x5585311df730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311df730 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009cf0 tx=0x7f70fc011810 comp rx=0 tx=0).stop 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 -- 172.21.15.66:0/3650313025 shutdown_connections 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311d3a80 0x5585311cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311caea0 0x5585311df730 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 --2- 172.21.15.66:0/3650313025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311ca580 0x5585311d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 -- 172.21.15.66:0/3650313025 >> 172.21.15.66:0/3650313025 conn(0x5585311c4cb0 msgr2=0x5585311c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 -- 172.21.15.66:0/3650313025 shutdown_connections 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 -- 172.21.15.66:0/3650313025 wait complete. 2024-04-21T09:15:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 -- start start 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 Processor -- start 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.665+0000 7f711386c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311ca580 0x5585311cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311d3a80 0x5585311ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1afce0 con 0x5585311ca580 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1b1150 con 0x5585311caea0 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1b03b0 con 0x5585311d3a80 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 10 client.0 _ll_register_callbacks cb 0x7f70f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311ca580 0x5585311cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311ca580 0x5585311cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59960/0 (socket says 172.21.15.66:59960) 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 -- 172.21.15.66:0/1343684278 learned_addr learned my addr 172.21.15.66:0/1343684278 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f710a59c640 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 -- 172.21.15.66:0/1343684278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311d3a80 msgr2=0x5585311ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311d3a80 0x5585311ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 -- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 msgr2=0x5585311cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 -- 172.21.15.66:0/1343684278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558531248fc0 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f7103fff640 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311ca580 0x5585311cd860 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009230 tx=0x7f70f402c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146ceea8259d4b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f70f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f70f1ffb640 1 -- 172.21.15.66:0/1343684278 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4059030 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f70f1ffb640 1 -- 172.21.15.66:0/1343684278 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f70f4043660 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc02b850 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f70f1ffb640 1 -- 172.21.15.66:0/1343684278 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70f4043db0 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 10 client.17553 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.666+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55853124a2b0 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f70f1ffb640 1 -- 172.21.15.66:0/1343684278 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc02b850 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f70f1ffb640 1 -- 172.21.15.66:0/1343684278 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70f404d070 con 0x5585311ca580 2024-04-21T09:15:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f711386c740 10 client.17553 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f711386c740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f711386c740 20 client.17553 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f711386c740 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55853124aa30 0x55853124ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f711386c740 4 client.17553 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558531136a70 con 0x55853124aa30 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f710a59c640 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55853124aa30 0x55853124ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f70f1ffb640 10 client.17553 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.667+0000 7f710a59c640 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55853124aa30 0x55853124ce50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb12b133c417f42d server_cookie=1bd02f86127fee90 in_seq=0 out_seq=0 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f70f1ffb640 1 -- 172.21.15.66:0/1343684278 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558531136a70 con 0x55853124aa30 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f70f1ffb640 10 client.17553 handle_command_reply: tid=0 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 client.17553 shutdown 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311ca580 msgr2=0x5585311cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311ca580 0x5585311cd860 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009230 tx=0x7f70f402c880 comp rx=0 tx=0).stop 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 shutdown_connections 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55853124aa30 0x55853124ce50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585311d3a80 0x5585311ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585311caea0 0x5585311cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f70f1ffb640 0 client.17553 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 --2- 172.21.15.66:0/1343684278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585311ca580 0x5585311cd860 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 >> 172.21.15.66:0/1343684278 conn(0x5585311c4cb0 msgr2=0x5585311c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 shutdown_connections 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 1 -- 172.21.15.66:0/1343684278 wait complete. 2024-04-21T09:15:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f711386c740 20 client.17553 trim_cache size 0 max 16384 2024-04-21T09:15:48.675 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:15:48.675 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:15:48.675 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:15:48.675 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:15:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f70f80a3d50 msgr2=0x7f70f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f7110c74640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f70f80a3d50 0x7f70f80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f710409c8c0 tx=0x7f7104087040 comp rx=0 tx=0).stop 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c10c590 msgr2=0x7f710c101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.668+0000 7f7110c74640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c10c590 0x7f710c101890 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9d0 tx=0x7f710400ee90 comp rx=0 tx=0).stop 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 shutdown_connections 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f70f80a3d50 0x7f70f80a6210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f710c10c590 0x7f710c101890 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f710c103ae0 0x7f710c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 --2- 172.21.15.66:0/1070343895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f710c1031c0 0x7f710c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 >> 172.21.15.66:0/1070343895 conn(0x7f710c077f70 msgr2=0x7f710c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 shutdown_connections 2024-04-21T09:15:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.669+0000 7f7110c74640 1 -- 172.21.15.66:0/1070343895 wait complete. 2024-04-21T09:15:48.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:15:48.679 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:15:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.781+0000 7f8c926a8640 1 -- 172.21.15.66:0/842998202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 msgr2=0x7f8c8c101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.781+0000 7f8c926a8640 1 --2- 172.21.15.66:0/842998202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c101620 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009930 tx=0x7f8c8002bc30 comp rx=0 tx=0).stop 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 -- 172.21.15.66:0/842998202 shutdown_connections 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 --2- 172.21.15.66:0/842998202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c8c101b60 0x7f8c8c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 --2- 172.21.15.66:0/842998202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c101620 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 --2- 172.21.15.66:0/842998202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c8c105e30 0x7f8c8c106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 -- 172.21.15.66:0/842998202 >> 172.21.15.66:0/842998202 conn(0x7f8c8c077e30 msgr2=0x7f8c8c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 -- 172.21.15.66:0/842998202 shutdown_connections 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 -- 172.21.15.66:0/842998202 wait complete. 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 Processor -- start 2024-04-21T09:15:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 -- start start 2024-04-21T09:15:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.782+0000 7f8c926a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c926a8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c8c101b60 0x7f8c8c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c926a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c8c105e30 0x7f8c8c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c926a8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c112200 con 0x7f8c8c1011a0 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c926a8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c112080 con 0x7f8c8c105e30 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c926a8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c112380 con 0x7f8c8c101b60 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59988/0 (socket says 172.21.15.66:59988) 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 -- 172.21.15.66:0/1498866005 learned_addr learned my addr 172.21.15.66:0/1498866005 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c90c45640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c8c105e30 0x7f8c8c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8b7fe640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c8c101b60 0x7f8c8c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 -- 172.21.15.66:0/1498866005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c8c101b60 msgr2=0x7f8c8c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c8c101b60 0x7f8c8c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 -- 172.21.15.66:0/1498866005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c8c105e30 msgr2=0x7f8c8c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c8c105e30 0x7f8c8c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 -- 172.21.15.66:0/1498866005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c19d710 con 0x7f8c8c1011a0 2024-04-21T09:15:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c8bfff640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c198720 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00e9f0 tx=0x7f8c7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c897fa640 1 -- 172.21.15.66:0/1498866005 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00cd80 con 0x7f8c8c1011a0 2024-04-21T09:15:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c897fa640 1 -- 172.21.15.66:0/1498866005 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c0108e0 con 0x7f8c8c1011a0 2024-04-21T09:15:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c19da00 con 0x7f8c8c1011a0 2024-04-21T09:15:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c897fa640 1 -- 172.21.15.66:0/1498866005 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c018980 con 0x7f8c8c1011a0 2024-04-21T09:15:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.783+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c1a5290 con 0x7f8c8c1011a0 2024-04-21T09:15:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.784+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8c068de0 con 0x7f8c8c1011a0 2024-04-21T09:15:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.786+0000 7f8c897fa640 1 -- 172.21.15.66:0/1498866005 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8c7c010430 con 0x7f8c8c1011a0 2024-04-21T09:15:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.786+0000 7f8c897fa640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c740a3d20 0x7f8c740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.786+0000 7f8c8b7fe640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c740a3d20 0x7f8c740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.786+0000 7f8c897fa640 1 -- 172.21.15.66:0/1498866005 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f8c7c0d1360 con 0x7f8c8c1011a0 2024-04-21T09:15:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.787+0000 7f8c8b7fe640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c740a3d20 0x7f8c740a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c80005ab0 tx=0x7f8c80005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:48.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:48.790+0000 7f8c897fa640 1 -- 172.21.15.66:0/1498866005 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c7c09cfe0 con 0x7f8c8c1011a0 2024-04-21T09:15:49.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.014+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c8c199a70 con 0x7f8c8c1011a0 2024-04-21T09:15:49.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.015+0000 7f8c897fa640 1 -- 172.21.15.66:0/1498866005 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f8c7c0a12e0 con 0x7f8c8c1011a0 2024-04-21T09:15:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:15:49.015 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:15:49.016 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c740a3d20 msgr2=0x7f8c740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c740a3d20 0x7f8c740a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c80005ab0 tx=0x7f8c80005fd0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 msgr2=0x7f8c8c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c198720 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00e9f0 tx=0x7f8c7c00eeb0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 shutdown_connections 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c740a3d20 0x7f8c740a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c8c105e30 0x7f8c8c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c8c101b60 0x7f8c8c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 --2- 172.21.15.66:0/1498866005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8c1011a0 0x7f8c8c198720 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 >> 172.21.15.66:0/1498866005 conn(0x7f8c8c077e30 msgr2=0x7f8c8c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 shutdown_connections 2024-04-21T09:15:49.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.017+0000 7f8c926a8640 1 -- 172.21.15.66:0/1498866005 wait complete. 2024-04-21T09:15:49.028 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:15:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 -- 172.21.15.66:0/2168183994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a81010b0 msgr2=0x7f52a8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2168183994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a81010b0 0x7f52a8106be0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5294009930 tx=0x7f529402bc80 comp rx=0 tx=0).stop 2024-04-21T09:15:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 -- 172.21.15.66:0/2168183994 shutdown_connections 2024-04-21T09:15:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2168183994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a8107310 0x7f52a810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2168183994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a81010b0 0x7f52a8106be0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2168183994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a8100700 0x7f52a8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 -- 172.21.15.66:0/2168183994 >> 172.21.15.66:0/2168183994 conn(0x7f52a8077410 msgr2=0x7f52a80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 -- 172.21.15.66:0/2168183994 shutdown_connections 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.133+0000 7f52aeac8640 1 -- 172.21.15.66:0/2168183994 wait complete. 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 Processor -- start 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 -- start start 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a8100700 0x7f52a8101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a8107310 0x7f52a8101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81020f0 0x7f52a81a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81121b0 con 0x7f52a8100700 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8112030 con 0x7f52a81020f0 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52aeac8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8112330 con 0x7f52a8107310 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a8100700 0x7f52a8101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a8100700 0x7f52a8101670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60018/0 (socket says 172.21.15.66:60018) 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 -- 172.21.15.66:0/2357224132 learned_addr learned my addr 172.21.15.66:0/2357224132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ad065640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81020f0 0x7f52a81a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 -- 172.21.15.66:0/2357224132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a8107310 msgr2=0x7f52a8101bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a8107310 0x7f52a8101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 -- 172.21.15.66:0/2357224132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81020f0 msgr2=0x7f52a81a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81020f0 0x7f52a81a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.134+0000 7f52ac864640 1 -- 172.21.15.66:0/2357224132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a81a7080 con 0x7f52a8100700 2024-04-21T09:15:49.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.135+0000 7f52ac864640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a8100700 0x7f52a8101670 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f529c00c970 tx=0x7f529c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.135+0000 7f52a5ffb640 1 -- 172.21.15.66:0/2357224132 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c007bb0 con 0x7f52a8100700 2024-04-21T09:15:49.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.135+0000 7f52a5ffb640 1 -- 172.21.15.66:0/2357224132 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c007d50 con 0x7f52a8100700 2024-04-21T09:15:49.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.135+0000 7f52a5ffb640 1 -- 172.21.15.66:0/2357224132 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c00f450 con 0x7f52a8100700 2024-04-21T09:15:49.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.135+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a81a7370 con 0x7f52a8100700 2024-04-21T09:15:49.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.135+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a81a7870 con 0x7f52a8100700 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.137+0000 7f52a5ffb640 1 -- 172.21.15.66:0/2357224132 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f529c004f20 con 0x7f52a8100700 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.137+0000 7f52b1798740 1 Processor -- start 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.137+0000 7f52b1798740 1 -- start start 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52a5ffb640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52980a3d20 0x7f52980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52b1798740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf69688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52b1798740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6971430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52b1798740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6973e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52b1798740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cbf68cd1f0 con 0x55cbf69684d0 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52b1798740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbf686a4b0 con 0x55cbf6968e80 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52b1798740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cbf68c91b0 con 0x55cbf6971a00 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf69688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ad065640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52980a3d20 0x7f52980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52a5ffb640 1 -- 172.21.15.66:0/2357224132 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f529c0d0a00 con 0x7f52a8100700 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf69688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60026/0 (socket says 172.21.15.66:60026) 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 -- 172.21.15.66:0/17790975 learned_addr learned my addr 172.21.15.66:0/17790975 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52a7fff640 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6971430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ad065640 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6973e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 -- 172.21.15.66:0/17790975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 msgr2=0x55cbf6973e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6973e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 -- 172.21.15.66:0/17790975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 msgr2=0x55cbf6971430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6971430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 -- 172.21.15.66:0/17790975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf685c1b0 con 0x55cbf69684d0 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ac864640 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf69688b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f529c098a30 tx=0x7f529c09bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad8dae0fab334e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.138+0000 7f52ad065640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52980a3d20 0x7f52980a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009370 tx=0x7f52a0009a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f5292ffd640 1 -- 172.21.15.66:0/17790975 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c01d070 con 0x55cbf69684d0 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f5292ffd640 1 -- 172.21.15.66:0/17790975 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c09ccd0 con 0x55cbf69684d0 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f5292ffd640 1 -- 172.21.15.66:0/17790975 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c0ce000 con 0x55cbf69684d0 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 -- 172.21.15.66:0/17790975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 msgr2=0x55cbf69688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf69688b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f529c098a30 tx=0x7f529c09bf30 comp rx=0 tx=0).stop 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 -- 172.21.15.66:0/17790975 shutdown_connections 2024-04-21T09:15:49.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6973e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6971430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 --2- 172.21.15.66:0/17790975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf69688b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 -- 172.21.15.66:0/17790975 >> 172.21.15.66:0/17790975 conn(0x55cbf6962ba0 msgr2=0x55cbf6962f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 -- 172.21.15.66:0/17790975 shutdown_connections 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.139+0000 7f52b1798740 1 -- 172.21.15.66:0/17790975 wait complete. 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 -- start start 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 Processor -- start 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf69684d0 0x55cbf6971320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf696e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf6971a00 0x55cbf696ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cbf6974970 con 0x55cbf6971a00 2024-04-21T09:15:49.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbf6886500 con 0x55cbf6968e80 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cbf6974af0 con 0x55cbf69684d0 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52b1798740 10 client.0 _ll_register_callbacks cb 0x55cbf6854b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52ad065640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf6971a00 0x55cbf696ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52ad065640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf6971a00 0x55cbf696ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60030/0 (socket says 172.21.15.66:60030) 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52ad065640 1 -- 172.21.15.66:0/1757853399 learned_addr learned my addr 172.21.15.66:0/1757853399 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52a7fff640 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf696e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.140+0000 7f52ac864640 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf69684d0 0x55cbf6971320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52ad065640 1 -- 172.21.15.66:0/1757853399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf69684d0 msgr2=0x55cbf6971320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52ad065640 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf69684d0 0x55cbf6971320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52ad065640 1 -- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 msgr2=0x55cbf696e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52ad065640 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf696e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52ad065640 1 -- 172.21.15.66:0/1757853399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf65df3f0 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52ad065640 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf6971a00 0x55cbf696ec50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f52a0014fd0 tx=0x7f52a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8665f72f0a41b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f5287fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f5287fff640 1 -- 172.21.15.66:0/1757853399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0024070 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f5287fff640 1 -- 172.21.15.66:0/1757853399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f52a0029460 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c09ccd0 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f5287fff640 1 -- 172.21.15.66:0/1757853399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52a0032050 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52b1798740 10 client.17571 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbf66a97d0 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f5287fff640 1 -- 172.21.15.66:0/1757853399 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c09ccd0 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.141+0000 7f5287fff640 1 -- 172.21.15.66:0/1757853399 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f52a000f3c0 con 0x55cbf6971a00 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52b1798740 10 client.17571 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:49.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52b1798740 10 client.17571 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52b1798740 20 client.17571 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52b1798740 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf6981c10 0x55cbf6984040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52b1798740 4 client.17571 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf686c930 con 0x55cbf6981c10 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52ac864640 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf6981c10 0x55cbf6984040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f5287fff640 10 client.17571 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.142+0000 7f52ac864640 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf6981c10 0x55cbf6984040 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a19765052e823aa server_cookie=d246099e1f2e09d9 in_seq=0 out_seq=0 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f5287fff640 1 -- 172.21.15.66:0/1757853399 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbf686c930 con 0x55cbf6981c10 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f5287fff640 10 client.17571 handle_command_reply: tid=0 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 client.17571 shutdown 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf6971a00 msgr2=0x55cbf696ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf6971a00 0x55cbf696ec50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f52a0014fd0 tx=0x7f52a0004ab0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 shutdown_connections 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf6981c10 0x55cbf6984040 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf6971a00 0x55cbf696ec50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf696e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f5287fff640 0 client.17571 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 --2- 172.21.15.66:0/1757853399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf69684d0 0x55cbf6971320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 >> 172.21.15.66:0/1757853399 conn(0x55cbf6962ba0 msgr2=0x55cbf69648f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 shutdown_connections 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 1 -- 172.21.15.66:0/1757853399 wait complete. 2024-04-21T09:15:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.143+0000 7f52b1798740 20 client.17571 trim_cache size 0 max 16384 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.156+0000 7f52b1798740 1 Processor -- start 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.156+0000 7f52b1798740 1 -- start start 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52b1798740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52b1798740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6961a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52b1798740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6970de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52b1798740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b3fe0 con 0x55cbf69684d0 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52b1798740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b18b0 con 0x55cbf6968e80 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52b1798740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81af720 con 0x55cbf6971a00 2024-04-21T09:15:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60042/0 (socket says 172.21.15.66:60042) 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 -- 172.21.15.66:0/3655189627 learned_addr learned my addr 172.21.15.66:0/3655189627 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ad065640 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6970de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52a7fff640 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6961a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 -- 172.21.15.66:0/3655189627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 msgr2=0x55cbf6970de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6970de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 -- 172.21.15.66:0/3655189627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 msgr2=0x55cbf6961a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6961a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 -- 172.21.15.66:0/3655189627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf6967560 con 0x55cbf69684d0 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52ac864640 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697d830 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f529c0ce380 tx=0x7f529c004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c4d3edcb5a867b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52927fc640 1 -- 172.21.15.66:0/3655189627 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c01d040 con 0x55cbf69684d0 2024-04-21T09:15:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52927fc640 1 -- 172.21.15.66:0/3655189627 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0046c0 con 0x55cbf69684d0 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.157+0000 7f52927fc640 1 -- 172.21.15.66:0/3655189627 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c005210 con 0x55cbf69684d0 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 -- 172.21.15.66:0/3655189627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 msgr2=0x55cbf697d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697d830 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f529c0ce380 tx=0x7f529c004980 comp rx=0 tx=0).stop 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 -- 172.21.15.66:0/3655189627 shutdown_connections 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6970de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf6961a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 --2- 172.21.15.66:0/3655189627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697d830 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 -- 172.21.15.66:0/3655189627 >> 172.21.15.66:0/3655189627 conn(0x55cbf6962ba0 msgr2=0x55cbf6961370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 -- 172.21.15.66:0/3655189627 shutdown_connections 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 -- 172.21.15.66:0/3655189627 wait complete. 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 -- start start 2024-04-21T09:15:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.158+0000 7f52b1798740 1 Processor -- start 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf69754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6975a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81afd70 con 0x55cbf69684d0 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b11e0 con 0x55cbf6968e80 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b0440 con 0x55cbf6971a00 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 10 client.0 _ll_register_callbacks cb 0x7f52a0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60044/0 (socket says 172.21.15.66:60044) 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 -- 172.21.15.66:0/139665038 learned_addr learned my addr 172.21.15.66:0/139665038 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ad065640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6975a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52a7fff640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf69754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 -- 172.21.15.66:0/139665038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 msgr2=0x55cbf6975a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6975a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 -- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 msgr2=0x55cbf69754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf69754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 -- 172.21.15.66:0/139665038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf6976280 con 0x55cbf69684d0 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52a7fff640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf69754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ad065640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6975a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52ac864640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697a180 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f529c0058c0 tx=0x7f529c00a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8054ce787e2fb4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f5287fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f5287fff640 1 -- 172.21.15.66:0/139665038 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c0b4070 con 0x55cbf69684d0 2024-04-21T09:15:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f5287fff640 1 -- 172.21.15.66:0/139665038 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f529c098c40 con 0x55cbf69684d0 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.159+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c0046c0 con 0x55cbf69684d0 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f5287fff640 1 -- 172.21.15.66:0/139665038 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f529c01d070 con 0x55cbf69684d0 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 10 client.17577 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbf69e8210 con 0x55cbf69684d0 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f5287fff640 1 -- 172.21.15.66:0/139665038 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0046c0 con 0x55cbf69684d0 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f5287fff640 1 -- 172.21.15.66:0/139665038 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f529c085b10 con 0x55cbf69684d0 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 10 client.17577 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 10 client.17577 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 20 client.17577 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf69e8440 0x55cbf69ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 4 client.17577 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:15:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbf68dac10 con 0x55cbf69e8440 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.160+0000 7f52a7fff640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf69e8440 0x55cbf69ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f5287fff640 10 client.17577 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52a7fff640 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf69e8440 0x55cbf69ea830 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89f5586bed2d2c46 server_cookie=ee3ef2739446f667 in_seq=0 out_seq=0 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f5287fff640 1 -- 172.21.15.66:0/139665038 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbf68dac10 con 0x55cbf69e8440 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f5287fff640 10 client.17577 handle_command_reply: tid=0 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 client.17577 shutdown 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 msgr2=0x55cbf697a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697a180 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f529c0058c0 tx=0x7f529c00a530 comp rx=0 tx=0).stop 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 shutdown_connections 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55cbf69e8440 0x55cbf69ea830 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbf6971a00 0x55cbf6975a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbf6968e80 0x55cbf69754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 --2- 172.21.15.66:0/139665038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbf69684d0 0x55cbf697a180 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f5287fff640 0 client.17577 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:15:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 >> 172.21.15.66:0/139665038 conn(0x55cbf6962ba0 msgr2=0x55cbf6963f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 shutdown_connections 2024-04-21T09:15:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 1 -- 172.21.15.66:0/139665038 wait complete. 2024-04-21T09:15:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.161+0000 7f52b1798740 20 client.17577 trim_cache size 0 max 16384 2024-04-21T09:15:49.169 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:15:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52980a3d20 msgr2=0x7f52980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52980a3d20 0x7f52980a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009370 tx=0x7f52a0009a40 comp rx=0 tx=0).stop 2024-04-21T09:15:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a8100700 msgr2=0x7f52a8101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a8100700 0x7f52a8101670 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f529c00c970 tx=0x7f529c00ce30 comp rx=0 tx=0).stop 2024-04-21T09:15:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 shutdown_connections 2024-04-21T09:15:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f52980a3d20 0x7f52980a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81020f0 0x7f52a81a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a8107310 0x7f52a8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 --2- 172.21.15.66:0/2357224132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52a8100700 0x7f52a8101670 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 >> 172.21.15.66:0/2357224132 conn(0x7f52a8077410 msgr2=0x7f52a80fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 shutdown_connections 2024-04-21T09:15:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.162+0000 7f52aeac8640 1 -- 172.21.15.66:0/2357224132 wait complete. 2024-04-21T09:15:49.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:15:49.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 -- 172.21.15.66:0/2343086191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 msgr2=0x7f8c1c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 --2- 172.21.15.66:0/2343086191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c101480 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009900 tx=0x7f8c1002bc30 comp rx=0 tx=0).stop 2024-04-21T09:15:49.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 -- 172.21.15.66:0/2343086191 shutdown_connections 2024-04-21T09:15:49.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 --2- 172.21.15.66:0/2343086191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c106170 0x7f8c1c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 --2- 172.21.15.66:0/2343086191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c1c101a50 0x7f8c1c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 --2- 172.21.15.66:0/2343086191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c101480 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 -- 172.21.15.66:0/2343086191 >> 172.21.15.66:0/2343086191 conn(0x7f8c1c077eb0 msgr2=0x7f8c1c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 -- 172.21.15.66:0/2343086191 shutdown_connections 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.276+0000 7f8c230e0640 1 -- 172.21.15.66:0/2343086191 wait complete. 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 Processor -- start 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 -- start start 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c1c106170 0x7f8c1c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c1035b0 0x7f8c1c1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1110b0 con 0x7f8c1c1010a0 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c110f30 con 0x7f8c1c106170 2024-04-21T09:15:49.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c230e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c111230 con 0x7f8c1c1035b0 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60062/0 (socket says 172.21.15.66:60062) 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 -- 172.21.15.66:0/3801212538 learned_addr learned my addr 172.21.15.66:0/3801212538 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c1c106170 0x7f8c1c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c2167d640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c1035b0 0x7f8c1c1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 -- 172.21.15.66:0/3801212538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c1035b0 msgr2=0x7f8c1c1a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c1035b0 0x7f8c1c1a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 -- 172.21.15.66:0/3801212538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c1c106170 msgr2=0x7f8c1c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c1c106170 0x7f8c1c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 -- 172.21.15.66:0/3801212538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c1a7010 con 0x7f8c1c1010a0 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.277+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c102b30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8c10005940 tx=0x7f8c100045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.278+0000 7f8c19ffb640 1 -- 172.21.15.66:0/3801212538 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10043020 con 0x7f8c1c1010a0 2024-04-21T09:15:49.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.278+0000 7f8c19ffb640 1 -- 172.21.15.66:0/3801212538 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c1002caf0 con 0x7f8c1c1010a0 2024-04-21T09:15:49.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.278+0000 7f8c19ffb640 1 -- 172.21.15.66:0/3801212538 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10003af0 con 0x7f8c1c1010a0 2024-04-21T09:15:49.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.278+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c1a72a0 con 0x7f8c1c1010a0 2024-04-21T09:15:49.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.278+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c1a7840 con 0x7f8c1c1010a0 2024-04-21T09:15:49.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.280+0000 7f8c19ffb640 1 -- 172.21.15.66:0/3801212538 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8c10046020 con 0x7f8c1c1010a0 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 Processor -- start 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 -- start start 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a0625451f0 con 0x55a0625e0e80 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a0624e24b0 con 0x55a0625e04d0 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c25dae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a0625411b0 con 0x55a0625e9a00 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c19ffb640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c0c0a3da0 0x7f8c0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60074/0 (socket says 172.21.15.66:60074) 2024-04-21T09:15:49.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 -- 172.21.15.66:0/4059458212 learned_addr learned my addr 172.21.15.66:0/4059458212 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c2167d640 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c2167d640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c0c0a3da0 0x7f8c0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c19ffb640 1 -- 172.21.15.66:0/3801212538 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f8c100f2c00 con 0x7f8c1c1010a0 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 -- 172.21.15.66:0/4059458212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 msgr2=0x55a0625ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625ebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 -- 172.21.15.66:0/4059458212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 msgr2=0x55a0625e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c20e7c640 1 -- 172.21.15.66:0/4059458212 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0624d41b0 con 0x55a0625e0e80 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.281+0000 7f8c2167d640 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.282+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e9430 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f8c100bef40 tx=0x7f8c10003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a379a40a001d3dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.282+0000 7f8c2167d640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c0c0a3da0 0x7f8c0c0a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400bfd0 tx=0x7f8c14004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.282+0000 7f8c06ffd640 1 -- 172.21.15.66:0/4059458212 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100be8e0 con 0x55a0625e0e80 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.282+0000 7f8c06ffd640 1 -- 172.21.15.66:0/4059458212 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c100befe0 con 0x55a0625e0e80 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.282+0000 7f8c06ffd640 1 -- 172.21.15.66:0/4059458212 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100053b0 con 0x55a0625e0e80 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- 172.21.15.66:0/4059458212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 msgr2=0x55a0625e9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e9430 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f8c100bef40 tx=0x7f8c10003f90 comp rx=0 tx=0).stop 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- 172.21.15.66:0/4059458212 shutdown_connections 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e9430 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 --2- 172.21.15.66:0/4059458212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- 172.21.15.66:0/4059458212 >> 172.21.15.66:0/4059458212 conn(0x55a0625daba0 msgr2=0x55a0625daf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- 172.21.15.66:0/4059458212 shutdown_connections 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- 172.21.15.66:0/4059458212 wait complete. 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- start start 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 Processor -- start 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e04d0 0x55a0625e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625e6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e9a00 0x55a0625e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a0625ec730 con 0x55a0625e9a00 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a0624fe500 con 0x55a0625e0e80 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a0625ec8b0 con 0x55a0625e04d0 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.283+0000 7f8c25dae740 10 client.0 _ll_register_callbacks cb 0x55a0624ccb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e9a00 0x55a0625e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e9a00 0x55a0625e6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60086/0 (socket says 172.21.15.66:60086) 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 -- 172.21.15.66:0/318406615 learned_addr learned my addr 172.21.15.66:0/318406615 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e04d0 0x55a0625e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 -- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e04d0 msgr2=0x55a0625e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e04d0 0x55a0625e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 -- 172.21.15.66:0/318406615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 msgr2=0x55a0625e6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625e6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 -- 172.21.15.66:0/318406615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0622573f0 con 0x55a0625e9a00 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e04d0 0x55a0625e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c2167d640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e9a00 0x55a0625e6bb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8c14013390 tx=0x7f8c14004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394637157c7caac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8bfbfff640 1 -- 172.21.15.66:0/318406615 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14023030 con 0x55a0625e9a00 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c100befe0 con 0x55a0625e9a00 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8bfbfff640 1 -- 172.21.15.66:0/318406615 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f8c14004b20 con 0x55a0625e9a00 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8bfbfff640 1 -- 172.21.15.66:0/318406615 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c1402f050 con 0x55a0625e9a00 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c25dae740 10 client.17589 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.284+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0623217d0 con 0x55a0625e9a00 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8bfbfff640 1 -- 172.21.15.66:0/318406615 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c100befe0 con 0x55a0625e9a00 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8bfbfff640 1 -- 172.21.15.66:0/318406615 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c140057a0 con 0x55a0625e9a00 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8c25dae740 10 client.17589 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8c25dae740 10 client.17589 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8c25dae740 20 client.17589 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:49.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8c25dae740 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a0625f9890 0x55a0625fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8c25dae740 4 client.17589 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0624e4930 con 0x55a0625f9890 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.285+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a0625f9890 0x55a0625fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a0625f9890 0x55a0625fbd50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ddd47903d96b617 server_cookie=357dffb97048924a in_seq=0 out_seq=0 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8bfbfff640 10 client.17589 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8bfbfff640 1 -- 172.21.15.66:0/318406615 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0624e4930 con 0x55a0625f9890 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8bfbfff640 10 client.17589 handle_command_reply: tid=0 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 client.17589 shutdown 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e9a00 msgr2=0x55a0625e6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e9a00 0x55a0625e6bb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8c14013390 tx=0x7f8c14004ab0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 shutdown_connections 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a0625f9890 0x55a0625fbd50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e9a00 0x55a0625e6bb0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8bfbfff640 0 client.17589 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 --2- 172.21.15.66:0/318406615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e04d0 0x55a0625e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.286+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 >> 172.21.15.66:0/318406615 conn(0x55a0625daba0 msgr2=0x55a0625dc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.287+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 shutdown_connections 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.287+0000 7f8c25dae740 1 -- 172.21.15.66:0/318406615 wait complete. 2024-04-21T09:15:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.287+0000 7f8c25dae740 20 client.17589 trim_cache size 0 max 16384 2024-04-21T09:15:49.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 Processor -- start 2024-04-21T09:15:49.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 -- start start 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625f5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b3e50 con 0x55a0625e0e80 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b1720 con 0x55a0625e04d0 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c25dae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1af590 con 0x55a0625e9a00 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60100/0 (socket says 172.21.15.66:60100) 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c1bfff640 1 -- 172.21.15.66:0/3016987453 learned_addr learned my addr 172.21.15.66:0/3016987453 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c2167d640 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c1bfff640 1 -- 172.21.15.66:0/3016987453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 msgr2=0x55a0625e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c1bfff640 1 -- 172.21.15.66:0/3016987453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 msgr2=0x55a0625f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.300+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c1bfff640 1 -- 172.21.15.66:0/3016987453 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0625e4ba0 con 0x55a0625e0e80 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e8d80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ba0 tx=0x7f8c08010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3154871283b888c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c067fc640 1 -- 172.21.15.66:0/3016987453 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08020070 con 0x55a0625e0e80 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c067fc640 1 -- 172.21.15.66:0/3016987453 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c080035f0 con 0x55a0625e0e80 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c067fc640 1 -- 172.21.15.66:0/3016987453 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0801b640 con 0x55a0625e0e80 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c25dae740 1 -- 172.21.15.66:0/3016987453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 msgr2=0x55a0625e8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c25dae740 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e8d80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ba0 tx=0x7f8c08010800 comp rx=0 tx=0).stop 2024-04-21T09:15:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c25dae740 1 -- 172.21.15.66:0/3016987453 shutdown_connections 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c25dae740 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c25dae740 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e0e80 0x55a0625e8d80 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.301+0000 7f8c25dae740 1 --2- 172.21.15.66:0/3016987453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e04d0 0x55a0625f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 -- 172.21.15.66:0/3016987453 >> 172.21.15.66:0/3016987453 conn(0x55a0625daba0 msgr2=0x55a0625e5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 -- 172.21.15.66:0/3016987453 shutdown_connections 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 -- 172.21.15.66:0/3016987453 wait complete. 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 -- start start 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 Processor -- start 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e04d0 0x55a0625f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1afbe0 con 0x55a0625e04d0 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b1050 con 0x55a0625e0e80 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b02b0 con 0x55a0625e9a00 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c25dae740 10 client.0 _ll_register_callbacks cb 0x7f8c14013040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c20e7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e04d0 0x55a0625f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c20e7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e04d0 0x55a0625f1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60112/0 (socket says 172.21.15.66:60112) 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.302+0000 7f8c20e7c640 1 -- 172.21.15.66:0/1941246102 learned_addr learned my addr 172.21.15.66:0/1941246102 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c2167d640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c20e7c640 1 -- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 msgr2=0x55a0625e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c20e7c640 1 -- 172.21.15.66:0/1941246102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 msgr2=0x55a0625f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c20e7c640 1 -- 172.21.15.66:0/1941246102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0625e36b0 con 0x55a0625e04d0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c2167d640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c20e7c640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e04d0 0x55a0625f1a30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8c10003fd0 tx=0x7f8c100035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7188241efbd80aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8bfbfff640 1 -- 172.21.15.66:0/1941246102 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100befe0 con 0x55a0625e04d0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8bfbfff640 1 -- 172.21.15.66:0/1941246102 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f8c100053b0 con 0x55a0625e04d0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c080035f0 con 0x55a0625e04d0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8bfbfff640 1 -- 172.21.15.66:0/1941246102 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c100be8e0 con 0x55a0625e04d0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c25dae740 10 client.17595 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a06265fa70 con 0x55a0625e04d0 2024-04-21T09:15:49.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8bfbfff640 1 -- 172.21.15.66:0/1941246102 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c080035f0 con 0x55a0625e04d0 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.303+0000 7f8bfbfff640 1 -- 172.21.15.66:0/1941246102 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c100a90d0 con 0x55a0625e04d0 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c25dae740 10 client.17595 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c25dae740 10 client.17595 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c25dae740 20 client.17595 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c25dae740 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a06265ffb0 0x55a0626623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c25dae740 4 client.17595 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a062552c10 con 0x55a06265ffb0 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a06265ffb0 0x55a0626623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8bfbfff640 10 client.17595 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8c1bfff640 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a06265ffb0 0x55a0626623a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cacf7c32ccac335 server_cookie=a65bd53d14ac9277 in_seq=0 out_seq=0 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8bfbfff640 1 -- 172.21.15.66:0/1941246102 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a062552c10 con 0x55a06265ffb0 2024-04-21T09:15:49.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.304+0000 7f8bfbfff640 10 client.17595 handle_command_reply: tid=0 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 client.17595 shutdown 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e04d0 msgr2=0x55a0625f1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e04d0 0x55a0625f1a30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8c10003fd0 tx=0x7f8c100035f0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 shutdown_connections 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a0625e9a00 0x55a0625e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0625e0e80 0x55a0625f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55a06265ffb0 0x55a0626623a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 --2- 172.21.15.66:0/1941246102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0625e04d0 0x55a0625f1a30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 >> 172.21.15.66:0/1941246102 conn(0x55a0625daba0 msgr2=0x55a0625d9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8bfbfff640 0 client.17595 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 shutdown_connections 2024-04-21T09:15:49.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 1 -- 172.21.15.66:0/1941246102 wait complete. 2024-04-21T09:15:49.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c25dae740 20 client.17595 trim_cache size 0 max 16384 2024-04-21T09:15:49.313 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c0c0a3da0 msgr2=0x7f8c0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c0c0a3da0 0x7f8c0c0a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400bfd0 tx=0x7f8c14004a40 comp rx=0 tx=0).stop 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 msgr2=0x7f8c1c102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c102b30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8c10005940 tx=0x7f8c100045e0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 shutdown_connections 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8c0c0a3da0 0x7f8c0c0a6260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c1035b0 0x7f8c1c1a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c1c106170 0x7f8c1c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 --2- 172.21.15.66:0/3801212538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c1c1010a0 0x7f8c1c102b30 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.305+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 >> 172.21.15.66:0/3801212538 conn(0x7f8c1c077eb0 msgr2=0x7f8c1c06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.306+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 shutdown_connections 2024-04-21T09:15:49.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.306+0000 7f8c230e0640 1 -- 172.21.15.66:0/3801212538 wait complete. 2024-04-21T09:15:49.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 -- 172.21.15.66:0/3498658925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64107260 msgr2=0x7fbc6410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 --2- 172.21.15.66:0/3498658925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64107260 0x7fbc6410ee30 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009ca0 tx=0x7fbc5c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 -- 172.21.15.66:0/3498658925 shutdown_connections 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 --2- 172.21.15.66:0/3498658925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64107260 0x7fbc6410ee30 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 --2- 172.21.15.66:0/3498658925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64102b40 0x7fbc64106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 --2- 172.21.15.66:0/3498658925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64102190 0x7fbc64102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 -- 172.21.15.66:0/3498658925 >> 172.21.15.66:0/3498658925 conn(0x7fbc640fd9f0 msgr2=0x7fbc640ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 -- 172.21.15.66:0/3498658925 shutdown_connections 2024-04-21T09:15:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 -- 172.21.15.66:0/3498658925 wait complete. 2024-04-21T09:15:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 Processor -- start 2024-04-21T09:15:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.420+0000 7fbc6b352640 1 -- start start 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64102190 0x7fbc64198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64107260 0x7fbc64198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc6419cfc0 0x7fbc6419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641121b0 con 0x7fbc64102190 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64112030 con 0x7fbc6419cfc0 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64112330 con 0x7fbc64107260 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64102190 0x7fbc64198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64102190 0x7fbc64198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60134/0 (socket says 172.21.15.66:60134) 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 -- 172.21.15.66:0/17904921 learned_addr learned my addr 172.21.15.66:0/17904921 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc688ed640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64107260 0x7fbc64198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc698ef640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc6419cfc0 0x7fbc6419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 -- 172.21.15.66:0/17904921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64107260 msgr2=0x7fbc64198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64107260 0x7fbc64198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 -- 172.21.15.66:0/17904921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc6419cfc0 msgr2=0x7fbc6419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc6419cfc0 0x7fbc6419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 -- 172.21.15.66:0/17904921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6419dab0 con 0x7fbc64102190 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc698ef640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc6419cfc0 0x7fbc6419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc688ed640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64107260 0x7fbc64198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc690ee640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64102190 0x7fbc64198750 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000d950 tx=0x7fbc6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc567fc640 1 -- 172.21.15.66:0/17904921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60017070 con 0x7fbc64102190 2024-04-21T09:15:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc567fc640 1 -- 172.21.15.66:0/17904921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc60009c30 con 0x7fbc64102190 2024-04-21T09:15:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6419dda0 con 0x7fbc64102190 2024-04-21T09:15:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc567fc640 1 -- 172.21.15.66:0/17904921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6000f620 con 0x7fbc64102190 2024-04-21T09:15:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.421+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc641a55e0 con 0x7fbc64102190 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.424+0000 7fbc567fc640 1 -- 172.21.15.66:0/17904921 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbc60010460 con 0x7fbc64102190 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.424+0000 7fbc6e029740 1 Processor -- start 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.424+0000 7fbc6e029740 1 -- start start 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.424+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b7987114d0 0x55b7987118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.424+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b79871a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.424+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b79871ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc6e029740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b7986761f0 con 0x55b798711e80 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc6e029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7986134b0 con 0x55b79871aa00 2024-04-21T09:15:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc6e029740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b7986721b0 con 0x55b7987114d0 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b7987114d0 0x55b7987118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc690ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b79871a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b7987114d0 0x55b7987118b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:38146/0 (socket says 172.21.15.66:38146) 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc688ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b79871ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 -- 172.21.15.66:0/2667350658 learned_addr learned my addr 172.21.15.66:0/2667350658 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc567fc640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc500a3da0 0x7fbc500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 -- 172.21.15.66:0/2667350658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 msgr2=0x55b79871ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 --2- 172.21.15.66:0/2667350658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b79871ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 -- 172.21.15.66:0/2667350658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 msgr2=0x55b79871a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 --2- 172.21.15.66:0/2667350658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b79871a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 -- 172.21.15.66:0/2667350658 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7986051b0 con 0x55b7987114d0 2024-04-21T09:15:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc567fc640 1 -- 172.21.15.66:0/17904921 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fbc600d0840 con 0x7fbc64102190 2024-04-21T09:15:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc500a3da0 0x7fbc500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.425+0000 7fbc698ef640 1 --2- 172.21.15.66:0/2667350658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b7987114d0 0x55b7987118b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009430 tx=0x7fbc5c0098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38824d96035eec9f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.426+0000 7fbc698ef640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc500a3da0 0x7fbc500a6260 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c04a000 tx=0x7fbc5c04b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc4f7fe640 1 -- 172.21.15.66:0/2667350658 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c039050 con 0x55b7987114d0 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc4f7fe640 1 -- 172.21.15.66:0/2667350658 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5c046030 con 0x55b7987114d0 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc4f7fe640 1 -- 172.21.15.66:0/2667350658 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0346a0 con 0x55b7987114d0 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 -- 172.21.15.66:0/2667350658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b7987114d0 msgr2=0x55b7987118b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2667350658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b7987114d0 0x55b7987118b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009430 tx=0x7fbc5c0098a0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 -- 172.21.15.66:0/2667350658 shutdown_connections 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2667350658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b79871ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2667350658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b79871a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2667350658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b7987114d0 0x55b7987118b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 -- 172.21.15.66:0/2667350658 >> 172.21.15.66:0/2667350658 conn(0x55b79870bba0 msgr2=0x55b79870bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 -- 172.21.15.66:0/2667350658 shutdown_connections 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.429+0000 7fbc6e029740 1 -- 172.21.15.66:0/2667350658 wait complete. 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 -- start start 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 Processor -- start 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b79871a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b798717740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b79871aa00 0x55b798717e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b79871d970 con 0x55b798711e80 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b79862f500 con 0x55b7987114d0 2024-04-21T09:15:49.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b79871daf0 con 0x55b79871aa00 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc6e029740 10 client.0 _ll_register_callbacks cb 0x55b7985fdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc688ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b798717740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc688ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b798717740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60160/0 (socket says 172.21.15.66:60160) 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc688ed640 1 -- 172.21.15.66:0/3620130710 learned_addr learned my addr 172.21.15.66:0/3620130710 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.430+0000 7fbc690ee640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b79871a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc698ef640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b79871aa00 0x55b798717e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc688ed640 1 -- 172.21.15.66:0/3620130710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b79871aa00 msgr2=0x55b798717e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc688ed640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b79871aa00 0x55b798717e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc688ed640 1 -- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 msgr2=0x55b79871a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc688ed640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b79871a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc688ed640 1 -- 172.21.15.66:0/3620130710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7983883f0 con 0x55b798711e80 2024-04-21T09:15:49.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc698ef640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b79871aa00 0x55b798717e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc688ed640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b798717740 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800bf70 tx=0x7fbc58012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a964c7eb0971c5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc4cff9640 1 -- 172.21.15.66:0/3620130710 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc580035f0 con 0x55b798711e80 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc4cff9640 1 -- 172.21.15.66:0/3620130710 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fbc58003790 con 0x55b798711e80 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5c046030 con 0x55b798711e80 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc4cff9640 1 -- 172.21.15.66:0/3620130710 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc58003990 con 0x55b798711e80 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc6e029740 10 client.17607 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7984527d0 con 0x55b798711e80 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.431+0000 7fbc4cff9640 1 -- 172.21.15.66:0/3620130710 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5c046030 con 0x55b798711e80 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc4cff9640 1 -- 172.21.15.66:0/3620130710 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc58019070 con 0x55b798711e80 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc6e029740 10 client.17607 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:49.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc6e029740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc6e029740 20 client.17607 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b79872ade0 0x55b79872d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc6e029740 4 client.17607 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b798615930 con 0x55b79872ade0 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc690ee640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b79872ade0 0x55b79872d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc4cff9640 10 client.17607 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.432+0000 7fbc690ee640 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b79872ade0 0x55b79872d220 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae35f509f826ad6 server_cookie=21ec0a46177d7054 in_seq=0 out_seq=0 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc4cff9640 1 -- 172.21.15.66:0/3620130710 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b798615930 con 0x55b79872ade0 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc4cff9640 10 client.17607 handle_command_reply: tid=0 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 client.17607 shutdown 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 msgr2=0x55b798717740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b798717740 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800bf70 tx=0x7fbc58012820 comp rx=0 tx=0).stop 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 shutdown_connections 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b79872ade0 0x55b79872d220 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b79871aa00 0x55b798717e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b798711e80 0x55b798717740 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc4cff9640 0 client.17607 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3620130710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b79871a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 >> 172.21.15.66:0/3620130710 conn(0x55b79870bba0 msgr2=0x55b79870d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 shutdown_connections 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.433+0000 7fbc6e029740 1 -- 172.21.15.66:0/3620130710 wait complete. 2024-04-21T09:15:49.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.434+0000 7fbc6e029740 20 client.17607 trim_cache size 0 max 16384 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 Processor -- start 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 -- start start 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b798726830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79870d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b79871aa00 0x55b798732af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641b1c40 con 0x55b79871aa00 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641af510 con 0x55b7987114d0 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc6e029740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ad380 con 0x55b798711e80 2024-04-21T09:15:49.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b79871aa00 0x55b798732af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b79871aa00 0x55b798732af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60164/0 (socket says 172.21.15.66:60164) 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 -- 172.21.15.66:0/3162903721 learned_addr learned my addr 172.21.15.66:0/3162903721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc690ee640 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b798726830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc688ed640 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79870d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 -- 172.21.15.66:0/3162903721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 msgr2=0x55b79870d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79870d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 -- 172.21.15.66:0/3162903721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 msgr2=0x55b798726830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b798726830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc698ef640 1 -- 172.21.15.66:0/3162903721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b798715c90 con 0x55b79871aa00 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc688ed640 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79870d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.447+0000 7fbc690ee640 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b798726830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc698ef640 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b79871aa00 0x55b798732af0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c033000 tx=0x7fbc5c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc8bb9b140aa3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc4effd640 1 -- 172.21.15.66:0/3162903721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c004040 con 0x55b79871aa00 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc4effd640 1 -- 172.21.15.66:0/3162903721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5c046040 con 0x55b79871aa00 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc4effd640 1 -- 172.21.15.66:0/3162903721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0349a0 con 0x55b79871aa00 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 -- 172.21.15.66:0/3162903721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b79871aa00 msgr2=0x55b798732af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b79871aa00 0x55b798732af0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c033000 tx=0x7fbc5c0095c0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 -- 172.21.15.66:0/3162903721 shutdown_connections 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b79871aa00 0x55b798732af0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79870d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 --2- 172.21.15.66:0/3162903721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7987114d0 0x55b798726830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 -- 172.21.15.66:0/3162903721 >> 172.21.15.66:0/3162903721 conn(0x55b79870bba0 msgr2=0x55b79870cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 -- 172.21.15.66:0/3162903721 shutdown_connections 2024-04-21T09:15:49.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.448+0000 7fbc6e029740 1 -- 172.21.15.66:0/3162903721 wait complete. 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 -- start start 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 Processor -- start 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7987114d0 0x55b79871eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79871f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b798794770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ad9d0 con 0x55b7987114d0 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641aee40 con 0x55b79871aa00 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ae0a0 con 0x55b798711e80 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc6e029740 10 client.0 _ll_register_callbacks cb 0x7fbc5800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7987114d0 0x55b79871eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7987114d0 0x55b79871eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60168/0 (socket says 172.21.15.66:60168) 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 -- 172.21.15.66:0/2232392421 learned_addr learned my addr 172.21.15.66:0/2232392421 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc698ef640 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b798794770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 -- 172.21.15.66:0/2232392421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 msgr2=0x55b79871f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79871f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 -- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 msgr2=0x55b798794770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b798794770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc690ee640 1 -- 172.21.15.66:0/2232392421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b798794cb0 con 0x55b7987114d0 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.449+0000 7fbc698ef640 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b798794770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc690ee640 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7987114d0 0x55b79871eb10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cdc20 tx=0x7fbc60010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaef93ca8221a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc4cff9640 1 -- 172.21.15.66:0/2232392421 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60020040 con 0x55b7987114d0 2024-04-21T09:15:49.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc4cff9640 1 -- 172.21.15.66:0/2232392421 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fbc60098a80 con 0x55b7987114d0 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5c046040 con 0x55b7987114d0 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc4cff9640 1 -- 172.21.15.66:0/2232392421 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc6009cb10 con 0x55b7987114d0 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 10 client.17613 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b798795d80 con 0x55b7987114d0 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc4cff9640 1 -- 172.21.15.66:0/2232392421 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5c046040 con 0x55b7987114d0 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc4cff9640 1 -- 172.21.15.66:0/2232392421 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc600d0430 con 0x55b7987114d0 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 10 client.17613 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 20 client.17613 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b798796480 0x55b7987988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 4 client.17613 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:15:49.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.450+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b798683c10 con 0x55b798796480 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc688ed640 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b798796480 0x55b7987988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc4cff9640 10 client.17613 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc688ed640 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b798796480 0x55b7987988c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b17b60116fed562 server_cookie=1adf8b80a901b382 in_seq=0 out_seq=0 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc4cff9640 1 -- 172.21.15.66:0/2232392421 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b798683c10 con 0x55b798796480 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc4cff9640 10 client.17613 handle_command_reply: tid=0 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 client.17613 shutdown 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7987114d0 msgr2=0x55b79871eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7987114d0 0x55b79871eb10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cdc20 tx=0x7fbc60010f80 comp rx=0 tx=0).stop 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 shutdown_connections 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55b798796480 0x55b7987988c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b79871aa00 0x55b798794770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b798711e80 0x55b79871f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc4cff9640 0 client.17613 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:49.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 --2- 172.21.15.66:0/2232392421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7987114d0 0x55b79871eb10 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 >> 172.21.15.66:0/2232392421 conn(0x55b79870bba0 msgr2=0x55b79870d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 shutdown_connections 2024-04-21T09:15:49.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 1 -- 172.21.15.66:0/2232392421 wait complete. 2024-04-21T09:15:49.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.451+0000 7fbc6e029740 20 client.17613 trim_cache size 0 max 16384 2024-04-21T09:15:49.460 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:15:49.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc500a3da0 msgr2=0x7fbc500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc500a3da0 0x7fbc500a6260 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c04a000 tx=0x7fbc5c04b000 comp rx=0 tx=0).stop 2024-04-21T09:15:49.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64102190 msgr2=0x7fbc64198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:49.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64102190 0x7fbc64198750 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000d950 tx=0x7fbc6000de10 comp rx=0 tx=0).stop 2024-04-21T09:15:49.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 shutdown_connections 2024-04-21T09:15:49.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc500a3da0 0x7fbc500a6260 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc6419cfc0 0x7fbc6419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64107260 0x7fbc64198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 --2- 172.21.15.66:0/17904921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc64102190 0x7fbc64198750 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:49.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 >> 172.21.15.66:0/17904921 conn(0x7fbc640fd9f0 msgr2=0x7fbc640fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:49.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 shutdown_connections 2024-04-21T09:15:49.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:49.452+0000 7fbc6b352640 1 -- 172.21.15.66:0/17904921 wait complete. 2024-04-21T09:15:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:49 smithi066 ceph-mon[27953]: pgmap v233: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:49.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:49 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1498866005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:15:49.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:49 smithi186 ceph-mon[31505]: pgmap v233: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:49.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:49 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1498866005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:15:49.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:49 smithi083 ceph-mon[31413]: pgmap v233: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:15:49.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:49 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1498866005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:15:50.464 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:15:50.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:15:50.465 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 -- 172.21.15.66:0/3428993271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 msgr2=0x7f120810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 --2- 172.21.15.66:0/3428993271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f120810ee30 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1200009ca0 tx=0x7f120002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 -- 172.21.15.66:0/3428993271 shutdown_connections 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 --2- 172.21.15.66:0/3428993271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f120810ee30 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 --2- 172.21.15.66:0/3428993271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1208102b40 0x7f1208106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 --2- 172.21.15.66:0/3428993271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1208102190 0x7f1208102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 -- 172.21.15.66:0/3428993271 >> 172.21.15.66:0/3428993271 conn(0x7f12080fd990 msgr2=0x7f12080ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 -- 172.21.15.66:0/3428993271 shutdown_connections 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.585+0000 7f120fca5640 1 -- 172.21.15.66:0/3428993271 wait complete. 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 Processor -- start 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 -- start start 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1208102190 0x7f1208198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f1208198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120819d020 0x7f120819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1208112220 con 0x7f1208107260 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12081120a0 con 0x7f120819d020 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120fca5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f12081123a0 con 0x7f1208102190 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120d240640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f1208198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120d240640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f1208198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60178/0 (socket says 172.21.15.66:60178) 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.586+0000 7f120d240640 1 -- 172.21.15.66:0/2826377534 learned_addr learned my addr 172.21.15.66:0/2826377534 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120e242640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120819d020 0x7f120819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120da41640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1208102190 0x7f1208198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120d240640 1 -- 172.21.15.66:0/2826377534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1208102190 msgr2=0x7f1208198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120d240640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1208102190 0x7f1208198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120d240640 1 -- 172.21.15.66:0/2826377534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120819d020 msgr2=0x7f120819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120d240640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120819d020 0x7f120819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120d240640 1 -- 172.21.15.66:0/2826377534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120819db10 con 0x7f1208107260 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120d240640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f1208198ca0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f11f800d770 tx=0x7f11f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f11feffd640 1 -- 172.21.15.66:0/2826377534 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8017070 con 0x7f1208107260 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120819de00 con 0x7f1208107260 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f11feffd640 1 -- 172.21.15.66:0/2826377534 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f8009c30 con 0x7f1208107260 2024-04-21T09:15:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f11feffd640 1 -- 172.21.15.66:0/2826377534 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f800f620 con 0x7f1208107260 2024-04-21T09:15:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.587+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a55f0 con 0x7f1208107260 2024-04-21T09:15:50.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.590+0000 7f11feffd640 1 -- 172.21.15.66:0/2826377534 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f11f801a070 con 0x7f1208107260 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.590+0000 7f1212971740 1 Processor -- start 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.590+0000 7f1212971740 1 -- start start 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f1212971740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f1212971740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f1212971740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b56260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f1212971740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559fe7aaef00 con 0x559fe7b53e10 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f1212971740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559fe7a4c590 con 0x559fe7b4b290 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f1212971740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559fe7aaaec0 con 0x559fe7b4a8e0 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f11feffd640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f11f40a3d20 0x7f11f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120d240640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b56260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120d240640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b56260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60182/0 (socket says 172.21.15.66:60182) 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120d240640 1 -- 172.21.15.66:0/2641433819 learned_addr learned my addr 172.21.15.66:0/2641433819 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120da41640 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120e242640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f11f40a3d20 0x7f11f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f11feffd640 1 -- 172.21.15.66:0/2826377534 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f11f80d07a0 con 0x7f1208107260 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120da41640 1 -- 172.21.15.66:0/2641433819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 msgr2=0x559fe7b4acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120da41640 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120da41640 1 -- 172.21.15.66:0/2641433819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 msgr2=0x559fe7b56260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120da41640 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b56260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120da41640 1 -- 172.21.15.66:0/2641433819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe78897e0 con 0x559fe7b4b290 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.591+0000 7f120d240640 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b56260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.592+0000 7f120da41640 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b53840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f120400b760 tx=0x7f120400bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc0da057d508a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.592+0000 7f120e242640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f11f40a3d20 0x7f11f40a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1200004af0 tx=0x7f1200032380 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.592+0000 7f11f3fff640 1 -- 172.21.15.66:0/2641433819 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400d950 con 0x559fe7b4b290 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.592+0000 7f11f3fff640 1 -- 172.21.15.66:0/2641433819 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204019070 con 0x559fe7b4b290 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.592+0000 7f11f3fff640 1 -- 172.21.15.66:0/2641433819 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204014d30 con 0x559fe7b4b290 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 -- 172.21.15.66:0/2641433819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 msgr2=0x559fe7b53840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b53840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f120400b760 tx=0x7f120400bc20 comp rx=0 tx=0).stop 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 -- 172.21.15.66:0/2641433819 shutdown_connections 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b56260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b53840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 --2- 172.21.15.66:0/2641433819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 -- 172.21.15.66:0/2641433819 >> 172.21.15.66:0/2641433819 conn(0x559fe7b44fb0 msgr2=0x559fe7b45390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 -- 172.21.15.66:0/2641433819 shutdown_connections 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 -- 172.21.15.66:0/2641433819 wait complete. 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 -- start start 2024-04-21T09:15:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.593+0000 7f1212971740 1 Processor -- start 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f1212971740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b50df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f1212971740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4b290 0x559fe7b51330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f1212971740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b53e10 0x559fe7b51870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f1212971740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559fe7b56d80 con 0x559fe7b4b290 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f1212971740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559fe7a68580 con 0x559fe7b53e10 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f1212971740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559fe7b56f00 con 0x559fe7b4a8e0 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f1212971740 10 client.0 _ll_register_callbacks cb 0x559fe7a6e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4b290 0x559fe7b51330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4b290 0x559fe7b51330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60192/0 (socket says 172.21.15.66:60192) 2024-04-21T09:15:50.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 -- 172.21.15.66:0/3330089021 learned_addr learned my addr 172.21.15.66:0/3330089021 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120e242640 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b53e10 0x559fe7b51870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120da41640 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b50df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 -- 172.21.15.66:0/3330089021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 msgr2=0x559fe7b50df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b50df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 -- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b53e10 msgr2=0x559fe7b51870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b53e10 0x559fe7b51870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 -- 172.21.15.66:0/3330089021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe78e9f70 con 0x559fe7b4b290 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f120d240640 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4b290 0x559fe7b51330 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f11f80a4a50 tx=0x7f11f8010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd35de4f4271a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f11f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.594+0000 7f11f17fa640 1 -- 172.21.15.66:0/3330089021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8021040 con 0x559fe7b4b290 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f11f17fa640 1 -- 172.21.15.66:0/3330089021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f11f80d0350 con 0x559fe7b4b290 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1204019070 con 0x559fe7b4b290 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f11f17fa640 1 -- 172.21.15.66:0/3330089021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11f80cdf40 con 0x559fe7b4b290 2024-04-21T09:15:50.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 10 client.17625 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fe7781170 con 0x559fe7b4b290 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f11f17fa640 1 -- 172.21.15.66:0/3330089021 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f807d080 con 0x559fe7b4b290 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f11f17fa640 1 -- 172.21.15.66:0/3330089021 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f11f80d0b90 con 0x559fe7b4b290 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 10 client.17625 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 20 client.17625 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7b64300 0x559fe7b66720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 4 client.17625 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fe7abc920 con 0x559fe7b64300 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.595+0000 7f120da41640 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7b64300 0x559fe7b66720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.596+0000 7f120da41640 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7b64300 0x559fe7b66720 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ef9c67ca4fe962 server_cookie=d718fb4375cf9923 in_seq=0 out_seq=0 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.596+0000 7f11f17fa640 10 client.17625 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.596+0000 7f11f17fa640 1 -- 172.21.15.66:0/3330089021 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fe7abc920 con 0x559fe7b64300 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f11f17fa640 10 client.17625 handle_command_reply: tid=0 2024-04-21T09:15:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 client.17625 shutdown 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4b290 msgr2=0x559fe7b51330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4b290 0x559fe7b51330 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f11f80a4a50 tx=0x7f11f8010c00 comp rx=0 tx=0).stop 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 shutdown_connections 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7b64300 0x559fe7b66720 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b53e10 0x559fe7b51870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4b290 0x559fe7b51330 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 --2- 172.21.15.66:0/3330089021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b50df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f11f17fa640 0 client.17625 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 >> 172.21.15.66:0/3330089021 conn(0x559fe7b44fb0 msgr2=0x559fe7b46d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 shutdown_connections 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 1 -- 172.21.15.66:0/3330089021 wait complete. 2024-04-21T09:15:50.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.597+0000 7f1212971740 20 client.17625 trim_cache size 0 max 16384 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 Processor -- start 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 -- start start 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b53520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b1c60 con 0x559fe7b53e10 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12081af530 con 0x559fe7b4b290 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f1212971740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad3a0 con 0x559fe7b4a8e0 2024-04-21T09:15:50.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b6bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60202/0 (socket says 172.21.15.66:60202) 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 -- 172.21.15.66:0/2875886971 learned_addr learned my addr 172.21.15.66:0/2875886971 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120d240640 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120da41640 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b53520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 -- 172.21.15.66:0/2875886971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 msgr2=0x559fe7b53520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b53520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 -- 172.21.15.66:0/2875886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 msgr2=0x559fe7b5fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b5fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.615+0000 7f120e242640 1 -- 172.21.15.66:0/2875886971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe7b49b20 con 0x559fe7b53e10 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f120e242640 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b6bff0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1200037040 tx=0x7f1200040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac266ad9ebcb2eca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f11f37fe640 1 -- 172.21.15.66:0/2875886971 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200039040 con 0x559fe7b53e10 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f11f37fe640 1 -- 172.21.15.66:0/2875886971 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1200053070 con 0x559fe7b53e10 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f11f37fe640 1 -- 172.21.15.66:0/2875886971 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200034440 con 0x559fe7b53e10 2024-04-21T09:15:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 -- 172.21.15.66:0/2875886971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 msgr2=0x559fe7b6bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b6bff0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1200037040 tx=0x7f1200040000 comp rx=0 tx=0).stop 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 -- 172.21.15.66:0/2875886971 shutdown_connections 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b53e10 0x559fe7b6bff0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 --2- 172.21.15.66:0/2875886971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b4a8e0 0x559fe7b53520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 -- 172.21.15.66:0/2875886971 >> 172.21.15.66:0/2875886971 conn(0x559fe7b44fb0 msgr2=0x559fe7b51640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 -- 172.21.15.66:0/2875886971 shutdown_connections 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.616+0000 7f1212971740 1 -- 172.21.15.66:0/2875886971 wait complete. 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 -- start start 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 Processor -- start 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4a8e0 0x559fe7b57e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b53e10 0x559fe7b58890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad9f0 con 0x559fe7b4a8e0 2024-04-21T09:15:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12081aee60 con 0x559fe7b4b290 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ae0c0 con 0x559fe7b53e10 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f1212971740 10 client.0 _ll_register_callbacks cb 0x7f11f809c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f120da41640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4a8e0 0x559fe7b57e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f120da41640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4a8e0 0x559fe7b57e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60210/0 (socket says 172.21.15.66:60210) 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f120da41640 1 -- 172.21.15.66:0/63814466 learned_addr learned my addr 172.21.15.66:0/63814466 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f120e242640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b53e10 0x559fe7b58890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.617+0000 7f120d240640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f120da41640 1 -- 172.21.15.66:0/63814466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b53e10 msgr2=0x559fe7b58890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f120da41640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b53e10 0x559fe7b58890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f120da41640 1 -- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 msgr2=0x559fe7b58350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f120da41640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b58350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f120da41640 1 -- 172.21.15.66:0/63814466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe7bcdf10 con 0x559fe7b4a8e0 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f120d240640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f120da41640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4a8e0 0x559fe7b57e10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f120400b700 tx=0x7f1204014990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef15b3e237566cb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f11f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f11f17fa640 1 -- 172.21.15.66:0/63814466 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120402b070 con 0x559fe7b4a8e0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f11f17fa640 1 -- 172.21.15.66:0/63814466 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f1204027dd0 con 0x559fe7b4a8e0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1200053070 con 0x559fe7b4a8e0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f11f17fa640 1 -- 172.21.15.66:0/63814466 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1204036050 con 0x559fe7b4a8e0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f1212971740 10 client.17631 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fe7bcf250 con 0x559fe7b4a8e0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f11f17fa640 1 -- 172.21.15.66:0/63814466 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1200053070 con 0x559fe7b4a8e0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f11f17fa640 1 -- 172.21.15.66:0/63814466 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1204013070 con 0x559fe7b4a8e0 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.618+0000 7f1212971740 10 client.17631 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f1212971740 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f1212971740 20 client.17631 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f1212971740 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7bcf5e0 0x559fe7bd19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f1212971740 4 client.17631 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:15:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559fe7ab6790 con 0x559fe7bcf5e0 2024-04-21T09:15:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f120d240640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7bcf5e0 0x559fe7bd19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f11f17fa640 10 client.17631 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.619+0000 7f120d240640 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7bcf5e0 0x559fe7bd19d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32be0e60ec4c8263 server_cookie=905c9b151e22f2b1 in_seq=0 out_seq=0 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.620+0000 7f11f17fa640 1 -- 172.21.15.66:0/63814466 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559fe7ab6790 con 0x559fe7bcf5e0 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.620+0000 7f11f17fa640 10 client.17631 handle_command_reply: tid=0 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.620+0000 7f1212971740 1 client.17631 shutdown 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.620+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4a8e0 msgr2=0x559fe7b57e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.620+0000 7f1212971740 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4a8e0 0x559fe7b57e10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f120400b700 tx=0x7f1204014990 comp rx=0 tx=0).stop 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 shutdown_connections 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x559fe7bcf5e0 0x559fe7bd19d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559fe7b53e10 0x559fe7b58890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fe7b4b290 0x559fe7b58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f11f17fa640 0 client.17631 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 --2- 172.21.15.66:0/63814466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559fe7b4a8e0 0x559fe7b57e10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 >> 172.21.15.66:0/63814466 conn(0x559fe7b44fb0 msgr2=0x559fe7b43780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 shutdown_connections 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 1 -- 172.21.15.66:0/63814466 wait complete. 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f1212971740 20 client.17631 trim_cache size 0 max 16384 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "782a13fd-cd97-41a8-8c02-06b25d753b39", 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:15:50.626 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:15:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f11f40a3d20 msgr2=0x7f11f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f11f40a3d20 0x7f11f40a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1200004af0 tx=0x7f1200032380 comp rx=0 tx=0).stop 2024-04-21T09:15:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 msgr2=0x7f1208198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f1208198ca0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f11f800d770 tx=0x7f11f800dc30 comp rx=0 tx=0).stop 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 shutdown_connections 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f11f40a3d20 0x7f11f40a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120819d020 0x7f120819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1208107260 0x7f1208198ca0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 --2- 172.21.15.66:0/2826377534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1208102190 0x7f1208198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 >> 172.21.15.66:0/2826377534 conn(0x7f12080fd990 msgr2=0x7f12080fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.621+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 shutdown_connections 2024-04-21T09:15:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.622+0000 7f120fca5640 1 -- 172.21.15.66:0/2826377534 wait complete. 2024-04-21T09:15:50.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:15:50.632 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:15:50.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 -- 172.21.15.66:0/462549791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 msgr2=0x7f610010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 --2- 172.21.15.66:0/462549791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f610010dca0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009900 tx=0x7f60f802bc30 comp rx=0 tx=0).stop 2024-04-21T09:15:50.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 -- 172.21.15.66:0/462549791 shutdown_connections 2024-04-21T09:15:50.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 --2- 172.21.15.66:0/462549791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f610010dca0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 --2- 172.21.15.66:0/462549791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61001010a0 0x7f6100101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 --2- 172.21.15.66:0/462549791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6100105dd0 0x7f61001061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 -- 172.21.15.66:0/462549791 >> 172.21.15.66:0/462549791 conn(0x7f6100077c40 msgr2=0x7f6100078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 -- 172.21.15.66:0/462549791 shutdown_connections 2024-04-21T09:15:50.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.737+0000 7f6104893640 1 -- 172.21.15.66:0/462549791 wait complete. 2024-04-21T09:15:50.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 Processor -- start 2024-04-21T09:15:50.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 -- start start 2024-04-21T09:15:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61001010a0 0x7f6100198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f6100198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6100105dd0 0x7f610019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6100112290 con 0x7f6100101a40 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6100112110 con 0x7f61001010a0 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f6104893640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6100112410 con 0x7f6100105dd0 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f6100198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f6100198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60236/0 (socket says 172.21.15.66:60236) 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3276459722 learned_addr learned my addr 172.21.15.66:0/3276459722 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f60fe59c640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61001010a0 0x7f6100198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3276459722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6100105dd0 msgr2=0x7f610019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6100105dd0 0x7f610019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.738+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3276459722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61001010a0 msgr2=0x7f6100198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61001010a0 0x7f6100198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3276459722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610019d630 con 0x7f6100101a40 2024-04-21T09:15:50.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f60fe59c640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61001010a0 0x7f6100198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:15:50.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f6100198bd0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f60f000d770 tx=0x7f60f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f60ef7fe640 1 -- 172.21.15.66:0/3276459722 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0017070 con 0x7f6100101a40 2024-04-21T09:15:50.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610019d920 con 0x7f6100101a40 2024-04-21T09:15:50.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f60ef7fe640 1 -- 172.21.15.66:0/3276459722 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f00054d0 con 0x7f6100101a40 2024-04-21T09:15:50.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f60ef7fe640 1 -- 172.21.15.66:0/3276459722 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0004d00 con 0x7f6100101a40 2024-04-21T09:15:50.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.739+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61001a51b0 con 0x7f6100101a40 2024-04-21T09:15:50.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f60ef7fe640 1 -- 172.21.15.66:0/3276459722 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f60f0004ea0 con 0x7f6100101a40 2024-04-21T09:15:50.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 Processor -- start 2024-04-21T09:15:50.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 -- start start 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 0x563d7844f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d78458540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7845ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563d783b41e0 con 0x563d78458a80 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d78351390 con 0x563d7844fea0 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f6107567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563d783b01a0 con 0x563d7844f580 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f60ef7fe640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f60e80a3d20 0x7f60e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7845ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.742+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7845ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60238/0 (socket says 172.21.15.66:60238) 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3488729233 learned_addr learned my addr 172.21.15.66:0/3488729233 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fed9d640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f60e80a3d20 0x7f60e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fe59c640 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d78458540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fed9d640 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 0x563d7844f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60ef7fe640 1 -- 172.21.15.66:0/3276459722 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f60f00d0680 con 0x7f6100101a40 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3488729233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 msgr2=0x563d7844f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 0x563d7844f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3488729233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 msgr2=0x563d78458540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d78458540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fdd9b640 1 -- 172.21.15.66:0/3488729233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d781907d0 con 0x563d78458a80 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fe59c640 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d78458540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fed9d640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f60e80a3d20 0x7f60e80a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005940 tx=0x7f60f80058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7845ae70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f60f00986b0 tx=0x7f60f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433e2d74413ccb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60ecff9640 1 -- 172.21.15.66:0/3488729233 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0020070 con 0x563d78458a80 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60ecff9640 1 -- 172.21.15.66:0/3488729233 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f009c250 con 0x563d78458a80 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.743+0000 7f60ecff9640 1 -- 172.21.15.66:0/3488729233 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00988c0 con 0x563d78458a80 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 -- 172.21.15.66:0/3488729233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 msgr2=0x563d7845ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7845ae70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f60f00986b0 tx=0x7f60f00046c0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 -- 172.21.15.66:0/3488729233 shutdown_connections 2024-04-21T09:15:50.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7845ae70 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d78458540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 --2- 172.21.15.66:0/3488729233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 0x563d7844f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 -- 172.21.15.66:0/3488729233 >> 172.21.15.66:0/3488729233 conn(0x563d78449cb0 msgr2=0x563d7844a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 -- 172.21.15.66:0/3488729233 shutdown_connections 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 -- 172.21.15.66:0/3488729233 wait complete. 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 -- start start 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.744+0000 7f6107567740 1 Processor -- start 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 0x563d78455a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d78455fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 0x563d78456500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563d7845b7b0 con 0x563d7844fea0 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d7836d2f0 con 0x563d7844f580 2024-04-21T09:15:50.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563d7845b930 con 0x563d78458a80 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 10 client.0 _ll_register_callbacks cb 0x563d7837a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d78455fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d78455fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60250/0 (socket says 172.21.15.66:60250) 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 -- 172.21.15.66:0/1506695364 learned_addr learned my addr 172.21.15.66:0/1506695364 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fed9d640 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 0x563d78456500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fe59c640 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 0x563d78455a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 -- 172.21.15.66:0/1506695364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 msgr2=0x563d78456500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 0x563d78456500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 -- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 msgr2=0x563d78455a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 0x563d78455a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 -- 172.21.15.66:0/1506695364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d780c63f0 con 0x563d7844fea0 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d78455fc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ce000 tx=0x7f60f00cdf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c733fabd2e7a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:50.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60e5ffb640 1 -- 172.21.15.66:0/1506695364 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0020040 con 0x563d7844fea0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f60e5ffb640 1 -- 172.21.15.66:0/1506695364 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f60f00988c0 con 0x563d7844fea0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.745+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f009c250 con 0x563d7844fea0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f60e5ffb640 1 -- 172.21.15.66:0/1506695364 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f60f00a0600 con 0x563d7844fea0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 10 client.17643 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d780860f0 con 0x563d7844fea0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f60e5ffb640 1 -- 172.21.15.66:0/1506695364 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f009c250 con 0x563d7844fea0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f60e5ffb640 1 -- 172.21.15.66:0/1506695364 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f60f00d0240 con 0x563d7844fea0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 10 client.17643 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 20 client.17643 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d78468800 0x563d7846abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 4 client.17643 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d783c1c00 con 0x563d78468800 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.746+0000 7f60fe59c640 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d78468800 0x563d7846abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f60e5ffb640 10 client.17643 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f60fe59c640 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d78468800 0x563d7846abf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb77ba97551b39f6 server_cookie=924c096a32ce583f in_seq=0 out_seq=0 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f60e5ffb640 1 -- 172.21.15.66:0/1506695364 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d783c1c00 con 0x563d78468800 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f60e5ffb640 10 client.17643 handle_command_reply: tid=0 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 client.17643 shutdown 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 msgr2=0x563d78455fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d78455fc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ce000 tx=0x7f60f00cdf30 comp rx=0 tx=0).stop 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 shutdown_connections 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d78468800 0x563d7846abf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 0x563d78456500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d78455fc0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.748+0000 7f60e5ffb640 0 client.17643 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.747+0000 7f6107567740 1 --2- 172.21.15.66:0/1506695364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 0x563d78455a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.748+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 >> 172.21.15.66:0/1506695364 conn(0x563d78449cb0 msgr2=0x563d7844ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.748+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 shutdown_connections 2024-04-21T09:15:50.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.748+0000 7f6107567740 1 -- 172.21.15.66:0/1506695364 wait complete. 2024-04-21T09:15:50.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.748+0000 7f6107567740 20 client.17643 trim_cache size 0 max 16384 2024-04-21T09:15:50.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 Processor -- start 2024-04-21T09:15:50.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 -- start start 2024-04-21T09:15:50.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 0x563d784581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d7846dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 0x563d7846e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b1b50 con 0x563d7844fea0 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61001af420 con 0x563d7844f580 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f6107567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61001ad290 con 0x563d78458a80 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d7846dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d7846dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60266/0 (socket says 172.21.15.66:60266) 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.765+0000 7f60fdd9b640 1 -- 172.21.15.66:0/2452670462 learned_addr learned my addr 172.21.15.66:0/2452670462 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60fdd9b640 1 -- 172.21.15.66:0/2452670462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 msgr2=0x563d7846e150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/2452670462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 0x563d7846e150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60fdd9b640 1 -- 172.21.15.66:0/2452670462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 msgr2=0x563d784581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/2452670462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 0x563d784581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60fdd9b640 1 -- 172.21.15.66:0/2452670462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7845ae70 con 0x563d7844fea0 2024-04-21T09:15:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/2452670462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d7846dc10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f60f00986b0 tx=0x7f60f0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa621385905ac80e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60e7fff640 1 -- 172.21.15.66:0/2452670462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0020070 con 0x563d7844fea0 2024-04-21T09:15:50.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60e7fff640 1 -- 172.21.15.66:0/2452670462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f009c950 con 0x563d7844fea0 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f60e7fff640 1 -- 172.21.15.66:0/2452670462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0009ca0 con 0x563d7844fea0 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 -- 172.21.15.66:0/2452670462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 msgr2=0x563d7846dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 --2- 172.21.15.66:0/2452670462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d7846dc10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f60f00986b0 tx=0x7f60f0009f00 comp rx=0 tx=0).stop 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 -- 172.21.15.66:0/2452670462 shutdown_connections 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 --2- 172.21.15.66:0/2452670462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d78458a80 0x563d7846e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 --2- 172.21.15.66:0/2452670462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d7844fea0 0x563d7846dc10 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 --2- 172.21.15.66:0/2452670462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844f580 0x563d784581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 -- 172.21.15.66:0/2452670462 >> 172.21.15.66:0/2452670462 conn(0x563d78449cb0 msgr2=0x563d78448d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.766+0000 7f6107567740 1 -- 172.21.15.66:0/2452670462 shutdown_connections 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 -- 172.21.15.66:0/2452670462 wait complete. 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 -- start start 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 Processor -- start 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 0x563d7846b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d7846bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7846c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61001ad8e0 con 0x563d78458a80 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61001aed50 con 0x563d7844fea0 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61001adfb0 con 0x563d7844f580 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f6107567740 10 client.0 _ll_register_callbacks cb 0x7f60f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7846c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7846c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60278/0 (socket says 172.21.15.66:60278) 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f60fed9d640 1 -- 172.21.15.66:0/4001679965 learned_addr learned my addr 172.21.15.66:0/4001679965 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:15:50.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.767+0000 7f60fdd9b640 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d7846bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60fed9d640 1 -- 172.21.15.66:0/4001679965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 msgr2=0x563d7846b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60fed9d640 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 0x563d7846b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60fed9d640 1 -- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 msgr2=0x563d7846bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60fed9d640 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d7846bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60fed9d640 1 -- 172.21.15.66:0/4001679965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d784cd9e0 con 0x563d78458a80 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60fed9d640 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7846c130 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009900 tx=0x7f60f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35f0f4bde51c9af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60e5ffb640 1 -- 172.21.15.66:0/4001679965 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8052020 con 0x563d78458a80 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60e5ffb640 1 -- 172.21.15.66:0/4001679965 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f60f804e470 con 0x563d78458a80 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f009c950 con 0x563d78458a80 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60e5ffb640 1 -- 172.21.15.66:0/4001679965 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f60f805d050 con 0x563d78458a80 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 10 client.17649 fetch_fsmap learned FSMap version 25 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d784cecd0 con 0x563d78458a80 2024-04-21T09:15:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60e5ffb640 1 -- 172.21.15.66:0/4001679965 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f009c950 con 0x563d78458a80 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f60e5ffb640 1 -- 172.21.15.66:0/4001679965 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f60f80426b0 con 0x563d78458a80 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 10 client.17649 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 20 client.17649 populate_metadata read hostname 'smithi066' 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d784cf4d0 0x563d784d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 4 client.17649 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.768+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d783bba70 con 0x563d784cf4d0 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f60fe59c640 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d784cf4d0 0x563d784d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f60e5ffb640 10 client.17649 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f60fe59c640 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d784cf4d0 0x563d784d18f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b30f418894560c server_cookie=149b052748730ab5 in_seq=0 out_seq=0 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f60e5ffb640 1 -- 172.21.15.66:0/4001679965 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563d783bba70 con 0x563d784cf4d0 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f60e5ffb640 10 client.17649 handle_command_reply: tid=0 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 client.17649 shutdown 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 msgr2=0x563d7846c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7846c130 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009900 tx=0x7f60f8005fd0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 shutdown_connections 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563d784cf4d0 0x563d784d18f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d78458a80 0x563d7846c130 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d7844fea0 0x563d7846bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f60e5ffb640 0 client.17649 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 --2- 172.21.15.66:0/4001679965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d7844f580 0x563d7846b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.769+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 >> 172.21.15.66:0/4001679965 conn(0x563d78449cb0 msgr2=0x563d78455a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 shutdown_connections 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6107567740 1 -- 172.21.15.66:0/4001679965 wait complete. 2024-04-21T09:15:50.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6107567740 20 client.17649 trim_cache size 0 max 16384 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "782a13fd-cd97-41a8-8c02-06b25d753b39": { 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "782a13fd-cd97-41a8-8c02-06b25d753b39", 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:15:50.779 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f60e80a3d20 msgr2=0x7f60e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f60e80a3d20 0x7f60e80a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005940 tx=0x7f60f80058d0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 msgr2=0x7f6100198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f6100198bd0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f60f000d770 tx=0x7f60f000dc30 comp rx=0 tx=0).stop 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 shutdown_connections 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f60e80a3d20 0x7f60e80a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6100105dd0 0x7f610019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6100101a40 0x7f6100198bd0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 --2- 172.21.15.66:0/3276459722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61001010a0 0x7f6100198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 >> 172.21.15.66:0/3276459722 conn(0x7f6100077c40 msgr2=0x7f61000fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 shutdown_connections 2024-04-21T09:15:50.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:15:50.770+0000 7f6104893640 1 -- 172.21.15.66:0/3276459722 wait complete. 2024-04-21T09:15:50.782 INFO:tasks.cephfs.filesystem:scrub status for tag:782a13fd-cd97-41a8-8c02-06b25d753b39 - {'path': '/', 'tag': '782a13fd-cd97-41a8-8c02-06b25d753b39', 'options': 'recursive,force'} 2024-04-21T09:15:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:51 smithi186 ceph-mon[31505]: pgmap v234: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:51 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:15:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:51 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:51 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:15:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:51 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:51.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:51 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:15:51.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:51 smithi083 ceph-mon[31413]: pgmap v234: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:51.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:51 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:15:51.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:51 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:51.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:51 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:15:51.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:51 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:51.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:51 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:15:51.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:51 smithi066 ceph-mon[27953]: pgmap v234: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:51.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:51 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:15:51.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:51 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:51.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:51 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:15:51.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:51 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:15:51.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:51 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:15:53.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:53 smithi186 ceph-mon[31505]: pgmap v235: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:53.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:53 smithi083 ceph-mon[31413]: pgmap v235: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:53.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:53 smithi066 ceph-mon[27953]: pgmap v235: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:15:55.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:55 smithi186 ceph-mon[31505]: pgmap v236: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:55.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:55 smithi083 ceph-mon[31413]: pgmap v236: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:55.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:55 smithi066 ceph-mon[27953]: pgmap v236: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:57.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:57 smithi186 ceph-mon[31505]: pgmap v237: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:57.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:57 smithi083 ceph-mon[31413]: pgmap v237: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:57.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:57 smithi066 ceph-mon[27953]: pgmap v237: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:15:59.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:15:59 smithi186 ceph-mon[31505]: pgmap v238: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:59.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:15:59 smithi083 ceph-mon[31413]: pgmap v238: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:15:59.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:15:59 smithi066 ceph-mon[27953]: pgmap v238: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:16:01.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:01 smithi186 ceph-mon[31505]: pgmap v239: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:01.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:01 smithi083 ceph-mon[31413]: pgmap v239: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:01.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:01 smithi066 ceph-mon[27953]: pgmap v239: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:03.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:03 smithi186 ceph-mon[31505]: pgmap v240: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:03.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:03 smithi083 ceph-mon[31413]: pgmap v240: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:03.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:03 smithi066 ceph-mon[27953]: pgmap v240: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:05 smithi186 ceph-mon[31505]: pgmap v241: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:16:05.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:05 smithi083 ceph-mon[31413]: pgmap v241: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:16:05.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:05 smithi066 ceph-mon[27953]: pgmap v241: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:16:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:07 smithi186 ceph-mon[31505]: pgmap v242: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:16:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:07 smithi186 ceph-mon[31505]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T09:16:07.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:07 smithi083 ceph-mon[31413]: pgmap v242: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:16:07.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:07 smithi083 ceph-mon[31413]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T09:16:07.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:07 smithi066 ceph-mon[27953]: pgmap v242: 129 pgs: 129 active+clean; 259 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:16:07.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:07 smithi066 ceph-mon[27953]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T09:16:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:09 smithi186 ceph-mon[31505]: pgmap v244: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:16:09.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:09 smithi083 ceph-mon[31413]: pgmap v244: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:16:09.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:09 smithi066 ceph-mon[27953]: pgmap v244: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:16:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:10 smithi186 ceph-mon[31505]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T09:16:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:10 smithi186 ceph-mon[31505]: pgmap v246: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:16:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:10 smithi083 ceph-mon[31413]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T09:16:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:10 smithi083 ceph-mon[31413]: pgmap v246: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:16:11.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:10 smithi066 ceph-mon[27953]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T09:16:11.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:10 smithi066 ceph-mon[27953]: pgmap v246: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:16:13.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:13 smithi186 ceph-mon[31505]: pgmap v247: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:16:13.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:13 smithi083 ceph-mon[31413]: pgmap v247: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:16:13.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:13 smithi066 ceph-mon[27953]: pgmap v247: 129 pgs: 129 active+clean; 2.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:16:15.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:15 smithi186 ceph-mon[31505]: pgmap v248: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:16:15.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:15 smithi083 ceph-mon[31413]: pgmap v248: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:16:15.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:15 smithi066 ceph-mon[27953]: pgmap v248: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:16:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:17 smithi186 ceph-mon[31505]: pgmap v249: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-04-21T09:16:17.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:17 smithi083 ceph-mon[31413]: pgmap v249: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-04-21T09:16:17.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:17 smithi066 ceph-mon[27953]: pgmap v249: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-04-21T09:16:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:19 smithi186 ceph-mon[31505]: pgmap v250: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:16:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:19 smithi083 ceph-mon[31413]: pgmap v250: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:16:19.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:19 smithi066 ceph-mon[27953]: pgmap v250: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:16:20.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:16:20.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:16:20.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.899+0000 7f9fe9db6640 1 -- 172.21.15.66:0/4055624865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe410ec00 msgr2=0x7f9fe4110ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.899+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/4055624865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe410ec00 0x7f9fe4110ff0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009cd0 tx=0x7f9fdc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 -- 172.21.15.66:0/4055624865 shutdown_connections 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/4055624865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe410ec00 0x7f9fe4110ff0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/4055624865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe4103310 0x7f9fe41009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/4055624865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe4102ea0 0x7f9fe4100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 -- 172.21.15.66:0/4055624865 >> 172.21.15.66:0/4055624865 conn(0x7f9fe4077360 msgr2=0x7f9fe40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 -- 172.21.15.66:0/4055624865 shutdown_connections 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 -- 172.21.15.66:0/4055624865 wait complete. 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 Processor -- start 2024-04-21T09:16:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.900+0000 7f9fe9db6640 1 -- start start 2024-04-21T09:16:20.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe9db6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe4102ea0 0x7f9fe41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe9db6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe4103310 0x7f9fe41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe9db6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe410ec00 0x7f9fe419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe9db6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41143e0 con 0x7f9fe4102ea0 2024-04-21T09:16:20.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe9db6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4114260 con 0x7f9fe4103310 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe9db6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4114560 con 0x7f9fe410ec00 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe4102ea0 0x7f9fe41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe4102ea0 0x7f9fe41a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56640/0 (socket says 172.21.15.66:56640) 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 -- 172.21.15.66:0/3192091225 learned_addr learned my addr 172.21.15.66:0/3192091225 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe2ffd640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe4103310 0x7f9fe41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe410ec00 0x7f9fe419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 -- 172.21.15.66:0/3192091225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe410ec00 msgr2=0x7f9fe419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe410ec00 0x7f9fe419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 -- 172.21.15.66:0/3192091225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe4103310 msgr2=0x7f9fe41a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe4103310 0x7f9fe41a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 -- 172.21.15.66:0/3192091225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe419b000 con 0x7f9fe4102ea0 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe4102ea0 0x7f9fe41a0680 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400d950 tx=0x7f9fd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.901+0000 7f9fe0ff9640 1 -- 172.21.15.66:0/3192091225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4017070 con 0x7f9fe4102ea0 2024-04-21T09:16:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.902+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe419b2f0 con 0x7f9fe4102ea0 2024-04-21T09:16:20.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.902+0000 7f9fe0ff9640 1 -- 172.21.15.66:0/3192091225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd4009c30 con 0x7f9fe4102ea0 2024-04-21T09:16:20.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.902+0000 7f9fe0ff9640 1 -- 172.21.15.66:0/3192091225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400f620 con 0x7f9fe4102ea0 2024-04-21T09:16:20.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.902+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe41a7600 con 0x7f9fe4102ea0 2024-04-21T09:16:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fe0ff9640 1 -- 172.21.15.66:0/3192091225 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9fd4010460 con 0x7f9fe4102ea0 2024-04-21T09:16:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 Processor -- start 2024-04-21T09:16:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 -- start start 2024-04-21T09:16:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97eae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e1b97441e0 con 0x55e1b97dfea0 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1b96e1390 con 0x55e1b97df580 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.904+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e1b97401a0 con 0x55e1b97e8a80 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97e8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56650/0 (socket says 172.21.15.66:56650) 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 -- 172.21.15.66:0/1044531690 learned_addr learned my addr 172.21.15.66:0/1044531690 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe0ff9640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9fcc0a3d20 0x7f9fcc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe2ffd640 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9fcc0a3d20 0x7f9fcc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 -- 172.21.15.66:0/1044531690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 msgr2=0x55e1b97eae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97eae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 -- 172.21.15.66:0/1044531690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 msgr2=0x55e1b97df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 -- 172.21.15.66:0/1044531690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b95207d0 con 0x55e1b97dfea0 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe0ff9640 1 -- 172.21.15.66:0/3192091225 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f9fd40d0780 con 0x7f9fe4102ea0 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97e8540 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4098860 tx=0x7f9fd405bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14aa4b2f1defe62 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9fcc0a3d20 0x7f9fcc0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc005cc0 tx=0x7f9fdc005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.905+0000 7f9fc9ffb640 1 -- 172.21.15.66:0/1044531690 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4020070 con 0x55e1b97dfea0 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fc9ffb640 1 -- 172.21.15.66:0/1044531690 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd4009de0 con 0x55e1b97dfea0 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fc9ffb640 1 -- 172.21.15.66:0/1044531690 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4018d10 con 0x55e1b97dfea0 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1044531690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 msgr2=0x55e1b97e8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97e8540 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4098860 tx=0x7f9fd405bf50 comp rx=0 tx=0).stop 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1044531690 shutdown_connections 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97e8540 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1044531690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1044531690 >> 172.21.15.66:0/1044531690 conn(0x55e1b97d9cb0 msgr2=0x55e1b97da090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1044531690 shutdown_connections 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.906+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1044531690 wait complete. 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 -- start start 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 Processor -- start 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97df580 0x55e1b97e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97dfea0 0x55e1b97e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97e8a80 0x55e1b97e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e1b97eb810 con 0x55e1b97e8a80 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1b96fd2f0 con 0x55e1b97dfea0 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e1b97eb990 con 0x55e1b97df580 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fec9ab740 10 client.0 _ll_register_callbacks cb 0x55e1b970a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97e8a80 0x55e1b97e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97e8a80 0x55e1b97e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56652/0 (socket says 172.21.15.66:56652) 2024-04-21T09:16:20.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fe3fff640 1 -- 172.21.15.66:0/1404802036 learned_addr learned my addr 172.21.15.66:0/1404802036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97df580 0x55e1b97e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.907+0000 7f9fe2ffd640 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97dfea0 0x55e1b97e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fe3fff640 1 -- 172.21.15.66:0/1404802036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97df580 msgr2=0x55e1b97e5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97df580 0x55e1b97e5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fe3fff640 1 -- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97dfea0 msgr2=0x55e1b97e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97dfea0 0x55e1b97e5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fe3fff640 1 -- 172.21.15.66:0/1404802036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b94563f0 con 0x55e1b97e8a80 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97e8a80 0x55e1b97e64d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc032ff0 tx=0x7f9fdc02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a227ce71be2874d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/1404802036 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc052060 con 0x55e1b97e8a80 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/1404802036 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f9fdc003d80 con 0x55e1b97e8a80 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd4009de0 con 0x55e1b97e8a80 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/1404802036 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9fdc039040 con 0x55e1b97e8a80 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fec9ab740 10 client.17661 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:20.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1b94160f0 con 0x55e1b97e8a80 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.908+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/1404802036 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd4009de0 con 0x55e1b97e8a80 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/1404802036 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9fdc034710 con 0x55e1b97e8a80 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fec9ab740 10 client.17661 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fec9ab740 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fec9ab740 20 client.17661 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b97f8a30 0x55e1b97faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fec9ab740 4 client.17661 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1b9751c00 con 0x55e1b97f8a30 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b97f8a30 0x55e1b97faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fc2ffd640 10 client.17661 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.909+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b97f8a30 0x55e1b97faeb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1be0d11e1a364d6 server_cookie=5e081437d0afe73d in_seq=0 out_seq=0 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/1404802036 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1b9751c00 con 0x55e1b97f8a30 2024-04-21T09:16:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fc2ffd640 10 client.17661 handle_command_reply: tid=0 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 client.17661 shutdown 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97e8a80 msgr2=0x55e1b97e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97e8a80 0x55e1b97e64d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc032ff0 tx=0x7f9fdc02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 shutdown_connections 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b97f8a30 0x55e1b97faeb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97e8a80 0x55e1b97e64d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97dfea0 0x55e1b97e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fc2ffd640 0 client.17661 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/1404802036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97df580 0x55e1b97e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 >> 172.21.15.66:0/1404802036 conn(0x55e1b97d9cb0 msgr2=0x55e1b97dba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 shutdown_connections 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 1 -- 172.21.15.66:0/1404802036 wait complete. 2024-04-21T09:16:20.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.910+0000 7f9fec9ab740 20 client.17661 trim_cache size 0 max 16384 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 Processor -- start 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 -- start start 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97df580 0x55e1b97e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97dfea0 0x55e1b97f4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97e8a80 0x55e1b97da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41b3fd0 con 0x55e1b97df580 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41b18a0 con 0x55e1b97e8a80 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.928+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41af710 con 0x55e1b97dfea0 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97df580 0x55e1b97e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97df580 0x55e1b97e8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56654/0 (socket says 172.21.15.66:56654) 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 -- 172.21.15.66:0/2917745286 learned_addr learned my addr 172.21.15.66:0/2917745286 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 -- 172.21.15.66:0/2917745286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97dfea0 msgr2=0x55e1b97f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97e8a80 0x55e1b97da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97dfea0 0x55e1b97f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 -- 172.21.15.66:0/2917745286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97e8a80 msgr2=0x55e1b97da090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97e8a80 0x55e1b97da090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 -- 172.21.15.66:0/2917745286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b97d9ae0 con 0x55e1b97df580 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97df580 0x55e1b97e8180 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40cdbb0 tx=0x7f9fd409bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ee8c5e1cc9596e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fc97fa640 1 -- 172.21.15.66:0/2917745286 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4020040 con 0x55e1b97df580 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fc97fa640 1 -- 172.21.15.66:0/2917745286 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd409c400 con 0x55e1b97df580 2024-04-21T09:16:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.929+0000 7f9fc97fa640 1 -- 172.21.15.66:0/2917745286 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40b0b40 con 0x55e1b97df580 2024-04-21T09:16:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 -- 172.21.15.66:0/2917745286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97df580 msgr2=0x55e1b97e8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97df580 0x55e1b97e8180 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40cdbb0 tx=0x7f9fd409bd60 comp rx=0 tx=0).stop 2024-04-21T09:16:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 -- 172.21.15.66:0/2917745286 shutdown_connections 2024-04-21T09:16:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97e8a80 0x55e1b97da090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97dfea0 0x55e1b97f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/2917745286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97df580 0x55e1b97e8180 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 -- 172.21.15.66:0/2917745286 >> 172.21.15.66:0/2917745286 conn(0x55e1b97d9cb0 msgr2=0x55e1b97db100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 -- 172.21.15.66:0/2917745286 shutdown_connections 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 -- 172.21.15.66:0/2917745286 wait complete. 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 -- start start 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.930+0000 7f9fec9ab740 1 Processor -- start 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97ecb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fec9ab740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41afd60 con 0x55e1b97dfea0 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41b11d0 con 0x55e1b97df580 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fec9ab740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41b0430 con 0x55e1b97e8a80 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fec9ab740 10 client.0 _ll_register_callbacks cb 0x7f9fdc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97ecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97ecb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56658/0 (socket says 172.21.15.66:56658) 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 -- 172.21.15.66:0/4072268669 learned_addr learned my addr 172.21.15.66:0/4072268669 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe3fff640 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 -- 172.21.15.66:0/4072268669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 msgr2=0x55e1b97ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 -- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 msgr2=0x55e1b97ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 -- 172.21.15.66:0/4072268669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b97eae70 con 0x55e1b97dfea0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fe2ffd640 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97ecb30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8005fd0 tx=0x7f9fd800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f80d23026cd0c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.931+0000 7f9fc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/4072268669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8003d70 con 0x55e1b97dfea0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/4072268669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f9fd80047a0 con 0x55e1b97dfea0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd409c400 con 0x55e1b97dfea0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/4072268669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9fd80049a0 con 0x55e1b97dfea0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 10 client.17667 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1b985f120 con 0x55e1b97dfea0 2024-04-21T09:16:20.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/4072268669 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd409c400 con 0x55e1b97dfea0 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/4072268669 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9fd801c470 con 0x55e1b97dfea0 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 10 client.17667 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 20 client.17667 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b985f900 0x55e1b9861d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 4 client.17667 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1b974ba70 con 0x55e1b985f900 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.932+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b985f900 0x55e1b9861d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fe37fe640 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b985f900 0x55e1b9861d20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3820a60d6e69a546 server_cookie=def31ad65904faaf in_seq=0 out_seq=0 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fc2ffd640 10 client.17667 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fc2ffd640 1 -- 172.21.15.66:0/4072268669 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e1b974ba70 con 0x55e1b985f900 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fc2ffd640 10 client.17667 handle_command_reply: tid=0 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 client.17667 shutdown 2024-04-21T09:16:20.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 msgr2=0x55e1b97ecb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97ecb30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8005fd0 tx=0x7f9fd800bd80 comp rx=0 tx=0).stop 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 shutdown_connections 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e1b985f900 0x55e1b9861d20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e1b97e8a80 0x55e1b97ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1b97dfea0 0x55e1b97ecb30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 --2- 172.21.15.66:0/4072268669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1b97df580 0x55e1b97ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fc2ffd640 0 client.17667 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 >> 172.21.15.66:0/4072268669 conn(0x55e1b97d9cb0 msgr2=0x55e1b97e62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 shutdown_connections 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 1 -- 172.21.15.66:0/4072268669 wait complete. 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.933+0000 7f9fec9ab740 20 client.17667 trim_cache size 0 max 16384 2024-04-21T09:16:20.939 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:16:20.940 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:16:20.940 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:16:20.940 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:16:20.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9fcc0a3d20 msgr2=0x7f9fcc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9fcc0a3d20 0x7f9fcc0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc005cc0 tx=0x7f9fdc005c50 comp rx=0 tx=0).stop 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe4102ea0 msgr2=0x7f9fe41a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe4102ea0 0x7f9fe41a0680 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400d950 tx=0x7f9fd400de10 comp rx=0 tx=0).stop 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 shutdown_connections 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f9fcc0a3d20 0x7f9fcc0a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe410ec00 0x7f9fe419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe4103310 0x7f9fe41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 --2- 172.21.15.66:0/3192091225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fe4102ea0 0x7f9fe41a0680 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 >> 172.21.15.66:0/3192091225 conn(0x7f9fe4077360 msgr2=0x7f9fe4110120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 shutdown_connections 2024-04-21T09:16:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:20.934+0000 7f9fe9db6640 1 -- 172.21.15.66:0/3192091225 wait complete. 2024-04-21T09:16:20.945 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:16:20.945 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:16:21.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 -- 172.21.15.66:0/3436549004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe241021a0 msgr2=0x7efe24102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 --2- 172.21.15.66:0/3436549004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe241021a0 0x7efe24102580 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efe14009900 tx=0x7efe1402bc50 comp rx=0 tx=0).stop 2024-04-21T09:16:21.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 -- 172.21.15.66:0/3436549004 shutdown_connections 2024-04-21T09:16:21.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 --2- 172.21.15.66:0/3436549004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe24107270 0x7efe2410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 --2- 172.21.15.66:0/3436549004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe24102b50 0x7efe24106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 --2- 172.21.15.66:0/3436549004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe241021a0 0x7efe24102580 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 -- 172.21.15.66:0/3436549004 >> 172.21.15.66:0/3436549004 conn(0x7efe240793a0 msgr2=0x7efe240797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 -- 172.21.15.66:0/3436549004 shutdown_connections 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.046+0000 7efe291f4640 1 -- 172.21.15.66:0/3436549004 wait complete. 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 Processor -- start 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 -- start start 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe241021a0 0x7efe24198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24107270 0x7efe24198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe2419d010 0x7efe2419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efe24112230 con 0x7efe24107270 2024-04-21T09:16:21.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe241120b0 con 0x7efe241021a0 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.047+0000 7efe291f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efe241123b0 con 0x7efe2419d010 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24107270 0x7efe24198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24107270 0x7efe24198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56664/0 (socket says 172.21.15.66:56664) 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 -- 172.21.15.66:0/2399175778 learned_addr learned my addr 172.21.15.66:0/2399175778 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2359e640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe2419d010 0x7efe2419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe22d9d640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe241021a0 0x7efe24198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 -- 172.21.15.66:0/2399175778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe2419d010 msgr2=0x7efe2419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe2419d010 0x7efe2419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 -- 172.21.15.66:0/2399175778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe241021a0 msgr2=0x7efe24198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe241021a0 0x7efe24198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 -- 172.21.15.66:0/2399175778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2419db00 con 0x7efe24107270 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe22d9d640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe241021a0 0x7efe24198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe2259c640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24107270 0x7efe24198c90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b470 tx=0x7efe1800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe0bfff640 1 -- 172.21.15.66:0/2399175778 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18015020 con 0x7efe24107270 2024-04-21T09:16:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2419ddf0 con 0x7efe24107270 2024-04-21T09:16:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe0bfff640 1 -- 172.21.15.66:0/2399175778 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe18011d40 con 0x7efe24107270 2024-04-21T09:16:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe0bfff640 1 -- 172.21.15.66:0/2399175778 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1800f340 con 0x7efe24107270 2024-04-21T09:16:21.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.048+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe241a55e0 con 0x7efe24107270 2024-04-21T09:16:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.049+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe24069230 con 0x7efe24107270 2024-04-21T09:16:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.050+0000 7efe0bfff640 1 -- 172.21.15.66:0/2399175778 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7efe18018020 con 0x7efe24107270 2024-04-21T09:16:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.051+0000 7efe0bfff640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7efe0c0a3d20 0x7efe0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.051+0000 7efe0bfff640 1 -- 172.21.15.66:0/2399175778 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7efe180d0630 con 0x7efe24107270 2024-04-21T09:16:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.051+0000 7efe22d9d640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7efe0c0a3d20 0x7efe0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.052+0000 7efe22d9d640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7efe0c0a3d20 0x7efe0c0a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe14036040 tx=0x7efe14000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.056+0000 7efe0bfff640 1 -- 172.21.15.66:0/2399175778 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe1801f020 con 0x7efe24107270 2024-04-21T09:16:21.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.280+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe24102b50 con 0x7efe24107270 2024-04-21T09:16:21.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.281+0000 7efe0bfff640 1 -- 172.21.15.66:0/2399175778 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7efe1800d040 con 0x7efe24107270 2024-04-21T09:16:21.281 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:16:21.282 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:16:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:16:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7efe0c0a3d20 msgr2=0x7efe0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7efe0c0a3d20 0x7efe0c0a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe14036040 tx=0x7efe14000bc0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24107270 msgr2=0x7efe24198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24107270 0x7efe24198c90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b470 tx=0x7efe1800b930 comp rx=0 tx=0).stop 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 shutdown_connections 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7efe0c0a3d20 0x7efe0c0a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe2419d010 0x7efe2419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24107270 0x7efe24198c90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 --2- 172.21.15.66:0/2399175778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe241021a0 0x7efe24198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 >> 172.21.15.66:0/2399175778 conn(0x7efe240793a0 msgr2=0x7efe240fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 shutdown_connections 2024-04-21T09:16:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.283+0000 7efe291f4640 1 -- 172.21.15.66:0/2399175778 wait complete. 2024-04-21T09:16:21.294 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 -- 172.21.15.66:0/3659233815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 msgr2=0x7f645c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 --2- 172.21.15.66:0/3659233815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c101470 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6458009900 tx=0x7f645802bc30 comp rx=0 tx=0).stop 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 -- 172.21.15.66:0/3659233815 shutdown_connections 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 --2- 172.21.15.66:0/3659233815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f645c106160 0x7f645c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 --2- 172.21.15.66:0/3659233815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c101a40 0x7f645c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 --2- 172.21.15.66:0/3659233815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c101470 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 -- 172.21.15.66:0/3659233815 >> 172.21.15.66:0/3659233815 conn(0x7f645c077f50 msgr2=0x7f645c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 -- 172.21.15.66:0/3659233815 shutdown_connections 2024-04-21T09:16:21.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.398+0000 7f646338e640 1 -- 172.21.15.66:0/3659233815 wait complete. 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 Processor -- start 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 -- start start 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f645c106160 0x7f645c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c19a0a0 0x7f645c1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1110c0 con 0x7f645c101090 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f645c110f40 con 0x7f645c106160 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646338e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f645c111240 con 0x7f645c19a0a0 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56684/0 (socket says 172.21.15.66:56684) 2024-04-21T09:16:21.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 -- 172.21.15.66:0/3656674489 learned_addr learned my addr 172.21.15.66:0/3656674489 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f6460929640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f645c106160 0x7f645c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 -- 172.21.15.66:0/3656674489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c19a0a0 msgr2=0x7f645c1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c19a0a0 0x7f645c1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 -- 172.21.15.66:0/3656674489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f645c106160 msgr2=0x7f645c199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f645c106160 0x7f645c199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.399+0000 7f646112a640 1 -- 172.21.15.66:0/3656674489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c1a4e20 con 0x7f645c101090 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.400+0000 7f646112a640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c199620 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6458005940 tx=0x7f645802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.400+0000 7f64527fc640 1 -- 172.21.15.66:0/3656674489 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458043020 con 0x7f645c101090 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.400+0000 7f64527fc640 1 -- 172.21.15.66:0/3656674489 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f645802cbe0 con 0x7f645c101090 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.400+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c1a50b0 con 0x7f645c101090 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.400+0000 7f64527fc640 1 -- 172.21.15.66:0/3656674489 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458003bf0 con 0x7f645c101090 2024-04-21T09:16:21.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.400+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c1a5650 con 0x7f645c101090 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.402+0000 7f64527fc640 1 -- 172.21.15.66:0/3656674489 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6458046020 con 0x7f645c101090 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 Processor -- start 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 -- start start 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f64527fc640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64480a3da0 0x7f64480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752591f250 0x55752591f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752592c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752592ce10 0x55752592f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5575258841f0 con 0x55752591fc00 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575258214b0 con 0x55752591f250 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6465f76740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5575258801b0 con 0x55752592ce10 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f64527fc640 1 -- 172.21.15.66:0/3656674489 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f64580f2d40 con 0x7f645c101090 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f646192b640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64480a3da0 0x7f64480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752592c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752592c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56694/0 (socket says 172.21.15.66:56694) 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 -- 172.21.15.66:0/74563920 learned_addr learned my addr 172.21.15.66:0/74563920 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f646112a640 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752591f250 0x55752591f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f646192b640 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752592ce10 0x55752592f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 -- 172.21.15.66:0/74563920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752592ce10 msgr2=0x55752592f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752592ce10 0x55752592f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 -- 172.21.15.66:0/74563920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752591f250 msgr2=0x55752591f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752591f250 0x55752591f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f6460929640 1 -- 172.21.15.66:0/74563920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575258131b0 con 0x55752591fc00 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f646112a640 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752591f250 0x55752591f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.403+0000 7f646192b640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64480a3da0 0x7f64480a6260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6454009370 tx=0x7f64540149d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6460929640 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752592c840 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f644c00bc50 tx=0x7f644c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d24789a7655b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f64477fe640 1 -- 172.21.15.66:0/74563920 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c015070 con 0x55752591fc00 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f64477fe640 1 -- 172.21.15.66:0/74563920 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f644c01a910 con 0x55752591fc00 2024-04-21T09:16:21.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f64477fe640 1 -- 172.21.15.66:0/74563920 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0117e0 con 0x55752591fc00 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 -- 172.21.15.66:0/74563920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 msgr2=0x55752592c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752592c840 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f644c00bc50 tx=0x7f644c00f730 comp rx=0 tx=0).stop 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 -- 172.21.15.66:0/74563920 shutdown_connections 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752592ce10 0x55752592f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752592c840 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 --2- 172.21.15.66:0/74563920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752591f250 0x55752591f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 -- 172.21.15.66:0/74563920 >> 172.21.15.66:0/74563920 conn(0x557525919ba0 msgr2=0x557525919f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 -- 172.21.15.66:0/74563920 shutdown_connections 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.404+0000 7f6465f76740 1 -- 172.21.15.66:0/74563920 wait complete. 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 -- start start 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 Processor -- start 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591f250 0x55752592c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591fc00 0x557525929a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55752592fd40 con 0x55752591f250 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55752583d500 con 0x55752592ce10 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55752592fec0 con 0x55752591fc00 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f6465f76740 10 client.0 _ll_register_callbacks cb 0x55752580bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f646112a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591f250 0x55752592c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f646112a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591f250 0x55752592c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56700/0 (socket says 172.21.15.66:56700) 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.405+0000 7f646112a640 1 -- 172.21.15.66:0/4236674917 learned_addr learned my addr 172.21.15.66:0/4236674917 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646192b640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6460929640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591fc00 0x557525929a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646112a640 1 -- 172.21.15.66:0/4236674917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591fc00 msgr2=0x557525929a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646112a640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591fc00 0x557525929a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646112a640 1 -- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 msgr2=0x55752592a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646112a640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646112a640 1 -- 172.21.15.66:0/4236674917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575255963f0 con 0x55752591f250 2024-04-21T09:16:21.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646192b640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f646112a640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591f250 0x55752592c700 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f64580be720 tx=0x7f64580be560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad330047a9e56a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6444ff9640 1 -- 172.21.15.66:0/4236674917 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645809be60 con 0x55752591f250 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6444ff9640 1 -- 172.21.15.66:0/4236674917 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f645809f0d0 con 0x55752591f250 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644c01a910 con 0x55752591f250 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6444ff9640 1 -- 172.21.15.66:0/4236674917 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f645803c350 con 0x55752591f250 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 10 client.17685 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575256607d0 con 0x55752591f250 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6444ff9640 1 -- 172.21.15.66:0/4236674917 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f644c01a910 con 0x55752591f250 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6444ff9640 1 -- 172.21.15.66:0/4236674917 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f64580bb090 con 0x55752591f250 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 10 client.17685 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:21.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 10 client.17685 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 20 client.17685 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55752593cff0 0x55752593f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 4 client.17685 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.406+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557525823930 con 0x55752593cff0 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.407+0000 7f6460929640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55752593cff0 0x55752593f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.407+0000 7f6444ff9640 10 client.17685 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.407+0000 7f6460929640 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55752593cff0 0x55752593f460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e0f13df8787cfa6 server_cookie=e6c8ea1d05e4423f in_seq=0 out_seq=0 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6444ff9640 1 -- 172.21.15.66:0/4236674917 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557525823930 con 0x55752593cff0 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6444ff9640 10 client.17685 handle_command_reply: tid=0 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 client.17685 shutdown 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591f250 msgr2=0x55752592c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591f250 0x55752592c700 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f64580be720 tx=0x7f64580be560 comp rx=0 tx=0).stop 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 shutdown_connections 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55752593cff0 0x55752593f460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591fc00 0x557525929a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 --2- 172.21.15.66:0/4236674917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591f250 0x55752592c700 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6444ff9640 0 client.17685 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:21.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 >> 172.21.15.66:0/4236674917 conn(0x557525919ba0 msgr2=0x55752591b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 shutdown_connections 2024-04-21T09:16:21.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 1 -- 172.21.15.66:0/4236674917 wait complete. 2024-04-21T09:16:21.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.408+0000 7f6465f76740 20 client.17685 trim_cache size 0 max 16384 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.421+0000 7f6465f76740 1 Processor -- start 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.421+0000 7f6465f76740 1 -- start start 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6465f76740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 0x557525938c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6465f76740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752591b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6465f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6465f76740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1afa30 con 0x55752591fc00 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6465f76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1ad300 con 0x55752592ce10 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6465f76740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1ab170 con 0x55752591f250 2024-04-21T09:16:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752591b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752591b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56716/0 (socket says 172.21.15.66:56716) 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 -- 172.21.15.66:0/3019712918 learned_addr learned my addr 172.21.15.66:0/3019712918 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 -- 172.21.15.66:0/3019712918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 msgr2=0x557525938c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f646192b640 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 0x557525938c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 -- 172.21.15.66:0/3019712918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 msgr2=0x55752592c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 -- 172.21.15.66:0/3019712918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55752592a380 con 0x55752591fc00 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f646192b640 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6460929640 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752591b700 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x557525920040 tx=0x7f644c0114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbfa927c2f877d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6446ffd640 1 -- 172.21.15.66:0/3019712918 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c02b070 con 0x55752591fc00 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6446ffd640 1 -- 172.21.15.66:0/3019712918 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f644c005580 con 0x55752591fc00 2024-04-21T09:16:21.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.422+0000 7f6446ffd640 1 -- 172.21.15.66:0/3019712918 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c015040 con 0x55752591fc00 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 -- 172.21.15.66:0/3019712918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 msgr2=0x55752591b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752591b700 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x557525920040 tx=0x7f644c0114a0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 -- 172.21.15.66:0/3019712918 shutdown_connections 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x55752592c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x55752591b700 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 --2- 172.21.15.66:0/3019712918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 0x557525938c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 -- 172.21.15.66:0/3019712918 >> 172.21.15.66:0/3019712918 conn(0x557525919ba0 msgr2=0x55752591aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 -- 172.21.15.66:0/3019712918 shutdown_connections 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 -- 172.21.15.66:0/3019712918 wait complete. 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 -- start start 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.423+0000 7f6465f76740 1 Processor -- start 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6465f76740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 0x557525930da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6465f76740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x5575259312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6465f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x557525931820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6465f76740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1ab7c0 con 0x55752591fc00 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6465f76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1acc30 con 0x55752592ce10 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6465f76740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1abe90 con 0x55752591f250 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6465f76740 10 client.0 _ll_register_callbacks cb 0x7f64580be7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x5575259312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x5575259312e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56718/0 (socket says 172.21.15.66:56718) 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 -- 172.21.15.66:0/64930053 learned_addr learned my addr 172.21.15.66:0/64930053 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f646112a640 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 0x557525930da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f646192b640 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x557525931820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 -- 172.21.15.66:0/64930053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 msgr2=0x557525930da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 0x557525930da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 -- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 msgr2=0x557525931820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x557525931820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 -- 172.21.15.66:0/64930053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575259a2830 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6460929640 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x5575259312e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f644c00f770 tx=0x7f644c026270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759e3e24798c64f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.424+0000 7f6444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6444ff9640 1 -- 172.21.15.66:0/64930053 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c02b070 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6444ff9640 1 -- 172.21.15.66:0/64930053 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f644c0265f0 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644c005580 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6444ff9640 1 -- 172.21.15.66:0/64930053 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f644c03c050 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 10 client.17691 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575259a3a80 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6444ff9640 1 -- 172.21.15.66:0/64930053 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f644c005580 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6444ff9640 1 -- 172.21.15.66:0/64930053 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f644c025470 con 0x55752591fc00 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 10 client.17691 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:21.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 10 client.17691 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 20 client.17691 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5575259a4020 0x5575259a6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 4 client.17691 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557525891c10 con 0x5575259a4020 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.425+0000 7f646112a640 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5575259a4020 0x5575259a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f646112a640 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5575259a4020 0x5575259a6440 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7c9f475587b421e server_cookie=1aa8ffbcafb800a2 in_seq=0 out_seq=0 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6444ff9640 10 client.17691 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6444ff9640 1 -- 172.21.15.66:0/64930053 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557525891c10 con 0x5575259a4020 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6444ff9640 10 client.17691 handle_command_reply: tid=0 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 client.17691 shutdown 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 msgr2=0x5575259312e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x5575259312e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f644c00f770 tx=0x7f644c026270 comp rx=0 tx=0).stop 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 shutdown_connections 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x5575259a4020 0x5575259a6440 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55752592ce10 0x557525931820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55752591fc00 0x5575259312e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6444ff9640 0 client.17691 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 --2- 172.21.15.66:0/64930053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55752591f250 0x557525930da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 >> 172.21.15.66:0/64930053 conn(0x557525919ba0 msgr2=0x557525918c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 shutdown_connections 2024-04-21T09:16:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 1 -- 172.21.15.66:0/64930053 wait complete. 2024-04-21T09:16:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.426+0000 7f6465f76740 20 client.17691 trim_cache size 0 max 16384 2024-04-21T09:16:21.434 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:16:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64480a3da0 msgr2=0x7f64480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64480a3da0 0x7f64480a6260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6454009370 tx=0x7f64540149d0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 msgr2=0x7f645c199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c199620 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6458005940 tx=0x7f645802c7e0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 shutdown_connections 2024-04-21T09:16:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64480a3da0 0x7f64480a6260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c19a0a0 0x7f645c1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f645c106160 0x7f645c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 --2- 172.21.15.66:0/3656674489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c101090 0x7f645c199620 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 >> 172.21.15.66:0/3656674489 conn(0x7f645c077f50 msgr2=0x7f645c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 shutdown_connections 2024-04-21T09:16:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.427+0000 7f646338e640 1 -- 172.21.15.66:0/3656674489 wait complete. 2024-04-21T09:16:21.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:16:21.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3444281811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc1010a0 msgr2=0x7f02dc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3444281811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc1010a0 0x7f02dc101480 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009ca0 tx=0x7f02d402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3444281811 shutdown_connections 2024-04-21T09:16:21.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3444281811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02dc106170 0x7f02dc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3444281811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02dc101a50 0x7f02dc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3444281811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc1010a0 0x7f02dc101480 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3444281811 >> 172.21.15.66:0/3444281811 conn(0x7f02dc077eb0 msgr2=0x7f02dc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3444281811 shutdown_connections 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.541+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3444281811 wait complete. 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 Processor -- start 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 -- start start 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02dc1010a0 0x7f02dc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02dc106170 0x7f02dc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc19cf70 0x7f02dc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1121f0 con 0x7f02dc19cf70 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc112070 con 0x7f02dc1010a0 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e2ea5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc112370 con 0x7f02dc106170 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e1442640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc19cf70 0x7f02dc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e1442640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc19cf70 0x7f02dc19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56734/0 (socket says 172.21.15.66:56734) 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e1442640 1 -- 172.21.15.66:0/3143666259 learned_addr learned my addr 172.21.15.66:0/3143666259 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.542+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02dc1010a0 0x7f02dc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e1442640 1 -- 172.21.15.66:0/3143666259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02dc106170 msgr2=0x7f02dc198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e1442640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02dc106170 0x7f02dc198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e1442640 1 -- 172.21.15.66:0/3143666259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02dc1010a0 msgr2=0x7f02dc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e1442640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02dc1010a0 0x7f02dc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e1442640 1 -- 172.21.15.66:0/3143666259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc19da60 con 0x7f02dc19cf70 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02dc1010a0 0x7f02dc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e1442640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc19cf70 0x7f02dc19d420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c970 tx=0x7f02cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02d9ffb640 1 -- 172.21.15.66:0/3143666259 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc007bb0 con 0x7f02dc19cf70 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc19dd50 con 0x7f02dc19cf70 2024-04-21T09:16:21.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02d9ffb640 1 -- 172.21.15.66:0/3143666259 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc007d50 con 0x7f02dc19cf70 2024-04-21T09:16:21.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02d9ffb640 1 -- 172.21.15.66:0/3143666259 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc004e60 con 0x7f02dc19cf70 2024-04-21T09:16:21.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.543+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc1a5590 con 0x7f02dc19cf70 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.545+0000 7f02d9ffb640 1 -- 172.21.15.66:0/3143666259 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f02cc005020 con 0x7f02dc19cf70 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.545+0000 7f02e5b74740 1 Processor -- start 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e5b74740 1 -- start start 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 0x55dde42d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 0x55dde42d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e5b74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dde42301f0 con 0x55dde42cb4d0 2024-04-21T09:16:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e5b74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dde41cd4b0 con 0x55dde42d4a00 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e5b74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dde422c1b0 con 0x55dde42cbe80 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56738/0 (socket says 172.21.15.66:56738) 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 -- 172.21.15.66:0/513528898 learned_addr learned my addr 172.21.15.66:0/513528898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02d9ffb640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f02c40a3d20 0x7f02c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02e1442640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f02c40a3d20 0x7f02c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 -- 172.21.15.66:0/513528898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 msgr2=0x55dde42d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 --2- 172.21.15.66:0/513528898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 0x55dde42d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 -- 172.21.15.66:0/513528898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 msgr2=0x55dde42d6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 --2- 172.21.15.66:0/513528898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 0x55dde42d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 -- 172.21.15.66:0/513528898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dde41bf1b0 con 0x55dde42cb4d0 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02d9ffb640 1 -- 172.21.15.66:0/3143666259 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f02cc0d0890 con 0x7f02dc19cf70 2024-04-21T09:16:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.546+0000 7f02dbfff640 1 --2- 172.21.15.66:0/513528898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42cb8b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f02d000ab20 tx=0x7f02d000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cad879c76a6b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02c2ffd640 1 -- 172.21.15.66:0/513528898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0005020 con 0x55dde42cb4d0 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02c2ffd640 1 -- 172.21.15.66:0/513528898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d00047a0 con 0x55dde42cb4d0 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02c2ffd640 1 -- 172.21.15.66:0/513528898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0010660 con 0x55dde42cb4d0 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 -- 172.21.15.66:0/513528898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 msgr2=0x55dde42cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e1442640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f02c40a3d20 0x7f02c40a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02cc09c100 tx=0x7f02cc098040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 --2- 172.21.15.66:0/513528898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42cb8b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f02d000ab20 tx=0x7f02d000e2f0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 -- 172.21.15.66:0/513528898 shutdown_connections 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 --2- 172.21.15.66:0/513528898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 0x55dde42d6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 --2- 172.21.15.66:0/513528898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 0x55dde42d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 --2- 172.21.15.66:0/513528898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42cb8b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 -- 172.21.15.66:0/513528898 >> 172.21.15.66:0/513528898 conn(0x55dde42c5ba0 msgr2=0x55dde42c5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 -- 172.21.15.66:0/513528898 shutdown_connections 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.547+0000 7f02e5b74740 1 -- 172.21.15.66:0/513528898 wait complete. 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 -- start start 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 Processor -- start 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42d4a00 0x55dde42d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dde42d7970 con 0x55dde42cb4d0 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dde41e9500 con 0x55dde42cbe80 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dde42d7af0 con 0x55dde42d4a00 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e5b74740 10 client.0 _ll_register_callbacks cb 0x55dde41b7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56746/0 (socket says 172.21.15.66:56746) 2024-04-21T09:16:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02dbfff640 1 -- 172.21.15.66:0/3272486979 learned_addr learned my addr 172.21.15.66:0/3272486979 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.548+0000 7f02dbfff640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42d4a00 0x55dde42d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02dbfff640 1 -- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42d4a00 msgr2=0x55dde42d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02dbfff640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42d4a00 0x55dde42d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02dbfff640 1 -- 172.21.15.66:0/3272486979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 msgr2=0x55dde42d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02dbfff640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02dbfff640 1 -- 172.21.15.66:0/3272486979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dde3f423f0 con 0x55dde42cb4d0 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02dbfff640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42d4340 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f02d0004700 tx=0x7f02d0005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b80a2ee0690a9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02bbfff640 1 -- 172.21.15.66:0/3272486979 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0028640 con 0x55dde42cb4d0 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02bbfff640 1 -- 172.21.15.66:0/3272486979 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f02d00287e0 con 0x55dde42cb4d0 2024-04-21T09:16:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d00047a0 con 0x55dde42cb4d0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02bbfff640 1 -- 172.21.15.66:0/3272486979 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02d00289e0 con 0x55dde42cb4d0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02e5b74740 10 client.17703 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dde400c7d0 con 0x55dde42cb4d0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02bbfff640 1 -- 172.21.15.66:0/3272486979 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d00047a0 con 0x55dde42cb4d0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.549+0000 7f02bbfff640 1 -- 172.21.15.66:0/3272486979 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02d0026020 con 0x55dde42cb4d0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e5b74740 10 client.17703 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e5b74740 10 client.17703 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e5b74740 20 client.17703 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde42e4bd0 0x55dde42e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e5b74740 4 client.17703 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dde41cf930 con 0x55dde42e4bd0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde42e4bd0 0x55dde42e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde42e4bd0 0x55dde42e6fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54f32058f7d64276 server_cookie=464a07f8b37bc607 in_seq=0 out_seq=0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.550+0000 7f02bbfff640 10 client.17703 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02bbfff640 1 -- 172.21.15.66:0/3272486979 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dde41cf930 con 0x55dde42e4bd0 2024-04-21T09:16:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02bbfff640 10 client.17703 handle_command_reply: tid=0 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 client.17703 shutdown 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 msgr2=0x55dde42d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42d4340 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f02d0004700 tx=0x7f02d0005e10 comp rx=0 tx=0).stop 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 shutdown_connections 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42d4a00 0x55dde42d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde42e4bd0 0x55dde42e6fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3272486979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42d4340 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 >> 172.21.15.66:0/3272486979 conn(0x55dde42c5ba0 msgr2=0x55dde42c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02bbfff640 0 client.17703 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 shutdown_connections 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 1 -- 172.21.15.66:0/3272486979 wait complete. 2024-04-21T09:16:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.551+0000 7f02e5b74740 20 client.17703 trim_cache size 0 max 16384 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 Processor -- start 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 -- start start 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cb4d0 0x55dde42d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42d4a00 0x55dde42d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1b1c40 con 0x55dde42d4a00 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1af510 con 0x55dde42cbe80 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e5b74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1ad380 con 0x55dde42cb4d0 2024-04-21T09:16:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42d4a00 0x55dde42d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42d4a00 0x55dde42d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56748/0 (socket says 172.21.15.66:56748) 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 -- 172.21.15.66:0/1792477528 learned_addr learned my addr 172.21.15.66:0/1792477528 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cb4d0 0x55dde42d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02e0c41640 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 -- 172.21.15.66:0/1792477528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cb4d0 msgr2=0x55dde42d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cb4d0 0x55dde42d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 -- 172.21.15.66:0/1792477528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 msgr2=0x55dde42d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.565+0000 7f02dbfff640 1 -- 172.21.15.66:0/1792477528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dde42c6930 con 0x55dde42d4a00 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02dbfff640 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42d4a00 0x55dde42d4340 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f02d00108f0 tx=0x7f02d0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58aaaac2f6f06e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02c27fc640 1 -- 172.21.15.66:0/1792477528 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d00047a0 con 0x55dde42d4a00 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02c27fc640 1 -- 172.21.15.66:0/1792477528 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d001d030 con 0x55dde42d4a00 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02c27fc640 1 -- 172.21.15.66:0/1792477528 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0028de0 con 0x55dde42d4a00 2024-04-21T09:16:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02e5b74740 1 -- 172.21.15.66:0/1792477528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42d4a00 msgr2=0x55dde42d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02e5b74740 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42d4a00 0x55dde42d4340 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f02d00108f0 tx=0x7f02d0001e70 comp rx=0 tx=0).stop 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02e5b74740 1 -- 172.21.15.66:0/1792477528 shutdown_connections 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02e5b74740 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42d4a00 0x55dde42d4340 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02e5b74740 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42cbe80 0x55dde42d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02e5b74740 1 --2- 172.21.15.66:0/1792477528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cb4d0 0x55dde42d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.566+0000 7f02e5b74740 1 -- 172.21.15.66:0/1792477528 >> 172.21.15.66:0/1792477528 conn(0x55dde42c5ba0 msgr2=0x55dde42cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 -- 172.21.15.66:0/1792477528 shutdown_connections 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 -- 172.21.15.66:0/1792477528 wait complete. 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 -- start start 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 Processor -- start 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 0x55dde42e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 0x55dde42e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1ad9d0 con 0x55dde42cb4d0 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1aee40 con 0x55dde42d4a00 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1ae0a0 con 0x55dde42cbe80 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02e5b74740 10 client.0 _ll_register_callbacks cb 0x7f02d0005540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.567+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42ea1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56764/0 (socket says 172.21.15.66:56764) 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 -- 172.21.15.66:0/3828017245 learned_addr learned my addr 172.21.15.66:0/3828017245 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 0x55dde42e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 -- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 msgr2=0x55dde42e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 0x55dde42e75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 -- 172.21.15.66:0/3828017245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 msgr2=0x55dde42e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 0x55dde42e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 -- 172.21.15.66:0/3828017245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dde42e8150 con 0x55dde42cb4d0 2024-04-21T09:16:21.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02dbfff640 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42ea1f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f02d00047d0 tx=0x7f02d0004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429b467eb5f666fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02bbfff640 1 -- 172.21.15.66:0/3828017245 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0028d80 con 0x55dde42cb4d0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d001d030 con 0x55dde42cb4d0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02bbfff640 1 -- 172.21.15.66:0/3828017245 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f02d0028440 con 0x55dde42cb4d0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02bbfff640 1 -- 172.21.15.66:0/3828017245 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02d0004b60 con 0x55dde42cb4d0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02e5b74740 10 client.17709 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dde434abe0 con 0x55dde42cb4d0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.568+0000 7f02bbfff640 1 -- 172.21.15.66:0/3828017245 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d001d030 con 0x55dde42cb4d0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02bbfff640 1 -- 172.21.15.66:0/3828017245 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02d0035070 con 0x55dde42cb4d0 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e5b74740 10 client.17709 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e5b74740 10 client.17709 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e5b74740 20 client.17709 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde434b140 0x55dde434d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e5b74740 4 client.17709 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dde423dc10 con 0x55dde434b140 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde434b140 0x55dde434d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02bbfff640 10 client.17709 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.569+0000 7f02e0c41640 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde434b140 0x55dde434d530 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=526fa89e77913479 server_cookie=6b4af4115e9137dc in_seq=0 out_seq=0 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02bbfff640 1 -- 172.21.15.66:0/3828017245 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dde423dc10 con 0x55dde434b140 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02bbfff640 10 client.17709 handle_command_reply: tid=0 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 client.17709 shutdown 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 msgr2=0x55dde42ea1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42ea1f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f02d00047d0 tx=0x7f02d0004800 comp rx=0 tx=0).stop 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 shutdown_connections 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dde42d4a00 0x55dde42e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dde42cbe80 0x55dde42e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55dde434b140 0x55dde434d530 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 --2- 172.21.15.66:0/3828017245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dde42cb4d0 0x55dde42ea1f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 >> 172.21.15.66:0/3828017245 conn(0x55dde42c5ba0 msgr2=0x55dde42d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02bbfff640 0 client.17709 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:21.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 shutdown_connections 2024-04-21T09:16:21.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 1 -- 172.21.15.66:0/3828017245 wait complete. 2024-04-21T09:16:21.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.570+0000 7f02e5b74740 20 client.17709 trim_cache size 0 max 16384 2024-04-21T09:16:21.577 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f02c40a3d20 msgr2=0x7f02c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f02c40a3d20 0x7f02c40a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02cc09c100 tx=0x7f02cc098040 comp rx=0 tx=0).stop 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc19cf70 msgr2=0x7f02dc19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc19cf70 0x7f02dc19d420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c970 tx=0x7f02cc00ce30 comp rx=0 tx=0).stop 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 shutdown_connections 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f02c40a3d20 0x7f02c40a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc19cf70 0x7f02dc19d420 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02dc106170 0x7f02dc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 --2- 172.21.15.66:0/3143666259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02dc1010a0 0x7f02dc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 >> 172.21.15.66:0/3143666259 conn(0x7f02dc077eb0 msgr2=0x7f02dc100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 shutdown_connections 2024-04-21T09:16:21.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.571+0000 7f02e2ea5640 1 -- 172.21.15.66:0/3143666259 wait complete. 2024-04-21T09:16:21.580 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:21 smithi066 ceph-mon[27953]: pgmap v251: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:16:21.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:16:21.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:21 smithi186 ceph-mon[31505]: pgmap v251: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:16:21.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:21 smithi083 ceph-mon[31413]: pgmap v251: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:16:21.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 -- 172.21.15.66:0/1332164687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c107270 msgr2=0x7f262c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 --2- 172.21.15.66:0/1332164687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c107270 0x7f262c10ee40 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2624009ca0 tx=0x7f262402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 -- 172.21.15.66:0/1332164687 shutdown_connections 2024-04-21T09:16:21.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 --2- 172.21.15.66:0/1332164687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c107270 0x7f262c10ee40 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 --2- 172.21.15.66:0/1332164687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f262c06b1f0 0x7f262c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 --2- 172.21.15.66:0/1332164687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f262c06a840 0x7f262c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 -- 172.21.15.66:0/1332164687 >> 172.21.15.66:0/1332164687 conn(0x7f262c077eb0 msgr2=0x7f262c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 -- 172.21.15.66:0/1332164687 shutdown_connections 2024-04-21T09:16:21.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.685+0000 7f2632c6a640 1 -- 172.21.15.66:0/1332164687 wait complete. 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 Processor -- start 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 -- start start 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f262c06a840 0x7f262c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f262c107270 0x7f262c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c19ceb0 0x7f262c19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1121b0 con 0x7f262c19ceb0 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f262c112030 con 0x7f262c06a840 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2632c6a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f262c112330 con 0x7f262c107270 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2631207640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c19ceb0 0x7f262c19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2631207640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c19ceb0 0x7f262c19d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56784/0 (socket says 172.21.15.66:56784) 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2631207640 1 -- 172.21.15.66:0/3778398668 learned_addr learned my addr 172.21.15.66:0/3778398668 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.686+0000 7f2630a06640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f262c06a840 0x7f262c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2631207640 1 -- 172.21.15.66:0/3778398668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f262c107270 msgr2=0x7f262c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2631207640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f262c107270 0x7f262c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2631207640 1 -- 172.21.15.66:0/3778398668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f262c06a840 msgr2=0x7f262c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2631207640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f262c06a840 0x7f262c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2631207640 1 -- 172.21.15.66:0/3778398668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c19d9a0 con 0x7f262c19ceb0 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2630a06640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f262c06a840 0x7f262c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2631207640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c19ceb0 0x7f262c19d360 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2624005e30 tx=0x7f2624034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2629ffb640 1 -- 172.21.15.66:0/3778398668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624039040 con 0x7f262c19ceb0 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c19dc30 con 0x7f262c19ceb0 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2629ffb640 1 -- 172.21.15.66:0/3778398668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2624044070 con 0x7f262c19ceb0 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2629ffb640 1 -- 172.21.15.66:0/3778398668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624003e60 con 0x7f262c19ceb0 2024-04-21T09:16:21.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.687+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1a5450 con 0x7f262c19ceb0 2024-04-21T09:16:21.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 Processor -- start 2024-04-21T09:16:21.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f2629ffb640 1 -- 172.21.15.66:0/3778398668 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f26240324c0 con 0x7f262c19ceb0 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 -- start start 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f614d0 0x55f9b8f618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8ec61f0 con 0x55f9b8f614d0 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8e634b0 con 0x55f9b8f6aa00 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.690+0000 7f263585e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8ec21b0 con 0x55f9b8f61e80 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f614d0 0x55f9b8f618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f614d0 0x55f9b8f618b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56786/0 (socket says 172.21.15.66:56786) 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 -- 172.21.15.66:0/1105877656 learned_addr learned my addr 172.21.15.66:0/1105877656 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f2630a06640 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f2629ffb640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26180a3d20 0x7f26180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 -- 172.21.15.66:0/1105877656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f61e80 msgr2=0x55f9b8f6a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 -- 172.21.15.66:0/1105877656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 msgr2=0x55f9b8f6ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f2631207640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26180a3d20 0x7f26180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 -- 172.21.15.66:0/1105877656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b8e551b0 con 0x55f9b8f614d0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f2629ffb640 1 -- 172.21.15.66:0/3778398668 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f26240f2eb0 con 0x7f262c19ceb0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f262bfff640 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f614d0 0x55f9b8f618b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f261c00bfc0 tx=0x7f261c00b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c81ed267a159a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f2622ffd640 1 -- 172.21.15.66:0/1105877656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c005220 con 0x55f9b8f614d0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f2622ffd640 1 -- 172.21.15.66:0/1105877656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f261c00fe10 con 0x55f9b8f614d0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.691+0000 7f2622ffd640 1 -- 172.21.15.66:0/1105877656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c012650 con 0x55f9b8f614d0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 -- 172.21.15.66:0/1105877656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f614d0 msgr2=0x55f9b8f618b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f2631207640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26180a3d20 0x7f26180a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26240be830 tx=0x7f2624091620 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f614d0 0x55f9b8f618b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f261c00bfc0 tx=0x7f261c00b440 comp rx=0 tx=0).stop 2024-04-21T09:16:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 -- 172.21.15.66:0/1105877656 shutdown_connections 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 --2- 172.21.15.66:0/1105877656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f614d0 0x55f9b8f618b0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 -- 172.21.15.66:0/1105877656 >> 172.21.15.66:0/1105877656 conn(0x55f9b8f5bba0 msgr2=0x55f9b8f5bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 -- 172.21.15.66:0/1105877656 shutdown_connections 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 -- 172.21.15.66:0/1105877656 wait complete. 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.692+0000 7f263585e740 1 -- start start 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 1 Processor -- start 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f6a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f61e80 0x55f9b8f676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f67cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8f6d970 con 0x55f9b8f6aa00 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8e7f500 con 0x55f9b8f61e80 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8f6daf0 con 0x55f9b8f614d0 2024-04-21T09:16:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f263585e740 10 client.0 _ll_register_callbacks cb 0x55f9b8e4db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f262bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f67cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f262bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f67cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56794/0 (socket says 172.21.15.66:56794) 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f262bfff640 1 -- 172.21.15.66:0/555145102 learned_addr learned my addr 172.21.15.66:0/555145102 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f262bfff640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f6a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.693+0000 7f2630a06640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f61e80 0x55f9b8f676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f262bfff640 1 -- 172.21.15.66:0/555145102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 msgr2=0x55f9b8f6a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f262bfff640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f6a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f262bfff640 1 -- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f61e80 msgr2=0x55f9b8f676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f262bfff640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f61e80 0x55f9b8f676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f262bfff640 1 -- 172.21.15.66:0/555145102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b8bd83f0 con 0x55f9b8f6aa00 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f2630a06640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f61e80 0x55f9b8f676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f262bfff640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f67cd0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b740 tx=0x7f261c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f42b87d6dea7a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f260bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f260bfff640 1 -- 172.21.15.66:0/555145102 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c005480 con 0x55f9b8f6aa00 2024-04-21T09:16:21.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f260bfff640 1 -- 172.21.15.66:0/555145102 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f261c0047a0 con 0x55f9b8f6aa00 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261c00fe10 con 0x55f9b8f6aa00 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f260bfff640 1 -- 172.21.15.66:0/555145102 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f261c0049a0 con 0x55f9b8f6aa00 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 10 client.17721 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9b8ca27d0 con 0x55f9b8f6aa00 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f260bfff640 1 -- 172.21.15.66:0/555145102 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f261c00fe10 con 0x55f9b8f6aa00 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f260bfff640 1 -- 172.21.15.66:0/555145102 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f261c01db60 con 0x55f9b8f6aa00 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 10 client.17721 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 20 client.17721 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8f7ab40 0x55f9b8f7cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.694+0000 7f263585e740 4 client.17721 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.695+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b8e65930 con 0x55f9b8f7ab40 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.695+0000 7f2630a06640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8f7ab40 0x55f9b8f7cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.695+0000 7f260bfff640 10 client.17721 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.695+0000 7f2630a06640 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8f7ab40 0x55f9b8f7cf30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c20fa5f21a7ad06 server_cookie=6871caeb6bbcddf4 in_seq=0 out_seq=0 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f260bfff640 1 -- 172.21.15.66:0/555145102 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9b8e65930 con 0x55f9b8f7ab40 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f260bfff640 10 client.17721 handle_command_reply: tid=0 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 client.17721 shutdown 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f6aa00 msgr2=0x55f9b8f67cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f67cd0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b740 tx=0x7f261c00b260 comp rx=0 tx=0).stop 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 shutdown_connections 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8f7ab40 0x55f9b8f7cf30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f67cd0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f260bfff640 0 client.17721 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f61e80 0x55f9b8f676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 --2- 172.21.15.66:0/555145102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f6a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 >> 172.21.15.66:0/555145102 conn(0x55f9b8f5bba0 msgr2=0x55f9b8f5d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 shutdown_connections 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 1 -- 172.21.15.66:0/555145102 wait complete. 2024-04-21T09:16:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.696+0000 7f263585e740 20 client.17721 trim_cache size 0 max 16384 2024-04-21T09:16:21.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.709+0000 7f263585e740 1 Processor -- start 2024-04-21T09:16:21.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f263585e740 1 -- start start 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f263585e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f76860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f263585e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f5da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f263585e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f263585e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1b1bc0 con 0x55f9b8f61e80 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f263585e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1af490 con 0x55f9b8f6aa00 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f263585e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1ad300 con 0x55f9b8f614d0 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f5da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f5da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56802/0 (socket says 172.21.15.66:56802) 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 -- 172.21.15.66:0/680652189 learned_addr learned my addr 172.21.15.66:0/680652189 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 -- 172.21.15.66:0/680652189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 msgr2=0x55f9b8f76860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 --2- 172.21.15.66:0/680652189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f76860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 -- 172.21.15.66:0/680652189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 msgr2=0x55f9b8f69df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 --2- 172.21.15.66:0/680652189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 -- 172.21.15.66:0/680652189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b8f7d390 con 0x55f9b8f61e80 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f2630a06640 1 --2- 172.21.15.66:0/680652189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f5da80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2614008cf0 tx=0x7f2614004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75ae9e425faa4e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f26227fc640 1 -- 172.21.15.66:0/680652189 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614003d00 con 0x55f9b8f61e80 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f26227fc640 1 -- 172.21.15.66:0/680652189 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f261401c8d0 con 0x55f9b8f61e80 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.710+0000 7f26227fc640 1 -- 172.21.15.66:0/680652189 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614004a00 con 0x55f9b8f61e80 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 -- 172.21.15.66:0/680652189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 msgr2=0x55f9b8f5da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 --2- 172.21.15.66:0/680652189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f5da80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2614008cf0 tx=0x7f2614004580 comp rx=0 tx=0).stop 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 -- 172.21.15.66:0/680652189 shutdown_connections 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 --2- 172.21.15.66:0/680652189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 --2- 172.21.15.66:0/680652189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f5da80 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 --2- 172.21.15.66:0/680652189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f76860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 -- 172.21.15.66:0/680652189 >> 172.21.15.66:0/680652189 conn(0x55f9b8f5bba0 msgr2=0x55f9b8f666f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 -- 172.21.15.66:0/680652189 shutdown_connections 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 -- 172.21.15.66:0/680652189 wait complete. 2024-04-21T09:16:21.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 -- start start 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.711+0000 7f263585e740 1 Processor -- start 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f72e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1ad950 con 0x55f9b8f61e80 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1aedc0 con 0x55f9b8f6aa00 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1ae020 con 0x55f9b8f614d0 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 10 client.0 _ll_register_callbacks cb 0x7f261c005270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56804/0 (socket says 172.21.15.66:56804) 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 -- 172.21.15.66:0/2278241795 learned_addr learned my addr 172.21.15.66:0/2278241795 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f262bfff640 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f72e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 -- 172.21.15.66:0/2278241795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 msgr2=0x55f9b8f72e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f72e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f262bfff640 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 -- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 msgr2=0x55f9b8f6e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 -- 172.21.15.66:0/2278241795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b8f6ed50 con 0x55f9b8f61e80 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f2630a06640 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6e2d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f261401c650 tx=0x7f261401c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466aa500ed44841e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f260bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f260bfff640 1 -- 172.21.15.66:0/2278241795 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261402a070 con 0x55f9b8f61e80 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f260bfff640 1 -- 172.21.15.66:0/2278241795 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2614005230 con 0x55f9b8f61e80 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.712+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261401c8d0 con 0x55f9b8f61e80 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f260bfff640 1 -- 172.21.15.66:0/2278241795 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2614036050 con 0x55f9b8f61e80 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 10 client.17727 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9b8fe0b10 con 0x55f9b8f61e80 2024-04-21T09:16:21.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f260bfff640 1 -- 172.21.15.66:0/2278241795 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f261401c8d0 con 0x55f9b8f61e80 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f260bfff640 1 -- 172.21.15.66:0/2278241795 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f26140104c0 con 0x55f9b8f61e80 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 10 client.17727 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 20 client.17727 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8fe1290 0x55f9b8fe36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 4 client.17727 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9b8ed3c10 con 0x55f9b8fe1290 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.713+0000 7f262bfff640 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8fe1290 0x55f9b8fe36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f260bfff640 10 client.17727 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f262bfff640 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8fe1290 0x55f9b8fe36b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670fb6be550fbee1 server_cookie=dbda23a42d4eed0e in_seq=0 out_seq=0 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f260bfff640 1 -- 172.21.15.66:0/2278241795 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9b8ed3c10 con 0x55f9b8fe1290 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f260bfff640 10 client.17727 handle_command_reply: tid=0 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 client.17727 shutdown 2024-04-21T09:16:21.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 msgr2=0x55f9b8f6e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6e2d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f261401c650 tx=0x7f261401c680 comp rx=0 tx=0).stop 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 shutdown_connections 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f9b8fe1290 0x55f9b8fe36b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b8f6aa00 0x55f9b8f6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9b8f61e80 0x55f9b8f6e2d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f260bfff640 0 client.17727 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 --2- 172.21.15.66:0/2278241795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f9b8f614d0 0x55f9b8f72e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 >> 172.21.15.66:0/2278241795 conn(0x55f9b8f5bba0 msgr2=0x55f9b8f622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 shutdown_connections 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 1 -- 172.21.15.66:0/2278241795 wait complete. 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.714+0000 7f263585e740 20 client.17727 trim_cache size 0 max 16384 2024-04-21T09:16:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:16:21.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26180a3d20 msgr2=0x7f26180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26180a3d20 0x7f26180a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26240be830 tx=0x7f2624091620 comp rx=0 tx=0).stop 2024-04-21T09:16:21.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c19ceb0 msgr2=0x7f262c19d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c19ceb0 0x7f262c19d360 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2624005e30 tx=0x7f2624034760 comp rx=0 tx=0).stop 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 shutdown_connections 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26180a3d20 0x7f26180a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f262c19ceb0 0x7f262c19d360 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f262c107270 0x7f262c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 --2- 172.21.15.66:0/3778398668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f262c06a840 0x7f262c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 >> 172.21.15.66:0/3778398668 conn(0x7f262c077eb0 msgr2=0x7f262c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 shutdown_connections 2024-04-21T09:16:21.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:21.715+0000 7f2632c6a640 1 -- 172.21.15.66:0/3778398668 wait complete. 2024-04-21T09:16:22.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:22 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2399175778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:16:22.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:22 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2399175778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:16:22.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:22 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2399175778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:16:22.727 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:16:22.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:16:22.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:16:22.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 -- 172.21.15.66:0/1943466974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 msgr2=0x7f075010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 --2- 172.21.15.66:0/1943466974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075010bfe0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0744009900 tx=0x7f074402bd60 comp rx=0 tx=0).stop 2024-04-21T09:16:22.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 -- 172.21.15.66:0/1943466974 shutdown_connections 2024-04-21T09:16:22.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 --2- 172.21.15.66:0/1943466974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f075010c520 0x7f075010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 --2- 172.21.15.66:0/1943466974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075010bfe0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 --2- 172.21.15.66:0/1943466974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07501031e0 0x7f07501035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 -- 172.21.15.66:0/1943466974 >> 172.21.15.66:0/1943466974 conn(0x7f07500771d0 msgr2=0x7f07500775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 -- 172.21.15.66:0/1943466974 shutdown_connections 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 -- 172.21.15.66:0/1943466974 wait complete. 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.847+0000 7f07552c7640 1 Processor -- start 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f07552c7640 1 -- start start 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f07552c7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07501031e0 0x7f075019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f07552c7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f07552c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075010c520 0x7f0750198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f07552c7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07501121b0 con 0x7f0750103b00 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f07552c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0750112030 con 0x7f075010c520 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f07552c7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0750112330 con 0x7f07501031e0 2024-04-21T09:16:22.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56834/0 (socket says 172.21.15.66:56834) 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 -- 172.21.15.66:0/3860875600 learned_addr learned my addr 172.21.15.66:0/3860875600 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074f7fe640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075010c520 0x7f0750198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074effd640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07501031e0 0x7f075019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 -- 172.21.15.66:0/3860875600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07501031e0 msgr2=0x7f075019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07501031e0 0x7f075019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 -- 172.21.15.66:0/3860875600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075010c520 msgr2=0x7f0750198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075010c520 0x7f0750198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 -- 172.21.15.66:0/3860875600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0750198d10 con 0x7f0750103b00 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074effd640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07501031e0 0x7f075019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.848+0000 7f074e7fc640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075019e920 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0744005940 tx=0x7f074402c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.849+0000 7f0737fff640 1 -- 172.21.15.66:0/3860875600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744043020 con 0x7f0750103b00 2024-04-21T09:16:22.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.849+0000 7f0737fff640 1 -- 172.21.15.66:0/3860875600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074402dbb0 con 0x7f0750103b00 2024-04-21T09:16:22.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.849+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0750198fa0 con 0x7f0750103b00 2024-04-21T09:16:22.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.849+0000 7f0737fff640 1 -- 172.21.15.66:0/3860875600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744003a10 con 0x7f0750103b00 2024-04-21T09:16:22.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.849+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a52a0 con 0x7f0750103b00 2024-04-21T09:16:22.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.851+0000 7f0737fff640 1 -- 172.21.15.66:0/3860875600 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0744046020 con 0x7f0750103b00 2024-04-21T09:16:22.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.851+0000 7f0757f8b740 1 Processor -- start 2024-04-21T09:16:22.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.851+0000 7f0757f8b740 1 -- start start 2024-04-21T09:16:22.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aec8e0 0x561f65aeccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0757f8b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561f65a50f00 con 0x561f65aec8e0 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0757f8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561f659ee590 con 0x561f65af5e10 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0757f8b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561f65a4cec0 con 0x561f65aed290 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aec8e0 0x561f65aeccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0737fff640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f07380a3d20 0x7f07380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aec8e0 0x561f65aeccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56850/0 (socket says 172.21.15.66:56850) 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 -- 172.21.15.66:0/1251161189 learned_addr learned my addr 172.21.15.66:0/1251161189 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:22.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074e7fc640 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f07380a3d20 0x7f07380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074effd640 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f0737fff640 1 -- 172.21.15.66:0/3860875600 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f07440f2b50 con 0x7f0750103b00 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 -- 172.21.15.66:0/1251161189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 msgr2=0x561f65af5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 -- 172.21.15.66:0/1251161189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 msgr2=0x561f65af8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 -- 172.21.15.66:0/1251161189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6582b7e0 con 0x561f65aec8e0 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074effd640 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074e7fc640 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.852+0000 7f074f7fe640 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aec8e0 0x561f65aeccc0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0748013880 tx=0x7f0748013d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f324c47e1939886 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f07357fa640 1 -- 172.21.15.66:0/1251161189 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748023020 con 0x561f65aec8e0 2024-04-21T09:16:22.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f07357fa640 1 -- 172.21.15.66:0/1251161189 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0748009d20 con 0x561f65aec8e0 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f07357fa640 1 -- 172.21.15.66:0/1251161189 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748010a90 con 0x561f65aec8e0 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 -- 172.21.15.66:0/1251161189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aec8e0 msgr2=0x561f65aeccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aec8e0 0x561f65aeccc0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0748013880 tx=0x7f0748013d40 comp rx=0 tx=0).stop 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f074f7fe640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f07380a3d20 0x7f07380a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0748024000 tx=0x7f0748022000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 -- 172.21.15.66:0/1251161189 shutdown_connections 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 --2- 172.21.15.66:0/1251161189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aec8e0 0x561f65aeccc0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 -- 172.21.15.66:0/1251161189 >> 172.21.15.66:0/1251161189 conn(0x561f65ae6fb0 msgr2=0x561f65ae7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 -- 172.21.15.66:0/1251161189 shutdown_connections 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.853+0000 7f0757f8b740 1 -- 172.21.15.66:0/1251161189 wait complete. 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 -- start start 2024-04-21T09:16:22.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 Processor -- start 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561f65af8d80 con 0x561f65af5e10 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561f65a0a580 con 0x561f65aec8e0 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561f65af8f00 con 0x561f65aed290 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f0757f8b740 10 client.0 _ll_register_callbacks cb 0x561f65a10650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56860/0 (socket says 172.21.15.66:56860) 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 -- 172.21.15.66:0/280602728 learned_addr learned my addr 172.21.15.66:0/280602728 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074e7fc640 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 -- 172.21.15.66:0/280602728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 msgr2=0x561f65af3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:22.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 -- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 msgr2=0x561f65af2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074f7fe640 1 -- 172.21.15.66:0/280602728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6588bf70 con 0x561f65af5e10 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.854+0000 7f074effd640 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f074f7fe640 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af3860 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f07480135c0 tx=0x7f07480098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d4abecb975ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0732ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0732ff5640 1 -- 172.21.15.66:0/280602728 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748023040 con 0x561f65af5e10 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0732ff5640 1 -- 172.21.15.66:0/280602728 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f0748010910 con 0x561f65af5e10 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f65a5e920 con 0x561f65af5e10 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0732ff5640 1 -- 172.21.15.66:0/280602728 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0748038050 con 0x561f65af5e10 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0757f8b740 10 client.17739 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f65723170 con 0x561f65af5e10 2024-04-21T09:16:22.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0732ff5640 1 -- 172.21.15.66:0/280602728 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f65a5e920 con 0x561f65af5e10 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0732ff5640 1 -- 172.21.15.66:0/280602728 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f074802f070 con 0x561f65af5e10 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0757f8b740 10 client.17739 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.855+0000 7f0757f8b740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.856+0000 7f0757f8b740 20 client.17739 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.856+0000 7f0757f8b740 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b05f60 0x561f65b08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.856+0000 7f0757f8b740 4 client.17739 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.856+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f65a58790 con 0x561f65b05f60 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.856+0000 7f074effd640 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b05f60 0x561f65b08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.856+0000 7f0732ff5640 10 client.17739 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.856+0000 7f074effd640 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b05f60 0x561f65b08350 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6a4923d02a01f6 server_cookie=e027b2279ba76dfb in_seq=0 out_seq=0 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0732ff5640 1 -- 172.21.15.66:0/280602728 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f65a58790 con 0x561f65b05f60 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0732ff5640 10 client.17739 handle_command_reply: tid=0 2024-04-21T09:16:22.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 client.17739 shutdown 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 msgr2=0x561f65af3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af3860 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f07480135c0 tx=0x7f07480098b0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 shutdown_connections 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b05f60 0x561f65b08350 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af3860 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65af3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0732ff5640 0 client.17739 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 --2- 172.21.15.66:0/280602728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 >> 172.21.15.66:0/280602728 conn(0x561f65ae6fb0 msgr2=0x561f65ae8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 shutdown_connections 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.857+0000 7f0757f8b740 1 -- 172.21.15.66:0/280602728 wait complete. 2024-04-21T09:16:22.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.858+0000 7f0757f8b740 20 client.17739 trim_cache size 0 max 16384 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 Processor -- start 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 -- start start 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65b01c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b1c70 con 0x561f65af5e10 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07501af540 con 0x561f65aec8e0 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.875+0000 7f0757f8b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ad3b0 con 0x561f65aed290 2024-04-21T09:16:22.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56876/0 (socket says 172.21.15.66:56876) 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 -- 172.21.15.66:0/2454986243 learned_addr learned my addr 172.21.15.66:0/2454986243 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 -- 172.21.15.66:0/2454986243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 msgr2=0x561f65b01c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074effd640 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65b01c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 -- 172.21.15.66:0/2454986243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 msgr2=0x561f65af5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 -- 172.21.15.66:0/2454986243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f65af1330 con 0x561f65af5e10 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f074f7fe640 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af1c90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0748013d80 tx=0x7f0748005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcbd6161f7cec77 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f0734ff9640 1 -- 172.21.15.66:0/2454986243 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748023040 con 0x561f65af5e10 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f0734ff9640 1 -- 172.21.15.66:0/2454986243 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074801f850 con 0x561f65af5e10 2024-04-21T09:16:22.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.876+0000 7f0734ff9640 1 -- 172.21.15.66:0/2454986243 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748005370 con 0x561f65af5e10 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- 172.21.15.66:0/2454986243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 msgr2=0x561f65af1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af1c90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0748013d80 tx=0x7f0748005970 comp rx=0 tx=0).stop 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- 172.21.15.66:0/2454986243 shutdown_connections 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65af5e10 0x561f65af1c90 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aed290 0x561f65b01c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 --2- 172.21.15.66:0/2454986243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65aec8e0 0x561f65af5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- 172.21.15.66:0/2454986243 >> 172.21.15.66:0/2454986243 conn(0x561f65ae6fb0 msgr2=0x561f65af1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- 172.21.15.66:0/2454986243 shutdown_connections 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- 172.21.15.66:0/2454986243 wait complete. 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- start start 2024-04-21T09:16:22.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 Processor -- start 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aec8e0 0x561f65afe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aed290 0x561f65af9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ada00 con 0x561f65aed290 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07501aee70 con 0x561f65af5e10 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ae0d0 con 0x561f65aec8e0 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f0757f8b740 10 client.0 _ll_register_callbacks cb 0x7f0748013680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f074e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aed290 0x561f65af9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.877+0000 7f074e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aed290 0x561f65af9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56886/0 (socket says 172.21.15.66:56886) 2024-04-21T09:16:22.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f074e7fc640 1 -- 172.21.15.66:0/206775623 learned_addr learned my addr 172.21.15.66:0/206775623 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f074e7fc640 1 -- 172.21.15.66:0/206775623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aec8e0 msgr2=0x561f65afe5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f074e7fc640 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aec8e0 0x561f65afe5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f074e7fc640 1 -- 172.21.15.66:0/206775623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 msgr2=0x561f65af9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f074e7fc640 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f074e7fc640 1 -- 172.21.15.66:0/206775623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f65afa6c0 con 0x561f65aed290 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f074e7fc640 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aed290 0x561f65af9930 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f074402dfd0 tx=0x7f074408e110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f5a5044c7503c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0732ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0732ff5640 1 -- 172.21.15.66:0/206775623 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440be190 con 0x561f65aed290 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0732ff5640 1 -- 172.21.15.66:0/206775623 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f07440baea0 con 0x561f65aed290 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074801f850 con 0x561f65aed290 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0732ff5640 1 -- 172.21.15.66:0/206775623 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0744089340 con 0x561f65aed290 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0757f8b740 10 client.17745 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f65b6c5a0 con 0x561f65aed290 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0732ff5640 1 -- 172.21.15.66:0/206775623 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074801f850 con 0x561f65aed290 2024-04-21T09:16:22.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.878+0000 7f0732ff5640 1 -- 172.21.15.66:0/206775623 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f07440bea90 con 0x561f65aed290 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f0757f8b740 10 client.17745 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f0757f8b740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f0757f8b740 20 client.17745 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f0757f8b740 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b6cc80 0x561f65b6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f0757f8b740 4 client.17745 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561f659d4c50 con 0x561f65b6cc80 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f074effd640 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b6cc80 0x561f65b6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f0732ff5640 10 client.17745 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:22.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.879+0000 7f074effd640 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b6cc80 0x561f65b6f0a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800425ed209d8c5d server_cookie=94e6b135975ffe3e in_seq=0 out_seq=0 2024-04-21T09:16:22.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0732ff5640 1 -- 172.21.15.66:0/206775623 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561f659d4c50 con 0x561f65b6cc80 2024-04-21T09:16:22.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0732ff5640 10 client.17745 handle_command_reply: tid=0 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 client.17745 shutdown 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aed290 msgr2=0x561f65af9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aed290 0x561f65af9930 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f074402dfd0 tx=0x7f074408e110 comp rx=0 tx=0).stop 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 shutdown_connections 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x561f65b6cc80 0x561f65b6f0a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f65af5e10 0x561f65af9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f65aed290 0x561f65af9930 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.880+0000 7f0757f8b740 1 --2- 172.21.15.66:0/206775623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f65aec8e0 0x561f65afe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 >> 172.21.15.66:0/206775623 conn(0x561f65ae6fb0 msgr2=0x561f65af3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f0732ff5640 0 client.17745 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 shutdown_connections 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f0757f8b740 1 -- 172.21.15.66:0/206775623 wait complete. 2024-04-21T09:16:22.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f0757f8b740 20 client.17745 trim_cache size 0 max 16384 2024-04-21T09:16:22.888 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:16:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:16:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "68c71d0a-961a-4a50-8322-9359f0c69e37", 2024-04-21T09:16:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:16:22.888 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f07380a3d20 msgr2=0x7f07380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f07380a3d20 0x7f07380a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0748024000 tx=0x7f0748022000 comp rx=0 tx=0).stop 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 msgr2=0x7f075019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075019e920 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0744005940 tx=0x7f074402c720 comp rx=0 tx=0).stop 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 shutdown_connections 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f07380a3d20 0x7f07380a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075010c520 0x7f0750198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0750103b00 0x7f075019e920 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 --2- 172.21.15.66:0/3860875600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07501031e0 0x7f075019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 >> 172.21.15.66:0/3860875600 conn(0x7f07500771d0 msgr2=0x7f07500fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 shutdown_connections 2024-04-21T09:16:22.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.881+0000 7f07552c7640 1 -- 172.21.15.66:0/3860875600 wait complete. 2024-04-21T09:16:22.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:16:22.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 -- 172.21.15.66:0/1700337611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a801072c0 msgr2=0x7f8a8010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 --2- 172.21.15.66:0/1700337611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a801072c0 0x7f8a8010ee70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009ca0 tx=0x7f8a7802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 -- 172.21.15.66:0/1700337611 shutdown_connections 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 --2- 172.21.15.66:0/1700337611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a801072c0 0x7f8a8010ee70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 --2- 172.21.15.66:0/1700337611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8006b2d0 0x7f8a80106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 --2- 172.21.15.66:0/1700337611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a8006a920 0x7f8a8006ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 -- 172.21.15.66:0/1700337611 >> 172.21.15.66:0/1700337611 conn(0x7f8a800771d0 msgr2=0x7f8a800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 -- 172.21.15.66:0/1700337611 shutdown_connections 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.996+0000 7f8a878c1640 1 -- 172.21.15.66:0/1700337611 wait complete. 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 Processor -- start 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 -- start start 2024-04-21T09:16:22.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8006a920 0x7f8a80198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a801072c0 0x7f8a80198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8019cfc0 0x7f8a8019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80112220 con 0x7f8a8019cfc0 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801120a0 con 0x7f8a8006a920 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a878c1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801123a0 con 0x7f8a801072c0 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a85e5e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8019cfc0 0x7f8a8019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a85e5e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8019cfc0 0x7f8a8019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56906/0 (socket says 172.21.15.66:56906) 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.997+0000 7f8a85e5e640 1 -- 172.21.15.66:0/3186595605 learned_addr learned my addr 172.21.15.66:0/3186595605 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a8565d640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8006a920 0x7f8a80198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a85e5e640 1 -- 172.21.15.66:0/3186595605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a801072c0 msgr2=0x7f8a80198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a85e5e640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a801072c0 0x7f8a80198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a85e5e640 1 -- 172.21.15.66:0/3186595605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8006a920 msgr2=0x7f8a80198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a85e5e640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8006a920 0x7f8a80198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a85e5e640 1 -- 172.21.15.66:0/3186595605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8019dab0 con 0x7f8a8019cfc0 2024-04-21T09:16:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a85e5e640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8019cfc0 0x7f8a8019d470 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8a78005e30 tx=0x7f8a78034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a767fc640 1 -- 172.21.15.66:0/3186595605 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78039040 con 0x7f8a8019cfc0 2024-04-21T09:16:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a767fc640 1 -- 172.21.15.66:0/3186595605 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78044070 con 0x7f8a8019cfc0 2024-04-21T09:16:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8019dd40 con 0x7f8a8019cfc0 2024-04-21T09:16:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a767fc640 1 -- 172.21.15.66:0/3186595605 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78003e60 con 0x7f8a8019cfc0 2024-04-21T09:16:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:22.998+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801a5560 con 0x7f8a8019cfc0 2024-04-21T09:16:23.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.000+0000 7f8a767fc640 1 -- 172.21.15.66:0/3186595605 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8a780324c0 con 0x7f8a8019cfc0 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 Processor -- start 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 -- start start 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459b9c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c459b011e0 con 0x55c459b9c580 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a767fc640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a6c0a3da0 0x7f8a6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c459a9e390 con 0x55c459b9cea0 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8a593740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c459afd1a0 con 0x55c459ba5a80 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a84e5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459b9c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a85e5e640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a6c0a3da0 0x7f8a6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a84e5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459b9c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56912/0 (socket says 172.21.15.66:56912) 2024-04-21T09:16:23.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a84e5c640 1 -- 172.21.15.66:0/4239458030 learned_addr learned my addr 172.21.15.66:0/4239458030 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a8565d640 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a767fc640 1 -- 172.21.15.66:0/3186595605 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8a780f2e70 con 0x7f8a8019cfc0 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.001+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a84e5c640 1 -- 172.21.15.66:0/4239458030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 msgr2=0x55c459ba7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a84e5c640 1 -- 172.21.15.66:0/4239458030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 msgr2=0x55c459ba5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a84e5c640 1 -- 172.21.15.66:0/4239458030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4598dd7d0 con 0x55c459b9c580 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a8565d640 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a85e5e640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a6c0a3da0 0x7f8a6c0a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009dc0 tx=0x7f8a780095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459b9c960 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000bfa0 tx=0x7f8a7000b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b402432e310afcc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a6b7fe640 1 -- 172.21.15.66:0/4239458030 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000fe60 con 0x55c459b9c580 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a6b7fe640 1 -- 172.21.15.66:0/4239458030 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a700054d0 con 0x55c459b9c580 2024-04-21T09:16:23.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.002+0000 7f8a6b7fe640 1 -- 172.21.15.66:0/4239458030 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700123f0 con 0x55c459b9c580 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 -- 172.21.15.66:0/4239458030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 msgr2=0x55c459b9c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459b9c960 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000bfa0 tx=0x7f8a7000b470 comp rx=0 tx=0).stop 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 -- 172.21.15.66:0/4239458030 shutdown_connections 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 --2- 172.21.15.66:0/4239458030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459b9c960 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 -- 172.21.15.66:0/4239458030 >> 172.21.15.66:0/4239458030 conn(0x55c459b96cb0 msgr2=0x55c459b97090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 -- 172.21.15.66:0/4239458030 shutdown_connections 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 -- 172.21.15.66:0/4239458030 wait complete. 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 -- start start 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.003+0000 7f8a8a593740 1 Processor -- start 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459b9c580 0x55c459ba2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459ba5a80 0x55c459ba3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c459ba8870 con 0x55c459ba5a80 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c459aba2f0 con 0x55c459b9cea0 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c459ba89f0 con 0x55c459b9c580 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 10 client.0 _ll_register_callbacks cb 0x55c459ac7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459ba5a80 0x55c459ba3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459ba5a80 0x55c459ba3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56918/0 (socket says 172.21.15.66:56918) 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 -- 172.21.15.66:0/1386476152 learned_addr learned my addr 172.21.15.66:0/1386476152 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8565d640 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 -- 172.21.15.66:0/1386476152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459b9c580 msgr2=0x55c459ba2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459b9c580 0x55c459ba2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 -- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 msgr2=0x55c459ba2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 -- 172.21.15.66:0/1386476152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4598133f0 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459ba5a80 0x55c459ba3520 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004ce0 tx=0x7f8a70004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf96a2e916ebb7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a68ff9640 1 -- 172.21.15.66:0/1386476152 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70024020 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a68ff9640 1 -- 172.21.15.66:0/1386476152 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8a700049b0 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.004+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a700054d0 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a68ff9640 1 -- 172.21.15.66:0/1386476152 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a7002f050 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 10 client.17757 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4597d30f0 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a68ff9640 1 -- 172.21.15.66:0/1386476152 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a700054d0 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a68ff9640 1 -- 172.21.15.66:0/1386476152 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a7001f040 con 0x55c459ba5a80 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 10 client.17757 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:23.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 20 client.17757 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459bb5a80 0x55c459bb7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 4 client.17757 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c459b0ec00 con 0x55c459bb5a80 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.005+0000 7f8a8565d640 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459bb5a80 0x55c459bb7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a68ff9640 10 client.17757 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8565d640 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459bb5a80 0x55c459bb7e70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab86521750eb36d server_cookie=2c116d93a81cd7a3 in_seq=0 out_seq=0 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a68ff9640 1 -- 172.21.15.66:0/1386476152 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c459b0ec00 con 0x55c459bb5a80 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a68ff9640 10 client.17757 handle_command_reply: tid=0 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 client.17757 shutdown 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459ba5a80 msgr2=0x55c459ba3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459ba5a80 0x55c459ba3520 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004ce0 tx=0x7f8a70004f90 comp rx=0 tx=0).stop 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 shutdown_connections 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459bb5a80 0x55c459bb7e70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459ba5a80 0x55c459ba3520 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459ba2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a68ff9640 0 client.17757 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1386476152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459b9c580 0x55c459ba2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.006+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 >> 172.21.15.66:0/1386476152 conn(0x55c459b96cb0 msgr2=0x55c459b98a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.007+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 shutdown_connections 2024-04-21T09:16:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.007+0000 7f8a8a593740 1 -- 172.21.15.66:0/1386476152 wait complete. 2024-04-21T09:16:23.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.007+0000 7f8a8a593740 20 client.17757 trim_cache size 0 max 16384 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 Processor -- start 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 -- start start 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9c580 0x55c459ba51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9cea0 0x55c459bb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801b1b80 con 0x55c459b9cea0 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801af450 con 0x55c459b9c580 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8a593740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ad2c0 con 0x55c459ba5a80 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8565d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9cea0 0x55c459bb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8565d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9cea0 0x55c459bb1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56930/0 (socket says 172.21.15.66:56930) 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.024+0000 7f8a8565d640 1 -- 172.21.15.66:0/1675366883 learned_addr learned my addr 172.21.15.66:0/1675366883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9c580 0x55c459ba51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a8565d640 1 -- 172.21.15.66:0/1675366883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 msgr2=0x55c459ba1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a8565d640 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a8565d640 1 -- 172.21.15.66:0/1675366883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9c580 msgr2=0x55c459ba51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a8565d640 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9c580 0x55c459ba51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a8565d640 1 -- 172.21.15.66:0/1675366883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c459ba7e70 con 0x55c459b9cea0 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a8565d640 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9cea0 0x55c459bb1760 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00b990 tx=0x7f8a7c00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fdb061b699eb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a6affd640 1 -- 172.21.15.66:0/1675366883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00be90 con 0x55c459b9cea0 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a6affd640 1 -- 172.21.15.66:0/1675366883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a7c015d80 con 0x55c459b9cea0 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.025+0000 7f8a6affd640 1 -- 172.21.15.66:0/1675366883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c0133a0 con 0x55c459b9cea0 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 -- 172.21.15.66:0/1675366883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9cea0 msgr2=0x55c459bb1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9cea0 0x55c459bb1760 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00b990 tx=0x7f8a7c00bbe0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 -- 172.21.15.66:0/1675366883 shutdown_connections 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9cea0 0x55c459bb1760 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 --2- 172.21.15.66:0/1675366883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9c580 0x55c459ba51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 -- 172.21.15.66:0/1675366883 >> 172.21.15.66:0/1675366883 conn(0x55c459b96cb0 msgr2=0x55c459b954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 -- 172.21.15.66:0/1675366883 shutdown_connections 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 -- 172.21.15.66:0/1675366883 wait complete. 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 -- start start 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.026+0000 7f8a8a593740 1 Processor -- start 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459badca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459bae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ad910 con 0x55c459b9c580 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801aed80 con 0x55c459b9cea0 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801adfe0 con 0x55c459ba5a80 2024-04-21T09:16:23.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 10 client.0 _ll_register_callbacks cb 0x7f8a7c00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459badca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459badca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56946/0 (socket says 172.21.15.66:56946) 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 -- 172.21.15.66:0/3393184293 learned_addr learned my addr 172.21.15.66:0/3393184293 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8565d640 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459bae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 -- 172.21.15.66:0/3393184293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 msgr2=0x55c459ba9220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba9220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 -- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 msgr2=0x55c459bae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459bae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 -- 172.21.15.66:0/3393184293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c459ba9a20 con 0x55c459b9c580 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8565d640 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459bae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a84e5c640 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459badca0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000ff50 tx=0x7f8a700058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302af50be7d2382 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:23.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a68ff9640 1 -- 172.21.15.66:0/3393184293 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70024070 con 0x55c459b9c580 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a7c015d80 con 0x55c459b9c580 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a68ff9640 1 -- 172.21.15.66:0/3393184293 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8a7000ba40 con 0x55c459b9c580 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a68ff9640 1 -- 172.21.15.66:0/3393184293 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a700123f0 con 0x55c459b9c580 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 10 client.17763 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.027+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c459c1b980 con 0x55c459b9c580 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a68ff9640 1 -- 172.21.15.66:0/3393184293 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a7c015d80 con 0x55c459b9c580 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a68ff9640 1 -- 172.21.15.66:0/3393184293 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8a70012be0 con 0x55c459b9c580 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8a593740 10 client.17763 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8a593740 10 client.17763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8a593740 20 client.17763 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8a593740 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459c1bf80 0x55c459c1e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8a593740 4 client.17763 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c459b08a70 con 0x55c459c1bf80 2024-04-21T09:16:23.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8565d640 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459c1bf80 0x55c459c1e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a8565d640 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459c1bf80 0x55c459c1e3a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2bde88958e75f60 server_cookie=76d97779a30177c6 in_seq=0 out_seq=0 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.028+0000 7f8a68ff9640 10 client.17763 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a68ff9640 1 -- 172.21.15.66:0/3393184293 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c459b08a70 con 0x55c459c1bf80 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a68ff9640 10 client.17763 handle_command_reply: tid=0 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 client.17763 shutdown 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 msgr2=0x55c459badca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459badca0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000ff50 tx=0x7f8a700058c0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 shutdown_connections 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c459c1bf80 0x55c459c1e3a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c459ba5a80 0x55c459ba9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c459b9cea0 0x55c459bae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a68ff9640 0 client.17763 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 --2- 172.21.15.66:0/3393184293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c459b9c580 0x55c459badca0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 >> 172.21.15.66:0/3393184293 conn(0x55c459b96cb0 msgr2=0x55c459ba32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 shutdown_connections 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 1 -- 172.21.15.66:0/3393184293 wait complete. 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a8a593740 20 client.17763 trim_cache size 0 max 16384 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "68c71d0a-961a-4a50-8322-9359f0c69e37": { 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "68c71d0a-961a-4a50-8322-9359f0c69e37", 2024-04-21T09:16:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:16:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:16:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:16:23.038 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:16:23.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a6c0a3da0 msgr2=0x7f8a6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a6c0a3da0 0x7f8a6c0a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009dc0 tx=0x7f8a780095c0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8019cfc0 msgr2=0x7f8a8019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:23.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8019cfc0 0x7f8a8019d470 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8a78005e30 tx=0x7f8a78034760 comp rx=0 tx=0).stop 2024-04-21T09:16:23.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 shutdown_connections 2024-04-21T09:16:23.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a6c0a3da0 0x7f8a6c0a6260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8019cfc0 0x7f8a8019d470 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a801072c0 0x7f8a80198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 --2- 172.21.15.66:0/3186595605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8006a920 0x7f8a80198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:23.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.029+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 >> 172.21.15.66:0/3186595605 conn(0x7f8a800771d0 msgr2=0x7f8a800fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:23.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.030+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 shutdown_connections 2024-04-21T09:16:23.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:23.030+0000 7f8a878c1640 1 -- 172.21.15.66:0/3186595605 wait complete. 2024-04-21T09:16:23.042 INFO:tasks.cephfs.filesystem:scrub status for tag:68c71d0a-961a-4a50-8322-9359f0c69e37 - {'path': '/', 'tag': '68c71d0a-961a-4a50-8322-9359f0c69e37', 'options': 'recursive,force'} 2024-04-21T09:16:23.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:23 smithi066 ceph-mon[27953]: pgmap v252: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:16:23.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:23 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:16:23.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:23 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:23.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:23 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:16:23.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:23 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:23.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:23 smithi186 ceph-mon[31505]: pgmap v252: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:16:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:23 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:16:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:23 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:23 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:16:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:23 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:23 smithi083 ceph-mon[31413]: pgmap v252: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:16:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:23 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:16:23.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:23 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:23.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:23 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:16:23.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:23 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:24.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:24 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:16:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:24 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:16:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:24 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:16:24.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:24 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:16:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:24 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:16:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:24 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:16:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:24 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:16:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:24 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:16:24.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:24 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:16:24.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:24 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:16:24.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:24 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:16:24.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:24 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:16:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:25 smithi186 ceph-mon[31505]: pgmap v253: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:16:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:25 smithi083 ceph-mon[31413]: pgmap v253: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:16:25.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:25 smithi066 ceph-mon[27953]: pgmap v253: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T09:16:26.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:26 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:16:26.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:26 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:16:26.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:26 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:16:27.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:27 smithi186 ceph-mon[31505]: pgmap v254: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:27.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:27 smithi083 ceph-mon[31413]: pgmap v254: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:27.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:27 smithi066 ceph-mon[27953]: pgmap v254: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:29.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:29 smithi186 ceph-mon[31505]: pgmap v255: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:16:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:29 smithi083 ceph-mon[31413]: pgmap v255: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:16:29.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:29 smithi066 ceph-mon[27953]: pgmap v255: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:16:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:31 smithi186 ceph-mon[31505]: pgmap v256: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:31.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:31 smithi083 ceph-mon[31413]: pgmap v256: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:31.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:31 smithi066 ceph-mon[27953]: pgmap v256: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:33.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:33 smithi186 ceph-mon[31505]: pgmap v257: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:33.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:33 smithi083 ceph-mon[31413]: pgmap v257: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:33.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:33 smithi066 ceph-mon[27953]: pgmap v257: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:16:35.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:35 smithi186 ceph-mon[31505]: pgmap v258: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-04-21T09:16:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:35 smithi083 ceph-mon[31413]: pgmap v258: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-04-21T09:16:35.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:35 smithi066 ceph-mon[27953]: pgmap v258: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-04-21T09:16:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:37 smithi186 ceph-mon[31505]: pgmap v259: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:37.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:37 smithi083 ceph-mon[31413]: pgmap v259: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:37.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:37 smithi066 ceph-mon[27953]: pgmap v259: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:39 smithi186 ceph-mon[31505]: pgmap v260: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:39 smithi083 ceph-mon[31413]: pgmap v260: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:39.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:39 smithi066 ceph-mon[27953]: pgmap v260: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:41.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:41 smithi186 ceph-mon[31505]: pgmap v261: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:41.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:41 smithi083 ceph-mon[31413]: pgmap v261: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:41.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:41 smithi066 ceph-mon[27953]: pgmap v261: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:43.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:43 smithi186 ceph-mon[31505]: pgmap v262: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:43.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:43 smithi083 ceph-mon[31413]: pgmap v262: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:43.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:43 smithi066 ceph-mon[27953]: pgmap v262: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T09:16:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:45 smithi186 ceph-mon[31505]: pgmap v263: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:45 smithi083 ceph-mon[31413]: pgmap v263: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:45.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:45 smithi066 ceph-mon[27953]: pgmap v263: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:47.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:47 smithi186 ceph-mon[31505]: pgmap v264: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:47 smithi083 ceph-mon[31413]: pgmap v264: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:47.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:47 smithi066 ceph-mon[27953]: pgmap v264: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:49 smithi186 ceph-mon[31505]: pgmap v265: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:49.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:49 smithi083 ceph-mon[31413]: pgmap v265: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:49 smithi066 ceph-mon[27953]: pgmap v265: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:51 smithi186 ceph-mon[31505]: pgmap v266: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:51.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:51 smithi083 ceph-mon[31413]: pgmap v266: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:51.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:51 smithi066 ceph-mon[27953]: pgmap v266: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:53.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:16:53.043 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:16:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 -- 172.21.15.66:0/1118883948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4101690 msgr2=0x7fccc410e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 --2- 172.21.15.66:0/1118883948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4101690 0x7fccc410e3e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009900 tx=0x7fccb802bc30 comp rx=0 tx=0).stop 2024-04-21T09:16:53.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 -- 172.21.15.66:0/1118883948 shutdown_connections 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 --2- 172.21.15.66:0/1118883948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccc410e920 0x7fccc4110d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 --2- 172.21.15.66:0/1118883948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4101690 0x7fccc410e3e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 --2- 172.21.15.66:0/1118883948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccc4100d70 0x7fccc4101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 -- 172.21.15.66:0/1118883948 >> 172.21.15.66:0/1118883948 conn(0x7fccc40771d0 msgr2=0x7fccc40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 -- 172.21.15.66:0/1118883948 shutdown_connections 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.161+0000 7fcccbd75640 1 -- 172.21.15.66:0/1118883948 wait complete. 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 Processor -- start 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 -- start start 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccc4100d70 0x7fccc41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccc4101690 0x7fccc41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc410e920 0x7fccc419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4114450 con 0x7fccc410e920 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41142d0 con 0x7fccc4101690 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fcccbd75640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41145d0 con 0x7fccc4100d70 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fccca312640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc410e920 0x7fccc419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fccca312640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc410e920 0x7fccc419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36706/0 (socket says 172.21.15.66:36706) 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.162+0000 7fccca312640 1 -- 172.21.15.66:0/3255972100 learned_addr learned my addr 172.21.15.66:0/3255972100 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccc9310640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccc4101690 0x7fccc41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccc9b11640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccc4100d70 0x7fccc41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccca312640 1 -- 172.21.15.66:0/3255972100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccc4100d70 msgr2=0x7fccc41a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccca312640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccc4100d70 0x7fccc41a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccca312640 1 -- 172.21.15.66:0/3255972100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccc4101690 msgr2=0x7fccc41a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccca312640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccc4101690 0x7fccc41a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccca312640 1 -- 172.21.15.66:0/3255972100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc419af80 con 0x7fccc410e920 2024-04-21T09:16:53.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccca312640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc410e920 0x7fccc419a7b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e9d0 tx=0x7fccbc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccb6ffd640 1 -- 172.21.15.66:0/3255972100 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00cd80 con 0x7fccc410e920 2024-04-21T09:16:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccb6ffd640 1 -- 172.21.15.66:0/3255972100 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccbc005020 con 0x7fccc410e920 2024-04-21T09:16:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc419b270 con 0x7fccc410e920 2024-04-21T09:16:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fccb6ffd640 1 -- 172.21.15.66:0/3255972100 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0116d0 con 0x7fccc410e920 2024-04-21T09:16:53.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.163+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc41a7490 con 0x7fccc410e920 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.165+0000 7fccb6ffd640 1 -- 172.21.15.66:0/3255972100 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fccbc0051d0 con 0x7fccc410e920 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 Processor -- start 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 -- start start 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04207960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04210540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04212e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dd0416c1e0 con 0x55dd04210a80 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd04109390 con 0x55dd04207ea0 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccce967740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dd041681a0 con 0x55dd04207580 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.166+0000 7fccb6ffd640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fccb00a3d20 0x7fccb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04212e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccca312640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fccb00a3d20 0x7fccb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04212e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36708/0 (socket says 172.21.15.66:36708) 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 -- 172.21.15.66:0/1500120036 learned_addr learned my addr 172.21.15.66:0/1500120036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9b11640 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04210540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04207960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccb6ffd640 1 -- 172.21.15.66:0/3255972100 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fccbc0d0ed0 con 0x7fccc410e920 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 -- 172.21.15.66:0/1500120036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 msgr2=0x55dd04207960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04207960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 -- 172.21.15.66:0/1500120036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 msgr2=0x55dd04210540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04210540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 -- 172.21.15.66:0/1500120036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd03f487d0 con 0x55dd04210a80 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9b11640 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04210540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:16:53.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccca312640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fccb00a3d20 0x7fccb00a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0ce300 tx=0x7fccbc0bd040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccc9310640 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04212e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005940 tx=0x7fccb80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115145ef98481925 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccaffff640 1 -- 172.21.15.66:0/1500120036 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8049070 con 0x55dd04210a80 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.167+0000 7fccaffff640 1 -- 172.21.15.66:0/1500120036 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb8005500 con 0x55dd04210a80 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccaffff640 1 -- 172.21.15.66:0/1500120036 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8004a50 con 0x55dd04210a80 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 -- 172.21.15.66:0/1500120036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 msgr2=0x55dd04212e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04212e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005940 tx=0x7fccb80058b0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 -- 172.21.15.66:0/1500120036 shutdown_connections 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04212e70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04210540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 --2- 172.21.15.66:0/1500120036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04207960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 -- 172.21.15.66:0/1500120036 >> 172.21.15.66:0/1500120036 conn(0x55dd04201cb0 msgr2=0x55dd04202090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 -- 172.21.15.66:0/1500120036 shutdown_connections 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.168+0000 7fccce967740 1 -- 172.21.15.66:0/1500120036 wait complete. 2024-04-21T09:16:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 -- start start 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 Processor -- start 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04207580 0x55dd0420db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207ea0 0x55dd0420e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04210a80 0x55dd0420e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dd04213810 con 0x55dd04207580 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd041252f0 con 0x55dd04210a80 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dd04213990 con 0x55dd04207ea0 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccce967740 10 client.0 _ll_register_callbacks cb 0x55dd04132580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04207580 0x55dd0420db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04207580 0x55dd0420db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36712/0 (socket says 172.21.15.66:36712) 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.169+0000 7fccc9310640 1 -- 172.21.15.66:0/1715678407 learned_addr learned my addr 172.21.15.66:0/1715678407 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9b11640 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207ea0 0x55dd0420e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9310640 1 -- 172.21.15.66:0/1715678407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207ea0 msgr2=0x55dd0420e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9310640 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207ea0 0x55dd0420e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9310640 1 -- 172.21.15.66:0/1715678407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04210a80 msgr2=0x55dd0420e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9310640 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04210a80 0x55dd0420e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9310640 1 -- 172.21.15.66:0/1715678407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd03e7e3f0 con 0x55dd04207580 2024-04-21T09:16:53.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9b11640 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207ea0 0x55dd0420e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccc9310640 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04207580 0x55dd0420db80 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb802cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6477b8324dde83c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccad7fa640 1 -- 172.21.15.66:0/1715678407 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8049040 con 0x55dd04207580 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccad7fa640 1 -- 172.21.15.66:0/1715678407 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fccb8003a00 con 0x55dd04207580 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb8005500 con 0x55dd04207580 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccad7fa640 1 -- 172.21.15.66:0/1715678407 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fccb805d050 con 0x55dd04207580 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccce967740 10 client.17775 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd03e3e0f0 con 0x55dd04207580 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.170+0000 7fccad7fa640 1 -- 172.21.15.66:0/1715678407 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb8005500 con 0x55dd04207580 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccad7fa640 1 -- 172.21.15.66:0/1715678407 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fccb8040040 con 0x55dd04207580 2024-04-21T09:16:53.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccce967740 10 client.17775 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccce967740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccce967740 20 client.17775 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccce967740 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04220980 0x55dd04222d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccce967740 4 client.17775 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd04179c00 con 0x55dd04220980 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccc9b11640 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04220980 0x55dd04222d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccad7fa640 10 client.17775 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.171+0000 7fccc9b11640 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04220980 0x55dd04222d70 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf979dbc9798f70 server_cookie=5e0f3d1d7caf2a1b in_seq=0 out_seq=0 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccad7fa640 1 -- 172.21.15.66:0/1715678407 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd04179c00 con 0x55dd04220980 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccad7fa640 10 client.17775 handle_command_reply: tid=0 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 client.17775 shutdown 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04207580 msgr2=0x55dd0420db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04207580 0x55dd0420db80 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb802cc10 comp rx=0 tx=0).stop 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 shutdown_connections 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04220980 0x55dd04222d70 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04210a80 0x55dd0420e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207ea0 0x55dd0420e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 --2- 172.21.15.66:0/1715678407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04207580 0x55dd0420db80 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 >> 172.21.15.66:0/1715678407 conn(0x55dd04201cb0 msgr2=0x55dd04203a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccad7fa640 0 client.17775 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 shutdown_connections 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 1 -- 172.21.15.66:0/1715678407 wait complete. 2024-04-21T09:16:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.172+0000 7fccce967740 20 client.17775 trim_cache size 0 max 16384 2024-04-21T09:16:53.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 Processor -- start 2024-04-21T09:16:53.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 -- start start 2024-04-21T09:16:53.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd042102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04208280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd0420ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b3e70 con 0x55dd04210a80 2024-04-21T09:16:53.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b1740 con 0x55dd04207ea0 2024-04-21T09:16:53.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.190+0000 7fccce967740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41af5b0 con 0x55dd04207580 2024-04-21T09:16:53.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd0420ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd0420ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36726/0 (socket says 172.21.15.66:36726) 2024-04-21T09:16:53.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 -- 172.21.15.66:0/2844689305 learned_addr learned my addr 172.21.15.66:0/2844689305 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9b11640 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04208280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd042102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 -- 172.21.15.66:0/2844689305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 msgr2=0x55dd042102b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd042102b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 -- 172.21.15.66:0/2844689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 msgr2=0x55dd04208280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04208280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 -- 172.21.15.66:0/2844689305 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0420c0d0 con 0x55dd04210a80 2024-04-21T09:16:53.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccc9310640 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd0420ade0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fccb8043c50 tx=0x7fccb80095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159f2fa435654ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccaf7fe640 1 -- 172.21.15.66:0/2844689305 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb802c5d0 con 0x55dd04210a80 2024-04-21T09:16:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccaf7fe640 1 -- 172.21.15.66:0/2844689305 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb8054070 con 0x55dd04210a80 2024-04-21T09:16:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.191+0000 7fccaf7fe640 1 -- 172.21.15.66:0/2844689305 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8003d00 con 0x55dd04210a80 2024-04-21T09:16:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 -- 172.21.15.66:0/2844689305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 msgr2=0x55dd0420ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd0420ade0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fccb8043c50 tx=0x7fccb80095f0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 -- 172.21.15.66:0/2844689305 shutdown_connections 2024-04-21T09:16:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd0420ade0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04208280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 --2- 172.21.15.66:0/2844689305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd042102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 -- 172.21.15.66:0/2844689305 >> 172.21.15.66:0/2844689305 conn(0x55dd04201cb0 msgr2=0x55dd0420c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 -- 172.21.15.66:0/2844689305 shutdown_connections 2024-04-21T09:16:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 -- 172.21.15.66:0/2844689305 wait complete. 2024-04-21T09:16:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 -- start start 2024-04-21T09:16:53.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.192+0000 7fccce967740 1 Processor -- start 2024-04-21T09:16:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04225c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04223000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04223540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41afc00 con 0x55dd04210a80 2024-04-21T09:16:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b1070 con 0x55dd04207ea0 2024-04-21T09:16:53.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b02d0 con 0x55dd04207580 2024-04-21T09:16:53.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 10 client.0 _ll_register_callbacks cb 0x7fccb8047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04223540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04223540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36734/0 (socket says 172.21.15.66:36734) 2024-04-21T09:16:53.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 -- 172.21.15.66:0/3302038721 learned_addr learned my addr 172.21.15.66:0/3302038721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04225c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9b11640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04223000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 -- 172.21.15.66:0/3302038721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 msgr2=0x55dd04225c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04225c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 -- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 msgr2=0x55dd04223000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04223000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 -- 172.21.15.66:0/3302038721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd04223c30 con 0x55dd04210a80 2024-04-21T09:16:53.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9b11640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04223000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccc9310640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04223540 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005860 tx=0x7fccb8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2189b5be29cf689f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccad7fa640 1 -- 172.21.15.66:0/3302038721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8067020 con 0x55dd04210a80 2024-04-21T09:16:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccad7fa640 1 -- 172.21.15.66:0/3302038721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fccb8003c80 con 0x55dd04210a80 2024-04-21T09:16:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.193+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb8054070 con 0x55dd04210a80 2024-04-21T09:16:53.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccad7fa640 1 -- 172.21.15.66:0/3302038721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fccb806f050 con 0x55dd04210a80 2024-04-21T09:16:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 10 client.17781 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd04286a70 con 0x55dd04210a80 2024-04-21T09:16:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccad7fa640 1 -- 172.21.15.66:0/3302038721 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb8054070 con 0x55dd04210a80 2024-04-21T09:16:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccad7fa640 1 -- 172.21.15.66:0/3302038721 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fccb8034910 con 0x55dd04210a80 2024-04-21T09:16:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 10 client.17781 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:53.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 20 client.17781 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04286e30 0x55dd04289250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 4 client.17781 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:16:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd04173a70 con 0x55dd04286e30 2024-04-21T09:16:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.194+0000 7fccc9b11640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04286e30 0x55dd04289250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccad7fa640 10 client.17781 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccc9b11640 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04286e30 0x55dd04289250 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee56737f2243ec7 server_cookie=8092e054d342655a in_seq=0 out_seq=0 2024-04-21T09:16:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccad7fa640 1 -- 172.21.15.66:0/3302038721 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd04173a70 con 0x55dd04286e30 2024-04-21T09:16:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccad7fa640 10 client.17781 handle_command_reply: tid=0 2024-04-21T09:16:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 client.17781 shutdown 2024-04-21T09:16:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 msgr2=0x55dd04223540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04223540 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005860 tx=0x7fccb8009780 comp rx=0 tx=0).stop 2024-04-21T09:16:53.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 shutdown_connections 2024-04-21T09:16:53.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55dd04286e30 0x55dd04289250 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd04210a80 0x55dd04223540 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd04207ea0 0x55dd04223000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccad7fa640 0 client.17781 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:53.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 --2- 172.21.15.66:0/3302038721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd04207580 0x55dd04225c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 >> 172.21.15.66:0/3302038721 conn(0x55dd04201cb0 msgr2=0x55dd0421bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 shutdown_connections 2024-04-21T09:16:53.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 1 -- 172.21.15.66:0/3302038721 wait complete. 2024-04-21T09:16:53.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.195+0000 7fccce967740 20 client.17781 trim_cache size 0 max 16384 2024-04-21T09:16:53.216 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:16:53.216 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:16:53.216 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:16:53.216 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:16:53.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fccb00a3d20 msgr2=0x7fccb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fccb00a3d20 0x7fccb00a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0ce300 tx=0x7fccbc0bd040 comp rx=0 tx=0).stop 2024-04-21T09:16:53.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc410e920 msgr2=0x7fccc419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc410e920 0x7fccc419a7b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e9d0 tx=0x7fccbc00ee90 comp rx=0 tx=0).stop 2024-04-21T09:16:53.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 shutdown_connections 2024-04-21T09:16:53.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fccb00a3d20 0x7fccb00a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc410e920 0x7fccc419a7b0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccc4101690 0x7fccc41a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 --2- 172.21.15.66:0/3255972100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccc4100d70 0x7fccc41a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 >> 172.21.15.66:0/3255972100 conn(0x7fccc40771d0 msgr2=0x7fccc410ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 shutdown_connections 2024-04-21T09:16:53.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.196+0000 7fcccbd75640 1 -- 172.21.15.66:0/3255972100 wait complete. 2024-04-21T09:16:53.220 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:16:53.221 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:16:53.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 -- 172.21.15.66:0/2224903306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b84102b50 msgr2=0x7f1b84106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2224903306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b84102b50 0x7f1b84106b90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009900 tx=0x7f1b7402bc30 comp rx=0 tx=0).stop 2024-04-21T09:16:53.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 -- 172.21.15.66:0/2224903306 shutdown_connections 2024-04-21T09:16:53.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2224903306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b84107270 0x7f1b8410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2224903306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b84102b50 0x7f1b84106b90 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2224903306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b841021a0 0x7f1b84102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 -- 172.21.15.66:0/2224903306 >> 172.21.15.66:0/2224903306 conn(0x7f1b840793a0 msgr2=0x7f1b840797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 -- 172.21.15.66:0/2224903306 shutdown_connections 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 -- 172.21.15.66:0/2224903306 wait complete. 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.324+0000 7f1b894c1640 1 Processor -- start 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b894c1640 1 -- start start 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b894c1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b841021a0 0x7f1b84074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b894c1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84107270 0x7f1b84074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b894c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b84074cd0 0x7f1b841a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b894c1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841121b0 con 0x7f1b841021a0 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b894c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b84112030 con 0x7f1b84074cd0 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b894c1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1b84112330 con 0x7f1b84107270 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b841021a0 0x7f1b84074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b841021a0 0x7f1b84074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36754/0 (socket says 172.21.15.66:36754) 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 -- 172.21.15.66:0/2348675702 learned_addr learned my addr 172.21.15.66:0/2348675702 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b837fe640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b84074cd0 0x7f1b841a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b827fc640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84107270 0x7f1b84074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 -- 172.21.15.66:0/2348675702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84107270 msgr2=0x7f1b84074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84107270 0x7f1b84074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 -- 172.21.15.66:0/2348675702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b84074cd0 msgr2=0x7f1b841a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b84074cd0 0x7f1b841a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 -- 172.21.15.66:0/2348675702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b841a7000 con 0x7f1b841021a0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b82ffd640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b841021a0 0x7f1b84074250 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c970 tx=0x7f1b7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.325+0000 7f1b6bfff640 1 -- 172.21.15.66:0/2348675702 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78007bb0 con 0x7f1b841021a0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.326+0000 7f1b6bfff640 1 -- 172.21.15.66:0/2348675702 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b78007d50 con 0x7f1b841021a0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.326+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b841a72f0 con 0x7f1b841021a0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.326+0000 7f1b6bfff640 1 -- 172.21.15.66:0/2348675702 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800f450 con 0x7f1b841021a0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.326+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b841a77c0 con 0x7f1b841021a0 2024-04-21T09:16:53.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.326+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b84069230 con 0x7f1b841021a0 2024-04-21T09:16:53.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.328+0000 7f1b6bfff640 1 -- 172.21.15.66:0/2348675702 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1b78004ee0 con 0x7f1b841021a0 2024-04-21T09:16:53.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.328+0000 7f1b6bfff640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1b6c0a3d50 0x7f1b6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.328+0000 7f1b827fc640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1b6c0a3d50 0x7f1b6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.328+0000 7f1b6bfff640 1 -- 172.21.15.66:0/2348675702 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f1b780d0cc0 con 0x7f1b841021a0 2024-04-21T09:16:53.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.329+0000 7f1b827fc640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1b6c0a3d50 0x7f1b6c0a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006010 tx=0x7f1b74004040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.332+0000 7f1b6bfff640 1 -- 172.21.15.66:0/2348675702 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b78014030 con 0x7f1b841021a0 2024-04-21T09:16:53.369 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:53 smithi066 ceph-mon[27953]: pgmap v267: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.558+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b84075780 con 0x7f1b841021a0 2024-04-21T09:16:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.559+0000 7f1b6bfff640 1 -- 172.21.15.66:0/2348675702 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f1b84075780 con 0x7f1b841021a0 2024-04-21T09:16:53.560 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:16:53.560 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:16:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:16:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1b6c0a3d50 msgr2=0x7f1b6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1b6c0a3d50 0x7f1b6c0a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006010 tx=0x7f1b74004040 comp rx=0 tx=0).stop 2024-04-21T09:16:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b841021a0 msgr2=0x7f1b84074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b841021a0 0x7f1b84074250 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c970 tx=0x7f1b7800ce30 comp rx=0 tx=0).stop 2024-04-21T09:16:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 shutdown_connections 2024-04-21T09:16:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1b6c0a3d50 0x7f1b6c0a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b84074cd0 0x7f1b841a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84107270 0x7f1b84074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 --2- 172.21.15.66:0/2348675702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b841021a0 0x7f1b84074250 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.561+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 >> 172.21.15.66:0/2348675702 conn(0x7f1b840793a0 msgr2=0x7f1b840fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.562+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 shutdown_connections 2024-04-21T09:16:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.562+0000 7f1b894c1640 1 -- 172.21.15.66:0/2348675702 wait complete. 2024-04-21T09:16:53.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:16:53.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:53 smithi186 ceph-mon[31505]: pgmap v267: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:53.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:53 smithi083 ceph-mon[31413]: pgmap v267: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 -- 172.21.15.66:0/1220299269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 msgr2=0x7f1afc06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 --2- 172.21.15.66:0/1220299269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc06ac20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009900 tx=0x7f1af802bc30 comp rx=0 tx=0).stop 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 -- 172.21.15.66:0/1220299269 shutdown_connections 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 --2- 172.21.15.66:0/1220299269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1afc107270 0x7f1afc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 --2- 172.21.15.66:0/1220299269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1afc06b1f0 0x7f1afc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 --2- 172.21.15.66:0/1220299269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc06ac20 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 -- 172.21.15.66:0/1220299269 >> 172.21.15.66:0/1220299269 conn(0x7f1afc077eb0 msgr2=0x7f1afc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.676+0000 7f1b03c81640 1 -- 172.21.15.66:0/1220299269 shutdown_connections 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 -- 172.21.15.66:0/1220299269 wait complete. 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 Processor -- start 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 -- start start 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1afc107270 0x7f1afc1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1afc101900 0x7f1afc1a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc112230 con 0x7f1afc06a840 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1120b0 con 0x7f1afc101900 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b03c81640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1123b0 con 0x7f1afc107270 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.677+0000 7f1b01a1d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36784/0 (socket says 172.21.15.66:36784) 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 -- 172.21.15.66:0/81327940 learned_addr learned my addr 172.21.15.66:0/81327940 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b0221e640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1afc101900 0x7f1afc1a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b0121c640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1afc107270 0x7f1afc1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 -- 172.21.15.66:0/81327940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1afc107270 msgr2=0x7f1afc1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1afc107270 0x7f1afc1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 -- 172.21.15.66:0/81327940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1afc101900 msgr2=0x7f1afc1a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1afc101900 0x7f1afc1a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 -- 172.21.15.66:0/81327940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc1a6fa0 con 0x7f1afc06a840 2024-04-21T09:16:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc100e80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005940 tx=0x7f1af802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1af2ffd640 1 -- 172.21.15.66:0/81327940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8043020 con 0x7f1afc06a840 2024-04-21T09:16:53.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1af2ffd640 1 -- 172.21.15.66:0/81327940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af802cc60 con 0x7f1afc06a840 2024-04-21T09:16:53.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc1a7230 con 0x7f1afc06a840 2024-04-21T09:16:53.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1af2ffd640 1 -- 172.21.15.66:0/81327940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8003b50 con 0x7f1afc06a840 2024-04-21T09:16:53.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.678+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc1a7780 con 0x7f1afc06a840 2024-04-21T09:16:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.680+0000 7f1af2ffd640 1 -- 172.21.15.66:0/81327940 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1af8046020 con 0x7f1afc06a840 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.681+0000 7f1b0687b740 1 Processor -- start 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.681+0000 7f1b0687b740 1 -- start start 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.681+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d258c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 0x55a2d2595430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2597e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0687b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a2d24f11f0 con 0x55a2d2595a00 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0687b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a2d248e4b0 con 0x55a2d258c4d0 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0687b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a2d24ed1b0 con 0x55a2d258ce80 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1af2ffd640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1ae80a3d50 0x7f1ae80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2597e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0221e640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1ae80a3d50 0x7f1ae80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2597e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36798/0 (socket says 172.21.15.66:36798) 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 -- 172.21.15.66:0/3415895751 learned_addr learned my addr 172.21.15.66:0/3415895751 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0221e640 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d258c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1af2ffd640 1 -- 172.21.15.66:0/81327940 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f1af80f2d20 con 0x7f1afc06a840 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 0x55a2d2595430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 -- 172.21.15.66:0/3415895751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 msgr2=0x55a2d2595430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 0x55a2d2595430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 -- 172.21.15.66:0/3415895751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 msgr2=0x55a2d258c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d258c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 -- 172.21.15.66:0/3415895751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d24801b0 con 0x55a2d2595a00 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0221e640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1ae80a3d50 0x7f1ae80a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009fd0 tx=0x7f1af4009ee0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1b0121c640 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2597e50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00ac70 tx=0x7f1aec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b19b22fdbb70a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.682+0000 7f1ae7fff640 1 -- 172.21.15.66:0/3415895751 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec01d070 con 0x55a2d2595a00 2024-04-21T09:16:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1ae7fff640 1 -- 172.21.15.66:0/3415895751 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aec018af0 con 0x55a2d2595a00 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1ae7fff640 1 -- 172.21.15.66:0/3415895751 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0109d0 con 0x55a2d2595a00 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 -- 172.21.15.66:0/3415895751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 msgr2=0x55a2d2597e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2597e50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00ac70 tx=0x7f1aec00f790 comp rx=0 tx=0).stop 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 -- 172.21.15.66:0/3415895751 shutdown_connections 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2597e50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 0x55a2d2595430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3415895751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d258c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 -- 172.21.15.66:0/3415895751 >> 172.21.15.66:0/3415895751 conn(0x55a2d2586ba0 msgr2=0x55a2d2586f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 -- 172.21.15.66:0/3415895751 shutdown_connections 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.683+0000 7f1b0687b740 1 -- 172.21.15.66:0/3415895751 wait complete. 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 -- start start 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 Processor -- start 2024-04-21T09:16:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258ce80 0x55a2d2592670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2592bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a2d2598970 con 0x55a2d2595a00 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a2d24aa500 con 0x55a2d258ce80 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a2d2598af0 con 0x55a2d258c4d0 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0687b740 10 client.0 _ll_register_callbacks cb 0x55a2d2478b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0221e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2592bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0221e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2592bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36808/0 (socket says 172.21.15.66:36808) 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0221e640 1 -- 172.21.15.66:0/192190680 learned_addr learned my addr 172.21.15.66:0/192190680 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b0121c640 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258ce80 0x55a2d2592670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.684+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0221e640 1 -- 172.21.15.66:0/192190680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 msgr2=0x55a2d25952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0221e640 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0221e640 1 -- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258ce80 msgr2=0x55a2d2592670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0221e640 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258ce80 0x55a2d2592670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0221e640 1 -- 172.21.15.66:0/192190680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d22033f0 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0221e640 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2592bd0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009c10 tx=0x7f1af401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d34b35e69626a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1ae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1ae57fa640 1 -- 172.21.15.66:0/192190680 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4014040 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1ae57fa640 1 -- 172.21.15.66:0/192190680 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f1af4026620 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aec018af0 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1ae57fa640 1 -- 172.21.15.66:0/192190680 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1af4031050 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0687b740 10 client.17799 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2d22cd7d0 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.685+0000 7f1ae57fa640 1 -- 172.21.15.66:0/192190680 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aec018af0 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1ae57fa640 1 -- 172.21.15.66:0/192190680 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1af4025770 con 0x55a2d2595a00 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b0687b740 10 client.17799 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:53.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b0687b740 10 client.17799 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b0687b740 20 client.17799 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b0687b740 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d25a5c20 0x55a2d25a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b0687b740 4 client.17799 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d2490930 con 0x55a2d25a5c20 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d25a5c20 0x55a2d25a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1ae57fa640 10 client.17799 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.686+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d25a5c20 0x55a2d25a8070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3904650d3afbdc30 server_cookie=19af9b0cc91ad4c2 in_seq=0 out_seq=0 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1ae57fa640 1 -- 172.21.15.66:0/192190680 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2d2490930 con 0x55a2d25a5c20 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1ae57fa640 10 client.17799 handle_command_reply: tid=0 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 client.17799 shutdown 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 msgr2=0x55a2d2592bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2592bd0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009c10 tx=0x7f1af401a000 comp rx=0 tx=0).stop 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 shutdown_connections 2024-04-21T09:16:53.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d25a5c20 0x55a2d25a8070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2592bd0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258ce80 0x55a2d2592670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1ae57fa640 0 client.17799 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 --2- 172.21.15.66:0/192190680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 >> 172.21.15.66:0/192190680 conn(0x55a2d2586ba0 msgr2=0x55a2d25888f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 shutdown_connections 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 1 -- 172.21.15.66:0/192190680 wait complete. 2024-04-21T09:16:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.687+0000 7f1b0687b740 20 client.17799 trim_cache size 0 max 16384 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 Processor -- start 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 -- start start 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d25a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 0x55a2d2585a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2594da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1b3d70 con 0x55a2d2595a00 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1b1640 con 0x55a2d258c4d0 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0687b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1af4b0 con 0x55a2d258ce80 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2594da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2594da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36816/0 (socket says 172.21.15.66:36816) 2024-04-21T09:16:53.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 -- 172.21.15.66:0/752054568 learned_addr learned my addr 172.21.15.66:0/752054568 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d25a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 -- 172.21.15.66:0/752054568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 msgr2=0x55a2d2585a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 0x55a2d2585a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 -- 172.21.15.66:0/752054568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 msgr2=0x55a2d25a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d25a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 -- 172.21.15.66:0/752054568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d258cc20 con 0x55a2d2595a00 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d25a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.701+0000 7f1b0221e640 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2594da0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1af4025b90 tx=0x7f1af4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385fc6e44e7728ad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1ae77fe640 1 -- 172.21.15.66:0/752054568 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af403a070 con 0x55a2d2595a00 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1ae77fe640 1 -- 172.21.15.66:0/752054568 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af4020030 con 0x55a2d2595a00 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1ae77fe640 1 -- 172.21.15.66:0/752054568 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4035890 con 0x55a2d2595a00 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 -- 172.21.15.66:0/752054568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 msgr2=0x55a2d2594da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2594da0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1af4025b90 tx=0x7f1af4004b50 comp rx=0 tx=0).stop 2024-04-21T09:16:53.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 -- 172.21.15.66:0/752054568 shutdown_connections 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d2595a00 0x55a2d2594da0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258ce80 0x55a2d2585a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 --2- 172.21.15.66:0/752054568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d258c4d0 0x55a2d25a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 -- 172.21.15.66:0/752054568 >> 172.21.15.66:0/752054568 conn(0x55a2d2586ba0 msgr2=0x55a2d2585370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 -- 172.21.15.66:0/752054568 shutdown_connections 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 -- 172.21.15.66:0/752054568 wait complete. 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.702+0000 7f1b0687b740 1 -- start start 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 1 Processor -- start 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d258ce80 0x55a2d2599c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d2595a00 0x55a2d259a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1afb00 con 0x55a2d258ce80 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1b0f70 con 0x55a2d2595a00 2024-04-21T09:16:53.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1b01d0 con 0x55a2d258c4d0 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0687b740 10 client.0 _ll_register_callbacks cb 0x7f1af4008f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d258ce80 0x55a2d2599c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d258ce80 0x55a2d2599c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36830/0 (socket says 172.21.15.66:36830) 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 -- 172.21.15.66:0/3496650116 learned_addr learned my addr 172.21.15.66:0/3496650116 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0221e640 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d2595a00 0x55a2d259a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 -- 172.21.15.66:0/3496650116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 msgr2=0x55a2d25996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 -- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d2595a00 msgr2=0x55a2d259a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d2595a00 0x55a2d259a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 -- 172.21.15.66:0/3496650116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d260b1b0 con 0x55a2d258ce80 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1b0121c640 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d258ce80 0x55a2d2599c00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00ab90 tx=0x7f1aec005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e926388df4a10b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.703+0000 7f1ae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1ae57fa640 1 -- 172.21.15.66:0/3496650116 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec027030 con 0x55a2d258ce80 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1ae57fa640 1 -- 172.21.15.66:0/3496650116 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f1aec0053e0 con 0x55a2d258ce80 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af4020030 con 0x55a2d258ce80 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1ae57fa640 1 -- 172.21.15.66:0/3496650116 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1aec034050 con 0x55a2d258ce80 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 10 client.17805 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2d260c420 con 0x55a2d258ce80 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1ae57fa640 1 -- 172.21.15.66:0/3496650116 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af4020030 con 0x55a2d258ce80 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1ae57fa640 1 -- 172.21.15.66:0/3496650116 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1aec004660 con 0x55a2d258ce80 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 10 client.17805 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 10 client.17805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 20 client.17805 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d260c8c0 0x55a2d260ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 4 client.17805 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2d24fec10 con 0x55a2d260c8c0 2024-04-21T09:16:53.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.704+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d260c8c0 0x55a2d260ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b01a1d640 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d260c8c0 0x55a2d260ece0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61120459c770ba2f server_cookie=fc1690736a74b09a in_seq=0 out_seq=0 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1ae57fa640 10 client.17805 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1ae57fa640 1 -- 172.21.15.66:0/3496650116 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2d24fec10 con 0x55a2d260c8c0 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1ae57fa640 10 client.17805 handle_command_reply: tid=0 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 client.17805 shutdown 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d258ce80 msgr2=0x55a2d2599c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d258ce80 0x55a2d2599c00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00ab90 tx=0x7f1aec005270 comp rx=0 tx=0).stop 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 shutdown_connections 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x55a2d260c8c0 0x55a2d260ece0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2d2595a00 0x55a2d259a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a2d258ce80 0x55a2d2599c00 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 --2- 172.21.15.66:0/3496650116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a2d258c4d0 0x55a2d25996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1ae57fa640 0 client.17805 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 >> 172.21.15.66:0/3496650116 conn(0x55a2d2586ba0 msgr2=0x55a2d258daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 shutdown_connections 2024-04-21T09:16:53.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 1 -- 172.21.15.66:0/3496650116 wait complete. 2024-04-21T09:16:53.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.705+0000 7f1b0687b740 20 client.17805 trim_cache size 0 max 16384 2024-04-21T09:16:53.713 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1ae80a3d50 msgr2=0x7f1ae80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1ae80a3d50 0x7f1ae80a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009fd0 tx=0x7f1af4009ee0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 msgr2=0x7f1afc100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc100e80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005940 tx=0x7f1af802c860 comp rx=0 tx=0).stop 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 shutdown_connections 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f1ae80a3d50 0x7f1ae80a6210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1afc101900 0x7f1afc1a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1afc107270 0x7f1afc1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 --2- 172.21.15.66:0/81327940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1afc06a840 0x7f1afc100e80 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 >> 172.21.15.66:0/81327940 conn(0x7f1afc077eb0 msgr2=0x7f1afc0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 shutdown_connections 2024-04-21T09:16:53.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.706+0000 7f1b03c81640 1 -- 172.21.15.66:0/81327940 wait complete. 2024-04-21T09:16:53.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 -- 172.21.15.66:0/1170209271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc101a40 msgr2=0x7f76bc105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1170209271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc101a40 0x7f76bc105a80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009900 tx=0x7f76a802bc30 comp rx=0 tx=0).stop 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 -- 172.21.15.66:0/1170209271 shutdown_connections 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1170209271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc106160 0x7f76bc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1170209271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc101a40 0x7f76bc105a80 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1170209271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76bc101090 0x7f76bc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 -- 172.21.15.66:0/1170209271 >> 172.21.15.66:0/1170209271 conn(0x7f76bc077f50 msgr2=0x7f76bc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 -- 172.21.15.66:0/1170209271 shutdown_connections 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 -- 172.21.15.66:0/1170209271 wait complete. 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 Processor -- start 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.821+0000 7f76c2b40640 1 -- start start 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c2b40640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc101090 0x7f76bc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c2b40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76bc106160 0x7f76bc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c2b40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc19a070 0x7f76bc1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c2b40640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1110a0 con 0x7f76bc19a070 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c2b40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc110f20 con 0x7f76bc106160 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c2b40640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc111220 con 0x7f76bc101090 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc19a070 0x7f76bc1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc19a070 0x7f76bc1a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36846/0 (socket says 172.21.15.66:36846) 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 -- 172.21.15.66:0/1524441390 learned_addr learned my addr 172.21.15.66:0/1524441390 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c08dc640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc101090 0x7f76bc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76bbfff640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76bc106160 0x7f76bc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 -- 172.21.15.66:0/1524441390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc101090 msgr2=0x7f76bc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc101090 0x7f76bc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 -- 172.21.15.66:0/1524441390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76bc106160 msgr2=0x7f76bc199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76bc106160 0x7f76bc199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 -- 172.21.15.66:0/1524441390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc1a4e20 con 0x7f76bc19a070 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76bbfff640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76bc106160 0x7f76bc199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76c10dd640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc19a070 0x7f76bc1a4620 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f76b400eb10 tx=0x7f76b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76b9ffb640 1 -- 172.21.15.66:0/1524441390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4004b20 con 0x7f76bc19a070 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.822+0000 7f76b9ffb640 1 -- 172.21.15.66:0/1524441390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b4004cc0 con 0x7f76bc19a070 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.823+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc1a5110 con 0x7f76bc19a070 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.823+0000 7f76b9ffb640 1 -- 172.21.15.66:0/1524441390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4010670 con 0x7f76bc19a070 2024-04-21T09:16:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.823+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc1a5660 con 0x7f76bc19a070 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.825+0000 7f76b9ffb640 1 -- 172.21.15.66:0/1524441390 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f76b4005020 con 0x7f76bc19a070 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.825+0000 7f76c580d740 1 Processor -- start 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.825+0000 7f76c580d740 1 -- start start 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c580d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb2408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76b9ffb640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c580d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb249430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c580d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c580d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5653bb1a51f0 con 0x5653bb249a00 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c580d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5653bb1424b0 con 0x5653bb2404d0 2024-04-21T09:16:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c580d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5653bb1a11b0 con 0x5653bb240e80 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76bbfff640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36862/0 (socket says 172.21.15.66:36862) 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 -- 172.21.15.66:0/3028432940 learned_addr learned my addr 172.21.15.66:0/3028432940 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76b9ffb640 1 -- 172.21.15.66:0/1524441390 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f76b40d1160 con 0x7f76bc19a070 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c08dc640 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb2408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76bbfff640 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb249430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 -- 172.21.15.66:0/3028432940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 msgr2=0x5653bb249430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb249430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 -- 172.21.15.66:0/3028432940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 msgr2=0x5653bb2408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb2408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76c10dd640 1 -- 172.21.15.66:0/3028432940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653bb1341b0 con 0x5653bb249a00 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.826+0000 7f76bbfff640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f76a8036040 tx=0x7f76a80042b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c10dd640 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24be50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f76b409ccc0 tx=0x7f76b4098970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d2519d84c71c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76a6ffd640 1 -- 172.21.15.66:0/3028432940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b409c690 con 0x5653bb249a00 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76a6ffd640 1 -- 172.21.15.66:0/3028432940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b409d430 con 0x5653bb249a00 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76a6ffd640 1 -- 172.21.15.66:0/3028432940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b400a250 con 0x5653bb249a00 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 -- 172.21.15.66:0/3028432940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 msgr2=0x5653bb24be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24be50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f76b409ccc0 tx=0x7f76b4098970 comp rx=0 tx=0).stop 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 -- 172.21.15.66:0/3028432940 shutdown_connections 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24be50 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb249430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 --2- 172.21.15.66:0/3028432940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb2408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 -- 172.21.15.66:0/3028432940 >> 172.21.15.66:0/3028432940 conn(0x5653bb23aba0 msgr2=0x5653bb23af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 -- 172.21.15.66:0/3028432940 shutdown_connections 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.827+0000 7f76c580d740 1 -- 172.21.15.66:0/3028432940 wait complete. 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 -- start start 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 Processor -- start 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb253a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb253f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5653bb24c970 con 0x5653bb249a00 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5653bb15e500 con 0x5653bb2404d0 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5653bb24caf0 con 0x5653bb240e80 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c580d740 10 client.0 _ll_register_callbacks cb 0x5653bb12cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c10dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c10dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36864/0 (socket says 172.21.15.66:36864) 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c10dd640 1 -- 172.21.15.66:0/2234825886 learned_addr learned my addr 172.21.15.66:0/2234825886 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.828+0000 7f76c08dc640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb253a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76bbfff640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb253f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c10dd640 1 -- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 msgr2=0x5653bb253f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c10dd640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb253f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c10dd640 1 -- 172.21.15.66:0/2234825886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 msgr2=0x5653bb253a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c10dd640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb253a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c10dd640 1 -- 172.21.15.66:0/2234825886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653baeb73f0 con 0x5653bb249a00 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c08dc640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb253a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c10dd640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24de50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f76b4099190 tx=0x7f76b409cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b3a344ba7c9eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f769bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f769bfff640 1 -- 172.21.15.66:0/2234825886 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b40bb060 con 0x5653bb249a00 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f769bfff640 1 -- 172.21.15.66:0/2234825886 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f76b409cf90 con 0x5653bb249a00 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b409d430 con 0x5653bb249a00 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f769bfff640 1 -- 172.21.15.66:0/2234825886 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76b40b64b0 con 0x5653bb249a00 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c580d740 10 client.17817 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653baf817d0 con 0x5653bb249a00 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f769bfff640 1 -- 172.21.15.66:0/2234825886 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b409d430 con 0x5653bb249a00 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f769bfff640 1 -- 172.21.15.66:0/2234825886 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76b40a5510 con 0x5653bb249a00 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c580d740 10 client.17817 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.829+0000 7f76c580d740 10 client.17817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.830+0000 7f76c580d740 20 client.17817 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.830+0000 7f76c580d740 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb25de40 0x5653bb260230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.830+0000 7f76c580d740 4 client.17817 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.830+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653bb144930 con 0x5653bb25de40 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.830+0000 7f76c08dc640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb25de40 0x5653bb260230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.830+0000 7f76c08dc640 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb25de40 0x5653bb260230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=610ee2c1ac583c3c server_cookie=59e4820c6a9b5feb in_seq=0 out_seq=0 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.830+0000 7f769bfff640 10 client.17817 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f769bfff640 1 -- 172.21.15.66:0/2234825886 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653bb144930 con 0x5653bb25de40 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f769bfff640 10 client.17817 handle_command_reply: tid=0 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 client.17817 shutdown 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 msgr2=0x5653bb24de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24de50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f76b4099190 tx=0x7f76b409cc00 comp rx=0 tx=0).stop 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 shutdown_connections 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb25de40 0x5653bb260230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb249a00 0x5653bb24de50 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb253f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 --2- 172.21.15.66:0/2234825886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb2404d0 0x5653bb253a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f769bfff640 0 client.17817 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 >> 172.21.15.66:0/2234825886 conn(0x5653bb23aba0 msgr2=0x5653bb23c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 shutdown_connections 2024-04-21T09:16:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 1 -- 172.21.15.66:0/2234825886 wait complete. 2024-04-21T09:16:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.831+0000 7f76c580d740 20 client.17817 trim_cache size 0 max 16384 2024-04-21T09:16:53.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.844+0000 7f76c580d740 1 Processor -- start 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.844+0000 7f76c580d740 1 -- start start 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c580d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb2404d0 0x5653bb253e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c580d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb240e80 0x5653bb245ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c580d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb247ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c580d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1afb20 con 0x5653bb240e80 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c580d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1ad3f0 con 0x5653bb249a00 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c580d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1ab260 con 0x5653bb2404d0 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb240e80 0x5653bb245ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb240e80 0x5653bb245ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36870/0 (socket says 172.21.15.66:36870) 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 -- 172.21.15.66:0/396897945 learned_addr learned my addr 172.21.15.66:0/396897945 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 -- 172.21.15.66:0/396897945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb2404d0 msgr2=0x5653bb253e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb2404d0 0x5653bb253e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 -- 172.21.15.66:0/396897945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 msgr2=0x5653bb247ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c10dd640 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb247ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb247ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 -- 172.21.15.66:0/396897945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653bb24deb0 con 0x5653bb240e80 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76c10dd640 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb247ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76bbfff640 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb240e80 0x5653bb245ef0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009900 tx=0x7f76a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f610d243bdf31f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76a67fc640 1 -- 172.21.15.66:0/396897945 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8054070 con 0x5653bb240e80 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76a67fc640 1 -- 172.21.15.66:0/396897945 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a80046c0 con 0x5653bb240e80 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.845+0000 7f76a67fc640 1 -- 172.21.15.66:0/396897945 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8003850 con 0x5653bb240e80 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 -- 172.21.15.66:0/396897945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb240e80 msgr2=0x5653bb245ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb240e80 0x5653bb245ef0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009900 tx=0x7f76a8000bc0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 -- 172.21.15.66:0/396897945 shutdown_connections 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb247ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb240e80 0x5653bb245ef0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 --2- 172.21.15.66:0/396897945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb2404d0 0x5653bb253e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 -- 172.21.15.66:0/396897945 >> 172.21.15.66:0/396897945 conn(0x5653bb23aba0 msgr2=0x5653bb2457e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 -- 172.21.15.66:0/396897945 shutdown_connections 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 -- 172.21.15.66:0/396897945 wait complete. 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 -- start start 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.846+0000 7f76c580d740 1 Processor -- start 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb2404d0 0x5653bb25a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb257550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb257ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1ab8b0 con 0x5653bb2404d0 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1acd20 con 0x5653bb249a00 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc1abf80 con 0x5653bb240e80 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 10 client.0 _ll_register_callbacks cb 0x7f76b4004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb2404d0 0x5653bb25a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb2404d0 0x5653bb25a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36886/0 (socket says 172.21.15.66:36886) 2024-04-21T09:16:53.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 -- 172.21.15.66:0/593117260 learned_addr learned my addr 172.21.15.66:0/593117260 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76bbfff640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb257550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c10dd640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb257ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 -- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 msgr2=0x5653bb257550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb257550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 -- 172.21.15.66:0/593117260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 msgr2=0x5653bb257ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb257ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 -- 172.21.15.66:0/593117260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653bb258330 con 0x5653bb2404d0 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c10dd640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb257ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76bbfff640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb257550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c08dc640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb2404d0 0x5653bb25a1c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b740 tx=0x7f76b00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569485500727d888 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f769bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f769bfff640 1 -- 172.21.15.66:0/593117260 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b001f2f0 con 0x5653bb2404d0 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f769bfff640 1 -- 172.21.15.66:0/593117260 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f76b001f490 con 0x5653bb2404d0 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.847+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a80046c0 con 0x5653bb2404d0 2024-04-21T09:16:53.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f769bfff640 1 -- 172.21.15.66:0/593117260 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76b002f050 con 0x5653bb2404d0 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 10 client.17823 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653bb2c4060 con 0x5653bb2404d0 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f769bfff640 1 -- 172.21.15.66:0/593117260 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a80046c0 con 0x5653bb2404d0 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f769bfff640 1 -- 172.21.15.66:0/593117260 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76b0011c00 con 0x5653bb2404d0 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 10 client.17823 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 10 client.17823 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 20 client.17823 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb2c4730 0x5653bb2c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 4 client.17823 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653bb1b2c10 con 0x5653bb2c4730 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.848+0000 7f76bbfff640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb2c4730 0x5653bb2c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f769bfff640 10 client.17823 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76bbfff640 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb2c4730 0x5653bb2c6b50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69be709d84aedabd server_cookie=665ddbe81eaafa4f in_seq=0 out_seq=0 2024-04-21T09:16:53.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f769bfff640 1 -- 172.21.15.66:0/593117260 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653bb1b2c10 con 0x5653bb2c4730 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f769bfff640 10 client.17823 handle_command_reply: tid=0 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 client.17823 shutdown 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb2404d0 msgr2=0x5653bb25a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb2404d0 0x5653bb25a1c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b740 tx=0x7f76b00035f0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 shutdown_connections 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5653bb249a00 0x5653bb257ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5653bb240e80 0x5653bb257550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x5653bb2c4730 0x5653bb2c6b50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 --2- 172.21.15.66:0/593117260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653bb2404d0 0x5653bb25a1c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f769bfff640 0 client.17823 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 >> 172.21.15.66:0/593117260 conn(0x5653bb23aba0 msgr2=0x5653bb246070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 shutdown_connections 2024-04-21T09:16:53.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 1 -- 172.21.15.66:0/593117260 wait complete. 2024-04-21T09:16:53.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.849+0000 7f76c580d740 20 client.17823 trim_cache size 0 max 16384 2024-04-21T09:16:53.858 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:16:53.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76ac0a3d20 msgr2=0x7f76ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f76a8036040 tx=0x7f76a80042b0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc19a070 msgr2=0x7f76bc1a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc19a070 0x7f76bc1a4620 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f76b400eb10 tx=0x7f76b400c2d0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 shutdown_connections 2024-04-21T09:16:53.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f76ac0a3d20 0x7f76ac0a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76bc19a070 0x7f76bc1a4620 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76bc106160 0x7f76bc199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 --2- 172.21.15.66:0/1524441390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76bc101090 0x7f76bc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 >> 172.21.15.66:0/1524441390 conn(0x7f76bc077f50 msgr2=0x7f76bc0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 shutdown_connections 2024-04-21T09:16:53.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.850+0000 7f76c2b40640 1 -- 172.21.15.66:0/1524441390 wait complete. 2024-04-21T09:16:53.861 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 -- 172.21.15.66:0/325141714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0102190 msgr2=0x7f97f0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/325141714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0102190 0x7f97f0102570 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e002bc30 comp rx=0 tx=0).stop 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 -- 172.21.15.66:0/325141714 shutdown_connections 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/325141714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f0107260 0x7f97f010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/325141714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f0102b40 0x7f97f0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/325141714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0102190 0x7f97f0102570 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 -- 172.21.15.66:0/325141714 >> 172.21.15.66:0/325141714 conn(0x7f97f00fd990 msgr2=0x7f97f00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 -- 172.21.15.66:0/325141714 shutdown_connections 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.965+0000 7f97f5c6a640 1 -- 172.21.15.66:0/325141714 wait complete. 2024-04-21T09:16:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 Processor -- start 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 -- start start 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f0102190 0x7f97f019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0107260 0x7f97f019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f0198610 0x7f97f0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01121c0 con 0x7f97f0107260 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0112040 con 0x7f97f0198610 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97f5c6a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0112340 con 0x7f97f0102190 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0107260 0x7f97f019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0107260 0x7f97f019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36906/0 (socket says 172.21.15.66:36906) 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97eeffd640 1 -- 172.21.15.66:0/3022723364 learned_addr learned my addr 172.21.15.66:0/3022723364 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f0102190 0x7f97f019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.966+0000 7f97effff640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f0198610 0x7f97f0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97eeffd640 1 -- 172.21.15.66:0/3022723364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f0102190 msgr2=0x7f97f019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97eeffd640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f0102190 0x7f97f019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97eeffd640 1 -- 172.21.15.66:0/3022723364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f0198610 msgr2=0x7f97f0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97eeffd640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f0198610 0x7f97f0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97eeffd640 1 -- 172.21.15.66:0/3022723364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f0199130 con 0x7f97f0107260 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97eeffd640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0107260 0x7f97f019e950 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e9f0 tx=0x7f97e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97ecff9640 1 -- 172.21.15.66:0/3022723364 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e400ccc0 con 0x7f97f0107260 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97ecff9640 1 -- 172.21.15.66:0/3022723364 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e40054d0 con 0x7f97f0107260 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f0199420 con 0x7f97f0107260 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97ecff9640 1 -- 172.21.15.66:0/3022723364 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4010430 con 0x7f97f0107260 2024-04-21T09:16:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.967+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f01a5620 con 0x7f97f0107260 2024-04-21T09:16:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.969+0000 7f97ecff9640 1 -- 172.21.15.66:0/3022723364 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f97e40046c0 con 0x7f97f0107260 2024-04-21T09:16:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.969+0000 7f97f885d740 1 Processor -- start 2024-04-21T09:16:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.969+0000 7f97f885d740 1 -- start start 2024-04-21T09:16:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97f885d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97f885d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97f885d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202d8a00 0x555a202dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97f885d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555a202341f0 con 0x555a202cfe80 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97f885d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a201d14b0 con 0x555a202cf4d0 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97f885d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a202301b0 con 0x555a202d8a00 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ecff9640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f97d80a3da0 0x7f97d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36920/0 (socket says 172.21.15.66:36920) 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 -- 172.21.15.66:0/2968115315 learned_addr learned my addr 172.21.15.66:0/2968115315 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97effff640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f97d80a3da0 0x7f97d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ecff9640 1 -- 172.21.15.66:0/3022723364 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f97e40d0ee0 con 0x7f97f0107260 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97effff640 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97eeffd640 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202d8a00 0x555a202dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 -- 172.21.15.66:0/2968115315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202d8a00 msgr2=0x555a202dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202d8a00 0x555a202dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 -- 172.21.15.66:0/2968115315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 msgr2=0x555a202cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 -- 172.21.15.66:0/2968115315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a201c31b0 con 0x555a202cfe80 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97eeffd640 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202d8a00 0x555a202dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97effff640 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d8430 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e00095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e38f73ee313655 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97effff640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f97d80a3da0 0x7f97d80a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97e8005fd0 tx=0x7f97e8009b60 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.970+0000 7f97d5ffb640 1 -- 172.21.15.66:0/2968115315 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0045020 con 0x555a202cfe80 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97d5ffb640 1 -- 172.21.15.66:0/2968115315 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e002c7e0 con 0x555a202cfe80 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97d5ffb640 1 -- 172.21.15.66:0/2968115315 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e00046c0 con 0x555a202cfe80 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 -- 172.21.15.66:0/2968115315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 msgr2=0x555a202d8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d8430 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e00095f0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 -- 172.21.15.66:0/2968115315 shutdown_connections 2024-04-21T09:16:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202d8a00 0x555a202dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d8430 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 --2- 172.21.15.66:0/2968115315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 -- 172.21.15.66:0/2968115315 >> 172.21.15.66:0/2968115315 conn(0x555a202c9ba0 msgr2=0x555a202c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 -- 172.21.15.66:0/2968115315 shutdown_connections 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.971+0000 7f97f885d740 1 -- 172.21.15.66:0/2968115315 wait complete. 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 -- start start 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 Processor -- start 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cf4d0 0x555a202d8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 0x555a202d56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555a202db910 con 0x555a202cf4d0 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a201ed500 con 0x555a202d8a00 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a202dba90 con 0x555a202cfe80 2024-04-21T09:16:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97f885d740 10 client.0 _ll_register_callbacks cb 0x555a201bbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cf4d0 0x555a202d8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cf4d0 0x555a202d8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36936/0 (socket says 172.21.15.66:36936) 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 -- 172.21.15.66:0/1220861014 learned_addr learned my addr 172.21.15.66:0/1220861014 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97effff640 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 -- 172.21.15.66:0/1220861014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 msgr2=0x555a202d56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 0x555a202d56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 -- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 msgr2=0x555a202d5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.972+0000 7f97ef7fe640 1 -- 172.21.15.66:0/1220861014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1ff463f0 con 0x555a202cf4d0 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cf4d0 0x555a202d8320 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f97e002c170 tx=0x7f97e002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6527dde4a5837e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97ceffd640 1 -- 172.21.15.66:0/1220861014 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0045070 con 0x555a202cf4d0 2024-04-21T09:16:53.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97ceffd640 1 -- 172.21.15.66:0/1220861014 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f97e0004450 con 0x555a202cf4d0 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e002c7e0 con 0x555a202cf4d0 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97ceffd640 1 -- 172.21.15.66:0/1220861014 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97e0058050 con 0x555a202cf4d0 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 10 client.17835 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a200107d0 con 0x555a202cf4d0 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97ceffd640 1 -- 172.21.15.66:0/1220861014 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e002c7e0 con 0x555a202cf4d0 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97ceffd640 1 -- 172.21.15.66:0/1220861014 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f97e0004a80 con 0x555a202cf4d0 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 10 client.17835 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 20 client.17835 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a202e89b0 0x555a202eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 4 client.17835 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.973+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a201d3930 con 0x555a202e89b0 2024-04-21T09:16:53.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.974+0000 7f97eeffd640 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a202e89b0 0x555a202eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.974+0000 7f97ceffd640 10 client.17835 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.974+0000 7f97eeffd640 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a202e89b0 0x555a202eada0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8971994adc76e9b server_cookie=eb49f26f5a72e637 in_seq=0 out_seq=0 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97ceffd640 1 -- 172.21.15.66:0/1220861014 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a201d3930 con 0x555a202e89b0 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97ceffd640 10 client.17835 handle_command_reply: tid=0 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 client.17835 shutdown 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cf4d0 msgr2=0x555a202d8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cf4d0 0x555a202d8320 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f97e002c170 tx=0x7f97e002c710 comp rx=0 tx=0).stop 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 shutdown_connections 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a202e89b0 0x555a202eada0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 0x555a202d56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 --2- 172.21.15.66:0/1220861014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cf4d0 0x555a202d8320 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97ceffd640 0 client.17835 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:53.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 >> 172.21.15.66:0/1220861014 conn(0x555a202c9ba0 msgr2=0x555a202cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 shutdown_connections 2024-04-21T09:16:53.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 1 -- 172.21.15.66:0/1220861014 wait complete. 2024-04-21T09:16:53.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.975+0000 7f97f885d740 20 client.17835 trim_cache size 0 max 16384 2024-04-21T09:16:53.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.988+0000 7f97f885d740 1 Processor -- start 2024-04-21T09:16:53.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.988+0000 7f97f885d740 1 -- start start 2024-04-21T09:16:53.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.988+0000 7f97f885d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.988+0000 7f97f885d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 0x555a202d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.988+0000 7f97f885d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202d8a00 0x555a202d7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.988+0000 7f97f885d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01b1be0 con 0x555a202d8a00 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.988+0000 7f97f885d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01af4b0 con 0x555a202cf4d0 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97f885d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01ad320 con 0x555a202cfe80 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202d8a00 0x555a202d7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202d8a00 0x555a202d7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36942/0 (socket says 172.21.15.66:36942) 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 -- 172.21.15.66:0/2181748441 learned_addr learned my addr 172.21.15.66:0/2181748441 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97eeffd640 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 0x555a202d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 -- 172.21.15.66:0/2181748441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 msgr2=0x555a202d4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 0x555a202d4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 -- 172.21.15.66:0/2181748441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 msgr2=0x555a202e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 -- 172.21.15.66:0/2181748441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a202e5340 con 0x555a202d8a00 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97effff640 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202d8a00 0x555a202d7de0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009fd0 tx=0x7f97e8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546e30294a8085b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97d57fa640 1 -- 172.21.15.66:0/2181748441 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8025070 con 0x555a202d8a00 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97d57fa640 1 -- 172.21.15.66:0/2181748441 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e8003e00 con 0x555a202d8a00 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.989+0000 7f97d57fa640 1 -- 172.21.15.66:0/2181748441 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e802a930 con 0x555a202d8a00 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 -- 172.21.15.66:0/2181748441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202d8a00 msgr2=0x555a202d7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202d8a00 0x555a202d7de0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009fd0 tx=0x7f97e8009500 comp rx=0 tx=0).stop 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 -- 172.21.15.66:0/2181748441 shutdown_connections 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202d8a00 0x555a202d7de0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cfe80 0x555a202d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 --2- 172.21.15.66:0/2181748441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202cf4d0 0x555a202e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 -- 172.21.15.66:0/2181748441 >> 172.21.15.66:0/2181748441 conn(0x555a202c9ba0 msgr2=0x555a202c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 -- 172.21.15.66:0/2181748441 shutdown_connections 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 -- 172.21.15.66:0/2181748441 wait complete. 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 -- start start 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.990+0000 7f97f885d740 1 Processor -- start 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cf4d0 0x555a202e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01ad970 con 0x555a202cfe80 2024-04-21T09:16:53.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01aede0 con 0x555a202d8a00 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01ae040 con 0x555a202cf4d0 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 10 client.0 _ll_register_callbacks cb 0x7f97e00056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36950/0 (socket says 172.21.15.66:36950) 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 -- 172.21.15.66:0/3821312862 learned_addr learned my addr 172.21.15.66:0/3821312862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97effff640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cf4d0 0x555a202e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 -- 172.21.15.66:0/3821312862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cf4d0 msgr2=0x555a202e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cf4d0 0x555a202e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 -- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 msgr2=0x555a202d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:53.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 -- 172.21.15.66:0/3821312862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a202d2a50 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cf4d0 0x555a202e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97eeffd640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d1ea0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f97e4098f10 tx=0x7f97e400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd939b1dd54398e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97ceffd640 1 -- 172.21.15.66:0/3821312862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e409cab0 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97ceffd640 1 -- 172.21.15.66:0/3821312862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f97e40ce4e0 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.991+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e8003e00 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97ceffd640 1 -- 172.21.15.66:0/3821312862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97e40a0e60 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 10 client.17841 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a2034eaf0 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97ceffd640 1 -- 172.21.15.66:0/3821312862 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e8003e00 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97ceffd640 1 -- 172.21.15.66:0/3821312862 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f97e409bf00 con 0x555a202cfe80 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 10 client.17841 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:54.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 20 client.17841 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a2034f270 0x555a20351690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 4 client.17841 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a20241c10 con 0x555a2034f270 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a2034f270 0x555a20351690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97ceffd640 10 client.17841 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.992+0000 7f97ef7fe640 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a2034f270 0x555a20351690 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf6e173eabdf30e server_cookie=84013874c840421f in_seq=0 out_seq=0 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97ceffd640 1 -- 172.21.15.66:0/3821312862 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a20241c10 con 0x555a2034f270 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97ceffd640 10 client.17841 handle_command_reply: tid=0 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 client.17841 shutdown 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 msgr2=0x555a202d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d1ea0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f97e4098f10 tx=0x7f97e400ce60 comp rx=0 tx=0).stop 2024-04-21T09:16:54.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 shutdown_connections 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x555a2034f270 0x555a20351690 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a202d8a00 0x555a202d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555a202cfe80 0x555a202d1ea0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97ceffd640 0 client.17841 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 --2- 172.21.15.66:0/3821312862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a202cf4d0 0x555a202e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 >> 172.21.15.66:0/3821312862 conn(0x555a202c9ba0 msgr2=0x555a202d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 shutdown_connections 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 1 -- 172.21.15.66:0/3821312862 wait complete. 2024-04-21T09:16:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f885d740 20 client.17841 trim_cache size 0 max 16384 2024-04-21T09:16:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:16:54.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f97d80a3da0 msgr2=0x7f97d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f97d80a3da0 0x7f97d80a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97e8005fd0 tx=0x7f97e8009b60 comp rx=0 tx=0).stop 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0107260 msgr2=0x7f97f019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0107260 0x7f97f019e950 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e9f0 tx=0x7f97e400eeb0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 shutdown_connections 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.993+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f97d80a3da0 0x7f97d80a6260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.994+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f0198610 0x7f97f0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.994+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97f0107260 0x7f97f019e950 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.994+0000 7f97f5c6a640 1 --2- 172.21.15.66:0/3022723364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f0102190 0x7f97f019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.994+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 >> 172.21.15.66:0/3022723364 conn(0x7f97f00fd990 msgr2=0x7f97f00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.994+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 shutdown_connections 2024-04-21T09:16:54.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:53.994+0000 7f97f5c6a640 1 -- 172.21.15.66:0/3022723364 wait complete. 2024-04-21T09:16:54.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:54 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2348675702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:16:54.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:54 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2348675702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:16:54.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:54 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2348675702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:16:55.006 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:16:55.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:16:55.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 -- 172.21.15.66:0/3373386009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974106160 msgr2=0x7ff97410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 --2- 172.21.15.66:0/3373386009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974106160 0x7ff97410dd30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009ca0 tx=0x7ff96c02c100 comp rx=0 tx=0).stop 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 -- 172.21.15.66:0/3373386009 shutdown_connections 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 --2- 172.21.15.66:0/3373386009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974106160 0x7ff97410dd30 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 --2- 172.21.15.66:0/3373386009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff974101a40 0x7ff974105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 --2- 172.21.15.66:0/3373386009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff974101090 0x7ff974101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 -- 172.21.15.66:0/3373386009 >> 172.21.15.66:0/3373386009 conn(0x7ff974077f50 msgr2=0x7ff974078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 -- 172.21.15.66:0/3373386009 shutdown_connections 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.209+0000 7ff97c575640 1 -- 172.21.15.66:0/3373386009 wait complete. 2024-04-21T09:16:55.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 Processor -- start 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 -- start start 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff974101090 0x7ff974197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff974106160 0x7ff974197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97419be30 0x7ff97419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741110b0 con 0x7ff97419be30 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff974110f30 con 0x7ff974101090 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97c575640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff974111230 con 0x7ff974106160 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97419be30 0x7ff97419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97419be30 0x7ff97419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56524/0 (socket says 172.21.15.66:56524) 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 -- 172.21.15.66:0/3666724428 learned_addr learned my addr 172.21.15.66:0/3666724428 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97a311640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff974101090 0x7ff974197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff979b10640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff974106160 0x7ff974197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 -- 172.21.15.66:0/3666724428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff974106160 msgr2=0x7ff974197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff974106160 0x7ff974197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 -- 172.21.15.66:0/3666724428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff974101090 msgr2=0x7ff974197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff974101090 0x7ff974197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.210+0000 7ff97ab12640 1 -- 172.21.15.66:0/3666724428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97419c920 con 0x7ff97419be30 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.211+0000 7ff97ab12640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97419be30 0x7ff97419c2e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005e30 tx=0x7ff96c035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.211+0000 7ff96b7fe640 1 -- 172.21.15.66:0/3666724428 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c039040 con 0x7ff97419be30 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.211+0000 7ff96b7fe640 1 -- 172.21.15.66:0/3666724428 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96c044070 con 0x7ff97419be30 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.211+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97419cbb0 con 0x7ff97419be30 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.211+0000 7ff96b7fe640 1 -- 172.21.15.66:0/3666724428 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0043c0 con 0x7ff97419be30 2024-04-21T09:16:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.211+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9741a4370 con 0x7ff97419be30 2024-04-21T09:16:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.214+0000 7ff96b7fe640 1 -- 172.21.15.66:0/3666724428 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff96c004560 con 0x7ff97419be30 2024-04-21T09:16:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.214+0000 7ff97edfd740 1 Processor -- start 2024-04-21T09:16:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.214+0000 7ff97edfd740 1 -- start start 2024-04-21T09:16:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.214+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b88cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.214+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b89290 0x55fa15b91840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b94260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff97edfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fa15aecf00 con 0x55fa15b91e10 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff97edfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa15a8a590 con 0x55fa15b89290 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff97edfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fa15ae8ec0 con 0x55fa15b888e0 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b94260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b94260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56528/0 (socket says 172.21.15.66:56528) 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 -- 172.21.15.66:0/1401229158 learned_addr learned my addr 172.21.15.66:0/1401229158 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b88cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff96b7fe640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff9600a3d20 0x7ff9600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff97a311640 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b89290 0x55fa15b91840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 -- 172.21.15.66:0/1401229158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 msgr2=0x55fa15b88cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b88cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 -- 172.21.15.66:0/1401229158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b89290 msgr2=0x55fa15b91840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b89290 0x55fa15b91840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff97ab12640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff9600a3d20 0x7ff9600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 -- 172.21.15.66:0/1401229158 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa158c77e0 con 0x55fa15b91e10 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff96b7fe640 1 -- 172.21.15.66:0/3666724428 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7ff96c0f30f0 con 0x7ff97419be30 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff979b10640 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b94260 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff96400efa0 tx=0x7ff96400e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200bdd9955d3fd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.215+0000 7ff968bfe640 1 -- 172.21.15.66:0/1401229158 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964004220 con 0x55fa15b91e10 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff968bfe640 1 -- 172.21.15.66:0/1401229158 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9640043c0 con 0x55fa15b91e10 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff968bfe640 1 -- 172.21.15.66:0/1401229158 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964003970 con 0x55fa15b91e10 2024-04-21T09:16:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97ab12640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff9600a3d20 0x7ff9600a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff96c033000 tx=0x7ff96c047fc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 -- 172.21.15.66:0/1401229158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 msgr2=0x55fa15b94260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b94260 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff96400efa0 tx=0x7ff96400e610 comp rx=0 tx=0).stop 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 -- 172.21.15.66:0/1401229158 shutdown_connections 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b94260 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b89290 0x55fa15b91840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 --2- 172.21.15.66:0/1401229158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b88cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 -- 172.21.15.66:0/1401229158 >> 172.21.15.66:0/1401229158 conn(0x55fa15b82fb0 msgr2=0x55fa15b83390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 -- 172.21.15.66:0/1401229158 shutdown_connections 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.216+0000 7ff97edfd740 1 -- 172.21.15.66:0/1401229158 wait complete. 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 -- start start 2024-04-21T09:16:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 Processor -- start 2024-04-21T09:16:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b888e0 0x55fa15b8ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 0x55fa15b8f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b8f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fa15b94d80 con 0x55fa15b91e10 2024-04-21T09:16:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa15aa6580 con 0x55fa15b888e0 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fa15b94f00 con 0x55fa15b89290 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97edfd740 10 client.0 _ll_register_callbacks cb 0x55fa15aac650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff979b10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b8f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff979b10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b8f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56530/0 (socket says 172.21.15.66:56530) 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff979b10640 1 -- 172.21.15.66:0/2384625684 learned_addr learned my addr 172.21.15.66:0/2384625684 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff97a311640 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 0x55fa15b8f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.217+0000 7ff979b10640 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b888e0 0x55fa15b8ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff979b10640 1 -- 172.21.15.66:0/2384625684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 msgr2=0x55fa15b8f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff979b10640 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 0x55fa15b8f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff979b10640 1 -- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b888e0 msgr2=0x55fa15b8ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff979b10640 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b888e0 0x55fa15b8ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff979b10640 1 -- 172.21.15.66:0/2384625684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa15927f70 con 0x55fa15b91e10 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff979b10640 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b8f860 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff964004610 tx=0x7ff964010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba524fdbfab6be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff95dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff95dffb640 1 -- 172.21.15.66:0/2384625684 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96402f020 con 0x55fa15b91e10 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9640043c0 con 0x55fa15b91e10 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff95dffb640 1 -- 172.21.15.66:0/2384625684 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7ff9640036f0 con 0x55fa15b91e10 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff95dffb640 1 -- 172.21.15.66:0/2384625684 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff96403a050 con 0x55fa15b91e10 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff97edfd740 10 client.17853 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa157bf170 con 0x55fa15b91e10 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.218+0000 7ff95dffb640 1 -- 172.21.15.66:0/2384625684 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9640043c0 con 0x55fa15b91e10 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff95dffb640 1 -- 172.21.15.66:0/2384625684 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff96402b720 con 0x55fa15b91e10 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97edfd740 10 client.17853 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97edfd740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97edfd740 20 client.17853 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15ba22c0 0x55fa15ba46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97edfd740 4 client.17853 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa15afa920 con 0x55fa15ba22c0 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97a311640 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15ba22c0 0x55fa15ba46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff95dffb640 10 client.17853 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.219+0000 7ff97a311640 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15ba22c0 0x55fa15ba46b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f9de9cdf2719de server_cookie=787fd40f3e78fed4 in_seq=0 out_seq=0 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff95dffb640 1 -- 172.21.15.66:0/2384625684 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa15afa920 con 0x55fa15ba22c0 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff95dffb640 10 client.17853 handle_command_reply: tid=0 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 client.17853 shutdown 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 msgr2=0x55fa15b8f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b8f860 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff964004610 tx=0x7ff964010f20 comp rx=0 tx=0).stop 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 shutdown_connections 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15ba22c0 0x55fa15ba46b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b91e10 0x55fa15b8f860 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 0x55fa15b8f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff95dffb640 0 client.17853 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2384625684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b888e0 0x55fa15b8ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 >> 172.21.15.66:0/2384625684 conn(0x55fa15b82fb0 msgr2=0x55fa15b84d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 shutdown_connections 2024-04-21T09:16:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.220+0000 7ff97edfd740 1 -- 172.21.15.66:0/2384625684 wait complete. 2024-04-21T09:16:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.221+0000 7ff97edfd740 20 client.17853 trim_cache size 0 max 16384 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.238+0000 7ff97edfd740 1 Processor -- start 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff97edfd740 1 -- start start 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b888e0 0x55fa15b91510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 0x55fa15b9dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 0x55fa15ba9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff97edfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741af9e0 con 0x55fa15b888e0 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff97edfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741ad2b0 con 0x55fa15b91e10 2024-04-21T09:16:55.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff97edfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741ab120 con 0x55fa15b89290 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b888e0 0x55fa15b91510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b888e0 0x55fa15b91510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56536/0 (socket says 172.21.15.66:56536) 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 -- 172.21.15.66:0/3784211603 learned_addr learned my addr 172.21.15.66:0/3784211603 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 -- 172.21.15.66:0/3784211603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 msgr2=0x55fa15b9dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 --2- 172.21.15.66:0/3784211603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 0x55fa15b9dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 -- 172.21.15.66:0/3784211603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 msgr2=0x55fa15ba9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 --2- 172.21.15.66:0/3784211603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 0x55fa15ba9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 -- 172.21.15.66:0/3784211603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa15b8f200 con 0x55fa15b888e0 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff979b10640 1 --2- 172.21.15.66:0/3784211603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b888e0 0x55fa15b91510 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff964010f90 tx=0x7ff96400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c05af4a2282069 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.239+0000 7ff95ffff640 1 -- 172.21.15.66:0/3784211603 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964043070 con 0x55fa15b888e0 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff95ffff640 1 -- 172.21.15.66:0/3784211603 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96402b450 con 0x55fa15b888e0 2024-04-21T09:16:55.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff95ffff640 1 -- 172.21.15.66:0/3784211603 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96402f040 con 0x55fa15b888e0 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 -- 172.21.15.66:0/3784211603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b888e0 msgr2=0x55fa15b91510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 --2- 172.21.15.66:0/3784211603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b888e0 0x55fa15b91510 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff964010f90 tx=0x7ff96400ed20 comp rx=0 tx=0).stop 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 -- 172.21.15.66:0/3784211603 shutdown_connections 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 --2- 172.21.15.66:0/3784211603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 0x55fa15ba9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 --2- 172.21.15.66:0/3784211603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b89290 0x55fa15b9dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 --2- 172.21.15.66:0/3784211603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b888e0 0x55fa15b91510 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 -- 172.21.15.66:0/3784211603 >> 172.21.15.66:0/3784211603 conn(0x55fa15b82fb0 msgr2=0x55fa15b82010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 -- 172.21.15.66:0/3784211603 shutdown_connections 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 -- 172.21.15.66:0/3784211603 wait complete. 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 -- start start 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.240+0000 7ff97edfd740 1 Processor -- start 2024-04-21T09:16:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b9a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b89290 0x55fa15b9a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97edfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 0x55fa15b97150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97edfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741ab770 con 0x55fa15b89290 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97edfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741acbe0 con 0x55fa15b91e10 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97edfd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741abe40 con 0x55fa15b888e0 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97edfd740 10 client.0 _ll_register_callbacks cb 0x7ff964012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b89290 0x55fa15b9a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b89290 0x55fa15b9a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56538/0 (socket says 172.21.15.66:56538) 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 -- 172.21.15.66:0/2207156905 learned_addr learned my addr 172.21.15.66:0/2207156905 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff979b10640 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b9a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff979b10640 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 0x55fa15b97150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 -- 172.21.15.66:0/2207156905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 msgr2=0x55fa15b9a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b9a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 -- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 msgr2=0x55fa15b97150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 0x55fa15b97150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 -- 172.21.15.66:0/2207156905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa15b97890 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff97a311640 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b89290 0x55fa15b9a6c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff97000bdc0 tx=0x7ff970013980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4481f666518102 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.241+0000 7ff95dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff95dffb640 1 -- 172.21.15.66:0/2207156905 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970004220 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff95dffb640 1 -- 172.21.15.66:0/2207156905 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7ff9700043c0 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff96402b450 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff95dffb640 1 -- 172.21.15.66:0/2207156905 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff97002b050 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 10 client.17859 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa15c09090 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff95dffb640 1 -- 172.21.15.66:0/2207156905 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96402b450 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff95dffb640 1 -- 172.21.15.66:0/2207156905 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff970003b20 con 0x55fa15b89290 2024-04-21T09:16:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 10 client.17859 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 20 client.17859 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15c09840 0x55fa15c0bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 4 client.17859 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa15af4790 con 0x55fa15c09840 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.242+0000 7ff979b10640 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15c09840 0x55fa15c0bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.243+0000 7ff95dffb640 10 client.17859 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.243+0000 7ff979b10640 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15c09840 0x55fa15c0bc60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a07a236ee386bd8 server_cookie=bac1a3c412c96f69 in_seq=0 out_seq=0 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff95dffb640 1 -- 172.21.15.66:0/2207156905 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa15af4790 con 0x55fa15c09840 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff95dffb640 10 client.17859 handle_command_reply: tid=0 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 client.17859 shutdown 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b89290 msgr2=0x55fa15b9a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b89290 0x55fa15b9a6c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff97000bdc0 tx=0x7ff970013980 comp rx=0 tx=0).stop 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 shutdown_connections 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55fa15c09840 0x55fa15c0bc60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa15b91e10 0x55fa15b97150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa15b89290 0x55fa15b9a6c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 --2- 172.21.15.66:0/2207156905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa15b888e0 0x55fa15b9a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 >> 172.21.15.66:0/2207156905 conn(0x55fa15b82fb0 msgr2=0x55fa15b8d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff95dffb640 0 client.17859 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 shutdown_connections 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 1 -- 172.21.15.66:0/2207156905 wait complete. 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.244+0000 7ff97edfd740 20 client.17859 trim_cache size 0 max 16384 2024-04-21T09:16:55.251 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:16:55.252 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:16:55.252 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "eb976465-54fb-4222-991d-eb19047a1d8b", 2024-04-21T09:16:55.252 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:16:55.252 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff9600a3d20 msgr2=0x7ff9600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff9600a3d20 0x7ff9600a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff96c033000 tx=0x7ff96c047fc0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97419be30 msgr2=0x7ff97419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97419be30 0x7ff97419c2e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005e30 tx=0x7ff96c035ce0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 shutdown_connections 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff9600a3d20 0x7ff9600a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97419be30 0x7ff97419c2e0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff974106160 0x7ff974197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 --2- 172.21.15.66:0/3666724428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff974101090 0x7ff974197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 >> 172.21.15.66:0/3666724428 conn(0x7ff974077f50 msgr2=0x7ff9740fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 shutdown_connections 2024-04-21T09:16:55.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.245+0000 7ff97c575640 1 -- 172.21.15.66:0/3666724428 wait complete. 2024-04-21T09:16:55.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:16:55.256 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:16:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 -- 172.21.15.66:0/2400508954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef0107270 msgr2=0x7feef010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 --2- 172.21.15.66:0/2400508954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef0107270 0x7feef010ee40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feee0009900 tx=0x7feee002bc30 comp rx=0 tx=0).stop 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 -- 172.21.15.66:0/2400508954 shutdown_connections 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 --2- 172.21.15.66:0/2400508954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef0107270 0x7feef010ee40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 --2- 172.21.15.66:0/2400508954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0102b50 0x7feef0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 --2- 172.21.15.66:0/2400508954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef01021a0 0x7feef0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 -- 172.21.15.66:0/2400508954 >> 172.21.15.66:0/2400508954 conn(0x7feef00fda40 msgr2=0x7feef00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 -- 172.21.15.66:0/2400508954 shutdown_connections 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 -- 172.21.15.66:0/2400508954 wait complete. 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.360+0000 7feef6bce640 1 Processor -- start 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef6bce640 1 -- start start 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef6bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef01021a0 0x7feef0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef6bce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0107270 0x7feef0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef6bce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef019cfb0 0x7feef019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef6bce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feef01121c0 con 0x7feef01021a0 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef6bce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feef0112040 con 0x7feef0107270 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef6bce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feef0112340 con 0x7feef019cfb0 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef01021a0 0x7feef0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef01021a0 0x7feef0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56566/0 (socket says 172.21.15.66:56566) 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 -- 172.21.15.66:0/2030051497 learned_addr learned my addr 172.21.15.66:0/2030051497 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feeeffff640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0107270 0x7feef0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef516b640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef019cfb0 0x7feef019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 -- 172.21.15.66:0/2030051497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef019cfb0 msgr2=0x7feef019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef019cfb0 0x7feef019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 -- 172.21.15.66:0/2030051497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0107270 msgr2=0x7feef0198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0107270 0x7feef0198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 -- 172.21.15.66:0/2030051497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef019daa0 con 0x7feef01021a0 2024-04-21T09:16:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feeeffff640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0107270 0x7feef0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:16:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feef496a640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef01021a0 0x7feef0198740 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7feee800db80 tx=0x7feee800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.361+0000 7feeedffb640 1 -- 172.21.15.66:0/2030051497 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8017070 con 0x7feef01021a0 2024-04-21T09:16:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.362+0000 7feeedffb640 1 -- 172.21.15.66:0/2030051497 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee8003b70 con 0x7feef01021a0 2024-04-21T09:16:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.362+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef019dd90 con 0x7feef01021a0 2024-04-21T09:16:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.362+0000 7feeedffb640 1 -- 172.21.15.66:0/2030051497 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80106c0 con 0x7feef01021a0 2024-04-21T09:16:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.362+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01a5580 con 0x7feef01021a0 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feeedffb640 1 -- 172.21.15.66:0/2030051497 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7feee8004020 con 0x7feef01021a0 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 Processor -- start 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 -- start start 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173da960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feeedffb640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7feedc0a3d20 0x7feedc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55751733f1e0 con 0x5575173daea0 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575172dc390 con 0x5575173da580 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.364+0000 7feef97be740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55751733b1a0 con 0x5575173e3a80 2024-04-21T09:16:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feef516b640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7feedc0a3d20 0x7feedc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56574/0 (socket says 172.21.15.66:56574) 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeedffb640 1 -- 172.21.15.66:0/2030051497 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7feee80d0d10 con 0x7feef01021a0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 -- 172.21.15.66:0/2268866620 learned_addr learned my addr 172.21.15.66:0/2268866620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feef496a640 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173da960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 -- 172.21.15.66:0/2268866620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 msgr2=0x5575173e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 -- 172.21.15.66:0/2268866620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 msgr2=0x5575173da960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173da960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 -- 172.21.15.66:0/2268866620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751711b7d0 con 0x5575173daea0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feef516b640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7feedc0a3d20 0x7feedc0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feee0006010 tx=0x7feee0009680 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feeeffff640 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e3540 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7feed800bc60 tx=0x7feed800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891f51caa9eba2e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feee6ffd640 1 -- 172.21.15.66:0/2268866620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8015070 con 0x5575173daea0 2024-04-21T09:16:55.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feee6ffd640 1 -- 172.21.15.66:0/2268866620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed801a910 con 0x5575173daea0 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.365+0000 7feee6ffd640 1 -- 172.21.15.66:0/2268866620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8011820 con 0x5575173daea0 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 -- 172.21.15.66:0/2268866620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 msgr2=0x5575173e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e3540 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7feed800bc60 tx=0x7feed800f730 comp rx=0 tx=0).stop 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 -- 172.21.15.66:0/2268866620 shutdown_connections 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e3540 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 --2- 172.21.15.66:0/2268866620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173da960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 -- 172.21.15.66:0/2268866620 >> 172.21.15.66:0/2268866620 conn(0x5575173d4cb0 msgr2=0x5575173d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 -- 172.21.15.66:0/2268866620 shutdown_connections 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 -- 172.21.15.66:0/2268866620 wait complete. 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 -- start start 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.366+0000 7feef97be740 1 Processor -- start 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef97be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef97be740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef97be740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173e1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef97be740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5575173e6870 con 0x5575173daea0 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef97be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575172f82f0 con 0x5575173da580 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef97be740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5575173e69f0 con 0x5575173e3a80 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef97be740 10 client.0 _ll_register_callbacks cb 0x557517305580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56586/0 (socket says 172.21.15.66:56586) 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 -- 172.21.15.66:0/1812865991 learned_addr learned my addr 172.21.15.66:0/1812865991 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef496a640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef516b640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 -- 172.21.15.66:0/1812865991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 msgr2=0x5575173e1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173e1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 -- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 msgr2=0x5575173e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feeeffff640 1 -- 172.21.15.66:0/1812865991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575170513f0 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.367+0000 7feef496a640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feeeffff640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e1040 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7feed800ffe0 tx=0x7feed801a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695093c0ace7eb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feecffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feecffff640 1 -- 172.21.15.66:0/1812865991 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8015030 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feecffff640 1 -- 172.21.15.66:0/1812865991 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7feed801aac0 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed801a910 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feecffff640 1 -- 172.21.15.66:0/1812865991 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feed801acc0 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 10 client.17871 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575170110f0 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feecffff640 1 -- 172.21.15.66:0/1812865991 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed801a910 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feecffff640 1 -- 172.21.15.66:0/1812865991 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feed8004ac0 con 0x5575173daea0 2024-04-21T09:16:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 10 client.17871 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 20 client.17871 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5575173f3ab0 0x5575173f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 4 client.17871 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.368+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55751734cc00 con 0x5575173f3ab0 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.369+0000 7feef496a640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5575173f3ab0 0x5575173f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.369+0000 7feecffff640 10 client.17871 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.369+0000 7feef496a640 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5575173f3ab0 0x5575173f5ea0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02d4772f8a9f6ae server_cookie=c9436c61adf6979e in_seq=0 out_seq=0 2024-04-21T09:16:55.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:55 smithi066 ceph-mon[27953]: pgmap v268: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.369+0000 7feecffff640 1 -- 172.21.15.66:0/1812865991 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55751734cc00 con 0x5575173f3ab0 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feecffff640 10 client.17871 handle_command_reply: tid=0 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 client.17871 shutdown 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 msgr2=0x5575173e1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e1040 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7feed800ffe0 tx=0x7feed801a460 comp rx=0 tx=0).stop 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 shutdown_connections 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5575173f3ab0 0x5575173f5ea0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173e1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feecffff640 0 client.17871 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173e1040 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 --2- 172.21.15.66:0/1812865991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173da580 0x5575173e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 >> 172.21.15.66:0/1812865991 conn(0x5575173d4cb0 msgr2=0x5575173d6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 shutdown_connections 2024-04-21T09:16:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 1 -- 172.21.15.66:0/1812865991 wait complete. 2024-04-21T09:16:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.370+0000 7feef97be740 20 client.17871 trim_cache size 0 max 16384 2024-04-21T09:16:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 Processor -- start 2024-04-21T09:16:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 -- start start 2024-04-21T09:16:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173da580 0x5575173e3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173ef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173e3a80 0x5575173ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b1ca0 con 0x5575173daea0 2024-04-21T09:16:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feef01af570 con 0x5575173e3a80 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.387+0000 7feef97be740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ad3e0 con 0x5575173da580 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173ef730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56596/0 (socket says 172.21.15.66:56596) 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 -- 172.21.15.66:0/4159108800 learned_addr learned my addr 172.21.15.66:0/4159108800 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 -- 172.21.15.66:0/4159108800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173da580 msgr2=0x5575173e3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 --2- 172.21.15.66:0/4159108800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173da580 0x5575173e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 -- 172.21.15.66:0/4159108800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173e3a80 msgr2=0x5575173ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 --2- 172.21.15.66:0/4159108800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173e3a80 0x5575173ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 -- 172.21.15.66:0/4159108800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575173e61a0 con 0x5575173daea0 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feeeffff640 1 --2- 172.21.15.66:0/4159108800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173ef730 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7feed8011660 tx=0x7feed800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8668c26ef5c95680 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feee67fc640 1 -- 172.21.15.66:0/4159108800 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8015040 con 0x5575173daea0 2024-04-21T09:16:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feee67fc640 1 -- 172.21.15.66:0/4159108800 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed800b500 con 0x5575173daea0 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.388+0000 7feee67fc640 1 -- 172.21.15.66:0/4159108800 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed801ae60 con 0x5575173daea0 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 -- 172.21.15.66:0/4159108800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 msgr2=0x5575173ef730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 --2- 172.21.15.66:0/4159108800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173ef730 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7feed8011660 tx=0x7feed800f730 comp rx=0 tx=0).stop 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 -- 172.21.15.66:0/4159108800 shutdown_connections 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 --2- 172.21.15.66:0/4159108800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173e3a80 0x5575173ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 --2- 172.21.15.66:0/4159108800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173daea0 0x5575173ef730 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 --2- 172.21.15.66:0/4159108800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173da580 0x5575173e3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 -- 172.21.15.66:0/4159108800 >> 172.21.15.66:0/4159108800 conn(0x5575173d4cb0 msgr2=0x5575173d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 -- 172.21.15.66:0/4159108800 shutdown_connections 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 -- 172.21.15.66:0/4159108800 wait complete. 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 -- start start 2024-04-21T09:16:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.389+0000 7feef97be740 1 Processor -- start 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173da580 0x5575173faea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173daea0 0x5575173fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173fb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ada30 con 0x5575173da580 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feef01aeea0 con 0x5575173daea0 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ae100 con 0x5575173e3a80 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 10 client.0 _ll_register_callbacks cb 0x7feee8004850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173da580 0x5575173faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173da580 0x5575173faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56604/0 (socket says 172.21.15.66:56604) 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 -- 172.21.15.66:0/3819327287 learned_addr learned my addr 172.21.15.66:0/3819327287 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feeeffff640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173daea0 0x5575173fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef516b640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 -- 172.21.15.66:0/3819327287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 msgr2=0x5575173fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 -- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173daea0 msgr2=0x5575173fb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173daea0 0x5575173fb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 -- 172.21.15.66:0/3819327287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55751745cd40 con 0x5575173da580 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef516b640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef496a640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173da580 0x5575173faea0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7feee809c580 tx=0x7feee8098c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af7af970a7363f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feecffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feecffff640 1 -- 172.21.15.66:0/3819327287 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee809c880 con 0x5575173da580 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feecffff640 1 -- 172.21.15.66:0/3819327287 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7feee809ab80 con 0x5575173da580 2024-04-21T09:16:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed800b500 con 0x5575173da580 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feecffff640 1 -- 172.21.15.66:0/3819327287 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feee80a0c30 con 0x5575173da580 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 10 client.17877 fetch_fsmap learned FSMap version 25 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.390+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55751745e010 con 0x5575173da580 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feecffff640 1 -- 172.21.15.66:0/3819327287 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed800b500 con 0x5575173da580 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feecffff640 1 -- 172.21.15.66:0/3819327287 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feee8023060 con 0x5575173da580 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feef97be740 10 client.17877 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feef97be740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feef97be740 20 client.17877 populate_metadata read hostname 'smithi066' 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feef97be740 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55751745e6e0 0x557517460b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feef97be740 4 client.17877 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557517346a70 con 0x55751745e6e0 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.391+0000 7feeeffff640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55751745e6e0 0x557517460b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:16:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feecffff640 10 client.17877 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feeeffff640 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55751745e6e0 0x557517460b00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698a879f6573871c server_cookie=ac54a0581e861f65 in_seq=0 out_seq=0 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feecffff640 1 -- 172.21.15.66:0/3819327287 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557517346a70 con 0x55751745e6e0 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feecffff640 10 client.17877 handle_command_reply: tid=0 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 client.17877 shutdown 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173da580 msgr2=0x5575173faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173da580 0x5575173faea0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7feee809c580 tx=0x7feee8098c50 comp rx=0 tx=0).stop 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 shutdown_connections 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55751745e6e0 0x557517460b00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575173e3a80 0x5575173fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575173daea0 0x5575173fb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feecffff640 0 client.17877 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:16:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 --2- 172.21.15.66:0/3819327287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5575173da580 0x5575173faea0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 >> 172.21.15.66:0/3819327287 conn(0x5575173d4cb0 msgr2=0x5575173e0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 shutdown_connections 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 1 -- 172.21.15.66:0/3819327287 wait complete. 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.392+0000 7feef97be740 20 client.17877 trim_cache size 0 max 16384 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stdout: "eb976465-54fb-4222-991d-eb19047a1d8b": { 2024-04-21T09:16:55.401 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:16:55.402 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "eb976465-54fb-4222-991d-eb19047a1d8b", 2024-04-21T09:16:55.402 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:16:55.402 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:16:55.402 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:16:55.402 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7feedc0a3d20 msgr2=0x7feedc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7feedc0a3d20 0x7feedc0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feee0006010 tx=0x7feee0009680 comp rx=0 tx=0).stop 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef01021a0 msgr2=0x7feef0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef01021a0 0x7feef0198740 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7feee800db80 tx=0x7feee800b630 comp rx=0 tx=0).stop 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 shutdown_connections 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7feedc0a3d20 0x7feedc0a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef019cfb0 0x7feef019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0107270 0x7feef0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 --2- 172.21.15.66:0/2030051497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feef01021a0 0x7feef0198740 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 >> 172.21.15.66:0/2030051497 conn(0x7feef00fda40 msgr2=0x7feef00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:16:55.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 shutdown_connections 2024-04-21T09:16:55.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:16:55.393+0000 7feef6bce640 1 -- 172.21.15.66:0/2030051497 wait complete. 2024-04-21T09:16:55.404 INFO:tasks.cephfs.filesystem:scrub status for tag:eb976465-54fb-4222-991d-eb19047a1d8b - {'path': '/', 'tag': 'eb976465-54fb-4222-991d-eb19047a1d8b', 'options': 'recursive,force'} 2024-04-21T09:16:55.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:55 smithi186 ceph-mon[31505]: pgmap v268: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:55 smithi083 ceph-mon[31413]: pgmap v268: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:16:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:56 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:16:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:56 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:56 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:16:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:56 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:56 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:16:56.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:56 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:16:56.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:56 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:56 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:16:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:56 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:56.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:56 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:16:56.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:56 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:16:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:56 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:56 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:16:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:56 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:16:56.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:56 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:16:57.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:57 smithi186 ceph-mon[31505]: pgmap v269: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:57.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:57 smithi083 ceph-mon[31413]: pgmap v269: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:57 smithi066 ceph-mon[27953]: pgmap v269: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:16:59.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:16:59 smithi186 ceph-mon[31505]: pgmap v270: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:16:59.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:16:59 smithi083 ceph-mon[31413]: pgmap v270: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:16:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:16:59 smithi066 ceph-mon[27953]: pgmap v270: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:01.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:01 smithi186 ceph-mon[31505]: pgmap v271: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:01.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:01 smithi083 ceph-mon[31413]: pgmap v271: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:01.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:01 smithi066 ceph-mon[27953]: pgmap v271: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:03.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:03 smithi186 ceph-mon[31505]: pgmap v272: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:03 smithi083 ceph-mon[31413]: pgmap v272: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:03 smithi066 ceph-mon[27953]: pgmap v272: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:05 smithi083 ceph-mon[31413]: pgmap v273: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:17:05.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:05 smithi066 ceph-mon[27953]: pgmap v273: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:17:05.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:05 smithi186 ceph-mon[31505]: pgmap v273: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:17:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:07 smithi083 ceph-mon[31413]: pgmap v274: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T09:17:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:07 smithi083 ceph-mon[31413]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T09:17:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:07 smithi066 ceph-mon[27953]: pgmap v274: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T09:17:07.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:07 smithi066 ceph-mon[27953]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T09:17:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:07 smithi186 ceph-mon[31505]: pgmap v274: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T09:17:07.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:07 smithi186 ceph-mon[31505]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T09:17:09.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:09 smithi083 ceph-mon[31413]: pgmap v276: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:17:09.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:09 smithi066 ceph-mon[27953]: pgmap v276: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:17:09.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:09 smithi186 ceph-mon[31505]: pgmap v276: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:17:11.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:10 smithi186 ceph-mon[31505]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T09:17:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:10 smithi186 ceph-mon[31505]: pgmap v278: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:17:11.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:10 smithi083 ceph-mon[31413]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T09:17:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:10 smithi083 ceph-mon[31413]: pgmap v278: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:17:11.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:10 smithi066 ceph-mon[27953]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T09:17:11.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:10 smithi066 ceph-mon[27953]: pgmap v278: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:17:13.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:13 smithi186 ceph-mon[31505]: pgmap v279: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:17:13.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:13 smithi083 ceph-mon[31413]: pgmap v279: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:17:13.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:13 smithi066 ceph-mon[27953]: pgmap v279: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:17:15.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:15 smithi186 ceph-mon[31505]: pgmap v280: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:17:15.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:15 smithi083 ceph-mon[31413]: pgmap v280: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:17:15.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:15 smithi066 ceph-mon[27953]: pgmap v280: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:17:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:17 smithi186 ceph-mon[31505]: pgmap v281: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T09:17:17.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:17 smithi083 ceph-mon[31413]: pgmap v281: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T09:17:17.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:17 smithi066 ceph-mon[27953]: pgmap v281: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T09:17:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:19 smithi186 ceph-mon[31505]: pgmap v282: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:17:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:19 smithi083 ceph-mon[31413]: pgmap v282: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:17:19.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:19 smithi066 ceph-mon[27953]: pgmap v282: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:17:21.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:20 smithi186 ceph-mon[31505]: pgmap v283: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:17:21.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:20 smithi083 ceph-mon[31413]: pgmap v283: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:17:21.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:20 smithi066 ceph-mon[27953]: pgmap v283: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:17:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:23 smithi186 ceph-mon[31505]: pgmap v284: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:17:23.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:23 smithi083 ceph-mon[31413]: pgmap v284: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:17:23.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:23 smithi066 ceph-mon[27953]: pgmap v284: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:17:24.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:24 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:17:24.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:24 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:17:24.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:24 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:17:25.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:17:25.405 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:17:25.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2051561328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4101080 msgr2=0x7fd7a4101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2051561328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4101080 0x7fd7a4101460 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd794009900 tx=0x7fd79402bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:25.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2051561328 shutdown_connections 2024-04-21T09:17:25.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2051561328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4106150 0x7fd7a410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2051561328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a4101a30 0x7fd7a4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2051561328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4101080 0x7fd7a4101460 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2051561328 >> 172.21.15.66:0/2051561328 conn(0x7fd7a4078000 msgr2=0x7fd7a4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2051561328 shutdown_connections 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2051561328 wait complete. 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 Processor -- start 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.514+0000 7fd7aa32c640 1 -- start start 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a4101080 0x7fd7a4102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4106150 0x7fd7a4103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4103590 0x7fd7a41a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4111110 con 0x7fd7a4106150 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4110f90 con 0x7fd7a4101080 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4111290 con 0x7fd7a4103590 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4106150 0x7fd7a4103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4106150 0x7fd7a4103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56546/0 (socket says 172.21.15.66:56546) 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 -- 172.21.15.66:0/2380899257 learned_addr learned my addr 172.21.15.66:0/2380899257 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a4101080 0x7fd7a4102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4103590 0x7fd7a41a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 -- 172.21.15.66:0/2380899257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4103590 msgr2=0x7fd7a41a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4103590 0x7fd7a41a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 -- 172.21.15.66:0/2380899257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a4101080 msgr2=0x7fd7a4102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a4101080 0x7fd7a4102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 -- 172.21.15.66:0/2380899257 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a41a7010 con 0x7fd7a4106150 2024-04-21T09:17:25.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a4101080 0x7fd7a4102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:17:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4106150 0x7fd7a4103050 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd79800e9c0 tx=0x7fd79800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a17fa640 1 -- 172.21.15.66:0/2380899257 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79800cd80 con 0x7fd7a4106150 2024-04-21T09:17:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a17fa640 1 -- 172.21.15.66:0/2380899257 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7980054d0 con 0x7fd7a4106150 2024-04-21T09:17:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a41a7300 con 0x7fd7a4106150 2024-04-21T09:17:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7a17fa640 1 -- 172.21.15.66:0/2380899257 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798010430 con 0x7fd7a4106150 2024-04-21T09:17:25.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.515+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a41a78a0 con 0x7fd7a4106150 2024-04-21T09:17:25.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a17fa640 1 -- 172.21.15.66:0/2380899257 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd798004f90 con 0x7fd7a4106150 2024-04-21T09:17:25.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 Processor -- start 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 -- start start 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5646f3c101e0 con 0x5646f3cabea0 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646f3bad390 con 0x5646f3cb4a80 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7acff0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5646f3c0c1a0 con 0x5646f3cab580 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56558/0 (socket says 172.21.15.66:56558) 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 -- 172.21.15.66:0/2091766484 learned_addr learned my addr 172.21.15.66:0/2091766484 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a17fa640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd78c0a3dd0 0x7fd78c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 -- 172.21.15.66:0/2091766484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 msgr2=0x5646f3cab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd78c0a3dd0 0x7fd78c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 -- 172.21.15.66:0/2091766484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 msgr2=0x5646f3cb6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a17fa640 1 -- 172.21.15.66:0/2380899257 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fd7980d0f90 con 0x7fd7a4106150 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.518+0000 7fd7a3fff640 1 -- 172.21.15.66:0/2091766484 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f39ec7d0 con 0x5646f3cabea0 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.519+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb4540 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd794009900 tx=0x7fd79402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55d92f8ef4541ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.519+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd78c0a3dd0 0x7fd78c0a6290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009fd0 tx=0x7fd79c005bd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.519+0000 7fd78a7fc640 1 -- 172.21.15.66:0/2091766484 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794045070 con 0x5646f3cabea0 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.519+0000 7fd78a7fc640 1 -- 172.21.15.66:0/2091766484 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79402c7e0 con 0x5646f3cabea0 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.519+0000 7fd78a7fc640 1 -- 172.21.15.66:0/2091766484 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794003ba0 con 0x5646f3cabea0 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- 172.21.15.66:0/2091766484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 msgr2=0x5646f3cb4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb4540 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd794009900 tx=0x7fd79402c5f0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- 172.21.15.66:0/2091766484 shutdown_connections 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb4540 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 --2- 172.21.15.66:0/2091766484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- 172.21.15.66:0/2091766484 >> 172.21.15.66:0/2091766484 conn(0x5646f3ca5cb0 msgr2=0x5646f3ca6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- 172.21.15.66:0/2091766484 shutdown_connections 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- 172.21.15.66:0/2091766484 wait complete. 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- start start 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 Processor -- start 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cb1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5646f3cb7810 con 0x5646f3cabea0 2024-04-21T09:17:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646f3bc92f0 con 0x5646f3cb4a80 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.520+0000 7fd7acff0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5646f3cb7990 con 0x5646f3cab580 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7acff0740 10 client.0 _ll_register_callbacks cb 0x5646f3bd6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56572/0 (socket says 172.21.15.66:56572) 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 -- 172.21.15.66:0/259400401 learned_addr learned my addr 172.21.15.66:0/259400401 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cb1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 -- 172.21.15.66:0/259400401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 msgr2=0x5646f3cb1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cb1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 -- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 msgr2=0x5646f3cb24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 -- 172.21.15.66:0/259400401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f39223f0 con 0x5646f3cabea0 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cb1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:25.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb1fb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd7980ce910 tx=0x7fd79806c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5622d0cab6e99aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7837fe640 1 -- 172.21.15.66:0/259400401 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79809c4c0 con 0x5646f3cabea0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7837fe640 1 -- 172.21.15.66:0/259400401 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fd79809ae60 con 0x5646f3cabea0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79402c7e0 con 0x5646f3cabea0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7837fe640 1 -- 172.21.15.66:0/259400401 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7980046c0 con 0x5646f3cabea0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7acff0740 10 client.17889 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.521+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646f38e20f0 con 0x5646f3cabea0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7837fe640 1 -- 172.21.15.66:0/259400401 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79402c7e0 con 0x5646f3cabea0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7837fe640 1 -- 172.21.15.66:0/259400401 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd7980991d0 con 0x5646f3cabea0 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7acff0740 10 client.17889 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7acff0740 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7acff0740 20 client.17889 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7acff0740 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3cc4960 0x5646f3cc6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7acff0740 4 client.17889 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646f3c1dc00 con 0x5646f3cc4960 2024-04-21T09:17:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3cc4960 0x5646f3cc6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3cc4960 0x5646f3cc6db0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c7a5d561949358 server_cookie=13747ff43d276fc9 in_seq=0 out_seq=0 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.522+0000 7fd7837fe640 10 client.17889 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7837fe640 1 -- 172.21.15.66:0/259400401 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646f3c1dc00 con 0x5646f3cc4960 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7837fe640 10 client.17889 handle_command_reply: tid=0 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 client.17889 shutdown 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 msgr2=0x5646f3cb1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb1fb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd7980ce910 tx=0x7fd79806c440 comp rx=0 tx=0).stop 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 shutdown_connections 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3cc4960 0x5646f3cc6db0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cb4a80 0x5646f3cb24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7837fe640 0 client.17889 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3cb1fb0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 --2- 172.21.15.66:0/259400401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cab580 0x5646f3cb1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 >> 172.21.15.66:0/259400401 conn(0x5646f3ca5cb0 msgr2=0x5646f3ca7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 shutdown_connections 2024-04-21T09:17:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.523+0000 7fd7acff0740 1 -- 172.21.15.66:0/259400401 wait complete. 2024-04-21T09:17:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.524+0000 7fd7acff0740 20 client.17889 trim_cache size 0 max 16384 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 Processor -- start 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 -- start start 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3cb41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cabea0 0x5646f3cc0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cb4a80 0x5646f3cb09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b3ce0 con 0x5646f3cb4a80 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b15b0 con 0x5646f3cab580 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7acff0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41af420 con 0x5646f3cabea0 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7a88c9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cb4a80 0x5646f3cb09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7a88c9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cb4a80 0x5646f3cb09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56584/0 (socket says 172.21.15.66:56584) 2024-04-21T09:17:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7a88c9640 1 -- 172.21.15.66:0/3643649517 learned_addr learned my addr 172.21.15.66:0/3643649517 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.541+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3cb41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cabea0 0x5646f3cc0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a88c9640 1 -- 172.21.15.66:0/3643649517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cabea0 msgr2=0x5646f3cc0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cabea0 0x5646f3cc0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a88c9640 1 -- 172.21.15.66:0/3643649517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 msgr2=0x5646f3cb41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3cb41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a88c9640 1 -- 172.21.15.66:0/3643649517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f3cabcd0 con 0x5646f3cb4a80 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cabea0 0x5646f3cc0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cb4a80 0x5646f3cb09d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0109c0 tx=0x7fd79c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1224d33c174d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd789ffb640 1 -- 172.21.15.66:0/3643649517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c004b20 con 0x5646f3cb4a80 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd789ffb640 1 -- 172.21.15.66:0/3643649517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79c020e30 con 0x5646f3cb4a80 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.542+0000 7fd789ffb640 1 -- 172.21.15.66:0/3643649517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c005760 con 0x5646f3cb4a80 2024-04-21T09:17:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- 172.21.15.66:0/3643649517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cb4a80 msgr2=0x5646f3cb09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cb4a80 0x5646f3cb09d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0109c0 tx=0x7fd79c0096c0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- 172.21.15.66:0/3643649517 shutdown_connections 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cb4a80 0x5646f3cb09d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cabea0 0x5646f3cc0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 --2- 172.21.15.66:0/3643649517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3cb41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- 172.21.15.66:0/3643649517 >> 172.21.15.66:0/3643649517 conn(0x5646f3ca5cb0 msgr2=0x5646f3ca7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- 172.21.15.66:0/3643649517 shutdown_connections 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- 172.21.15.66:0/3643649517 wait complete. 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- start start 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 Processor -- start 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3caea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3caefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cb4a80 0x5646f3d29ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41afa70 con 0x5646f3cabea0 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b0ee0 con 0x5646f3cab580 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b0140 con 0x5646f3cb4a80 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.543+0000 7fd7acff0740 10 client.0 _ll_register_callbacks cb 0x7fd798009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3caefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3caefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56590/0 (socket says 172.21.15.66:56590) 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a37fe640 1 -- 172.21.15.66:0/4277242473 learned_addr learned my addr 172.21.15.66:0/4277242473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3caea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a88c9640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cb4a80 0x5646f3d29ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a3fff640 1 -- 172.21.15.66:0/4277242473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cb4a80 msgr2=0x5646f3d29ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cb4a80 0x5646f3d29ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a3fff640 1 -- 172.21.15.66:0/4277242473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 msgr2=0x5646f3caefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3caefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a3fff640 1 -- 172.21.15.66:0/4277242473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f3caf880 con 0x5646f3cab580 2024-04-21T09:17:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3caefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7a3fff640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3caea70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd794036040 tx=0x7fd7940035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ac073eeb749ce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7837fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7837fe640 1 -- 172.21.15.66:0/4277242473 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794059020 con 0x5646f3cab580 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7837fe640 1 -- 172.21.15.66:0/4277242473 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fd7940042c0 con 0x5646f3cab580 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79c020e30 con 0x5646f3cab580 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7837fe640 1 -- 172.21.15.66:0/4277242473 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd794061050 con 0x5646f3cab580 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7acff0740 10 client.27158 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.544+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646f3d2b4e0 con 0x5646f3cab580 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7837fe640 1 -- 172.21.15.66:0/4277242473 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79c020e30 con 0x5646f3cab580 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7837fe640 1 -- 172.21.15.66:0/4277242473 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd794054070 con 0x5646f3cab580 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7acff0740 10 client.27158 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7acff0740 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:25.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7acff0740 20 client.27158 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7acff0740 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3d2b8e0 0x5646f3d2dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7acff0740 4 client.27158 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646f3c17a70 con 0x5646f3d2b8e0 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3d2b8e0 0x5646f3d2dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7a37fe640 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3d2b8e0 0x5646f3d2dcd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbce5aeb6250eec server_cookie=c4e38fdb5c6800c0 in_seq=0 out_seq=0 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.545+0000 7fd7837fe640 10 client.27158 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7837fe640 1 -- 172.21.15.66:0/4277242473 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5646f3c17a70 con 0x5646f3d2b8e0 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7837fe640 10 client.27158 handle_command_reply: tid=0 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 client.27158 shutdown 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 msgr2=0x5646f3caea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3caea70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd794036040 tx=0x7fd7940035f0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 shutdown_connections 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5646f3d2b8e0 0x5646f3d2dcd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f3cb4a80 0x5646f3d29ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5646f3cabea0 0x5646f3caefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7837fe640 0 client.27158 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 --2- 172.21.15.66:0/4277242473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f3cab580 0x5646f3caea70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 >> 172.21.15.66:0/4277242473 conn(0x5646f3ca5cb0 msgr2=0x5646f3cb1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 shutdown_connections 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 1 -- 172.21.15.66:0/4277242473 wait complete. 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.546+0000 7fd7acff0740 20 client.27158 trim_cache size 0 max 16384 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:17:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:17:25.556 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:17:25.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd78c0a3dd0 msgr2=0x7fd78c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd78c0a3dd0 0x7fd78c0a6290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009fd0 tx=0x7fd79c005bd0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4106150 msgr2=0x7fd7a4103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4106150 0x7fd7a4103050 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd79800e9c0 tx=0x7fd79800ee80 comp rx=0 tx=0).stop 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 shutdown_connections 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd78c0a3dd0 0x7fd78c0a6290 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4103590 0x7fd7a41a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a4106150 0x7fd7a4103050 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 --2- 172.21.15.66:0/2380899257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a4101080 0x7fd7a4102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 >> 172.21.15.66:0/2380899257 conn(0x7fd7a4078000 msgr2=0x7fd7a4069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 shutdown_connections 2024-04-21T09:17:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.547+0000 7fd7aa32c640 1 -- 172.21.15.66:0/2380899257 wait complete. 2024-04-21T09:17:25.557 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:25 smithi066 ceph-mon[27953]: pgmap v285: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T09:17:25.558 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:25 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:17:25.558 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:25 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:17:25.558 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:25 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:17:25.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:17:25.559 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:17:25.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:25 smithi186 ceph-mon[31505]: pgmap v285: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T09:17:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:25 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:17:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:25 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:17:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:25 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:17:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:25 smithi083 ceph-mon[31413]: pgmap v285: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T09:17:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:25 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:17:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:25 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:17:25.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:25 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:17:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 -- 172.21.15.66:0/3319956638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038101a60 msgr2=0x7fb038105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/3319956638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038101a60 0x7fb038105aa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb020009900 tx=0x7fb02002bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 -- 172.21.15.66:0/3319956638 shutdown_connections 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/3319956638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038106180 0x7fb03810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/3319956638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038101a60 0x7fb038105aa0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/3319956638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0381010b0 0x7fb038101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 -- 172.21.15.66:0/3319956638 >> 172.21.15.66:0/3319956638 conn(0x7fb0380771d0 msgr2=0x7fb0380775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 -- 172.21.15.66:0/3319956638 shutdown_connections 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 -- 172.21.15.66:0/3319956638 wait complete. 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 Processor -- start 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.661+0000 7fb03ef0d640 1 -- start start 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03ef0d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0381010b0 0x7fb038197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03ef0d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038106180 0x7fb038197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03ef0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb03819bee0 0x7fb03819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03ef0d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381110a0 con 0x7fb038106180 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03ef0d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb038110f20 con 0x7fb03819bee0 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03ef0d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb038111220 con 0x7fb0381010b0 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038106180 0x7fb038197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03cca9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0381010b0 0x7fb038197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038106180 0x7fb038197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56610/0 (socket says 172.21.15.66:56610) 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 -- 172.21.15.66:0/2281197643 learned_addr learned my addr 172.21.15.66:0/2281197643 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03d4aa640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb03819bee0 0x7fb03819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 -- 172.21.15.66:0/2281197643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0381010b0 msgr2=0x7fb038197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0381010b0 0x7fb038197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 -- 172.21.15.66:0/2281197643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb03819bee0 msgr2=0x7fb03819c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb03819bee0 0x7fb03819c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 -- 172.21.15.66:0/2281197643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03819c9d0 con 0x7fb038106180 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb037fff640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038106180 0x7fb038197b60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb020036040 tx=0x7fb02002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb035ffb640 1 -- 172.21.15.66:0/2281197643 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020043020 con 0x7fb038106180 2024-04-21T09:17:25.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb035ffb640 1 -- 172.21.15.66:0/2281197643 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02002cbe0 con 0x7fb038106180 2024-04-21T09:17:25.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03819cc60 con 0x7fb038106180 2024-04-21T09:17:25.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.662+0000 7fb035ffb640 1 -- 172.21.15.66:0/2281197643 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020003bf0 con 0x7fb038106180 2024-04-21T09:17:25.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.663+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a4480 con 0x7fb038106180 2024-04-21T09:17:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.664+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb038069230 con 0x7fb038106180 2024-04-21T09:17:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.665+0000 7fb035ffb640 1 -- 172.21.15.66:0/2281197643 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb020046020 con 0x7fb038106180 2024-04-21T09:17:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.666+0000 7fb035ffb640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb0240a3d20 0x7fb0240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.666+0000 7fb03cca9640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb0240a3d20 0x7fb0240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.666+0000 7fb035ffb640 1 -- 172.21.15.66:0/2281197643 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fb0200f2cb0 con 0x7fb038106180 2024-04-21T09:17:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.666+0000 7fb03cca9640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb0240a3d20 0x7fb0240a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb0280058c0 tx=0x7fb02800a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:25.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.671+0000 7fb035ffb640 1 -- 172.21.15.66:0/2281197643 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb02004d020 con 0x7fb038106180 2024-04-21T09:17:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.900+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb038101a60 con 0x7fb038106180 2024-04-21T09:17:25.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.901+0000 7fb035ffb640 1 -- 172.21.15.66:0/2281197643 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7fb0200bf090 con 0x7fb038106180 2024-04-21T09:17:25.901 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:17:25.901 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:17:25.901 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb0240a3d20 msgr2=0x7fb0240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb0240a3d20 0x7fb0240a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb0280058c0 tx=0x7fb02800a2d0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038106180 msgr2=0x7fb038197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038106180 0x7fb038197b60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb020036040 tx=0x7fb02002c7e0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 shutdown_connections 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fb0240a3d20 0x7fb0240a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb03819bee0 0x7fb03819c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb038106180 0x7fb038197b60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 --2- 172.21.15.66:0/2281197643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0381010b0 0x7fb038197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 >> 172.21.15.66:0/2281197643 conn(0x7fb0380771d0 msgr2=0x7fb0380fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 shutdown_connections 2024-04-21T09:17:25.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:25.903+0000 7fb03ef0d640 1 -- 172.21.15.66:0/2281197643 wait complete. 2024-04-21T09:17:25.914 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 -- 172.21.15.66:0/1013259284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac107270 msgr2=0x7f34ac10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 --2- 172.21.15.66:0/1013259284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac107270 0x7f34ac10ee40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009ca0 tx=0x7f34a402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 -- 172.21.15.66:0/1013259284 shutdown_connections 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 --2- 172.21.15.66:0/1013259284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac107270 0x7f34ac10ee40 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 --2- 172.21.15.66:0/1013259284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34ac06b1f0 0x7f34ac106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 --2- 172.21.15.66:0/1013259284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ac06a840 0x7f34ac06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 -- 172.21.15.66:0/1013259284 >> 172.21.15.66:0/1013259284 conn(0x7f34ac077eb0 msgr2=0x7f34ac0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 -- 172.21.15.66:0/1013259284 shutdown_connections 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.018+0000 7f34b46ce640 1 -- 172.21.15.66:0/1013259284 wait complete. 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 Processor -- start 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 -- start start 2024-04-21T09:17:26.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac06a840 0x7f34ac1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34ac107270 0x7f34ac198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ac19cf70 0x7f34ac19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1121d0 con 0x7f34ac06a840 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac112050 con 0x7f34ac107270 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b46ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac112350 con 0x7f34ac19cf70 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac06a840 0x7f34ac1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac06a840 0x7f34ac1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56638/0 (socket says 172.21.15.66:56638) 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 -- 172.21.15.66:0/3145400884 learned_addr learned my addr 172.21.15.66:0/3145400884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b1c69640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34ac107270 0x7f34ac198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ac19cf70 0x7f34ac19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 -- 172.21.15.66:0/3145400884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ac19cf70 msgr2=0x7f34ac19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ac19cf70 0x7f34ac19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 -- 172.21.15.66:0/3145400884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34ac107270 msgr2=0x7f34ac198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34ac107270 0x7f34ac198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.019+0000 7f34b246a640 1 -- 172.21.15.66:0/3145400884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac19da60 con 0x7f34ac06a840 2024-04-21T09:17:26.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.020+0000 7f34b246a640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac06a840 0x7f34ac1986b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f34a800d770 tx=0x7f34a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.020+0000 7f34a37fe640 1 -- 172.21.15.66:0/3145400884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8017070 con 0x7f34ac06a840 2024-04-21T09:17:26.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.020+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac19dd50 con 0x7f34ac06a840 2024-04-21T09:17:26.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.020+0000 7f34a37fe640 1 -- 172.21.15.66:0/3145400884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a80054d0 con 0x7f34ac06a840 2024-04-21T09:17:26.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.020+0000 7f34a37fe640 1 -- 172.21.15.66:0/3145400884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8004c30 con 0x7f34ac06a840 2024-04-21T09:17:26.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.020+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac1a54f0 con 0x7f34ac06a840 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.022+0000 7f34a37fe640 1 -- 172.21.15.66:0/3145400884 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f34a8004e10 con 0x7f34ac06a840 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.022+0000 7f34b7395740 1 Processor -- start 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.022+0000 7f34b7395740 1 -- start start 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b7395740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e1148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b7395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 0x56551e11d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b7395740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b7395740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56551e0791f0 con 0x56551e11da00 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b7395740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56551e0164b0 con 0x56551e1144d0 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b7395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56551e0751b0 con 0x56551e114e80 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56640/0 (socket says 172.21.15.66:56640) 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 -- 172.21.15.66:0/2325547198 learned_addr learned my addr 172.21.15.66:0/2325547198 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34a37fe640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34980a3d20 0x7f34980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b246a640 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 0x56551e11d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e1148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34980a3d20 0x7f34980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 -- 172.21.15.66:0/2325547198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 msgr2=0x56551e11d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 0x56551e11d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 -- 172.21.15.66:0/2325547198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 msgr2=0x56551e1148b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e1148b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 -- 172.21.15.66:0/2325547198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551e0081b0 con 0x56551e11da00 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34a37fe640 1 -- 172.21.15.66:0/3145400884 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f34a80d06a0 con 0x7f34ac06a840 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.023+0000 7f34b1c69640 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11fe50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f349c009c20 tx=0x7f349c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a60c923e6e9562e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34980a3d20 0x7f34980a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34a4004af0 tx=0x7f34a40095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34a0ff9640 1 -- 172.21.15.66:0/2325547198 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c004050 con 0x56551e11da00 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34a0ff9640 1 -- 172.21.15.66:0/2325547198 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c0054d0 con 0x56551e11da00 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34a0ff9640 1 -- 172.21.15.66:0/2325547198 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c004ce0 con 0x56551e11da00 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 -- 172.21.15.66:0/2325547198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 msgr2=0x56551e11fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11fe50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f349c009c20 tx=0x7f349c011810 comp rx=0 tx=0).stop 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 -- 172.21.15.66:0/2325547198 shutdown_connections 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11fe50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 0x56551e11d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 --2- 172.21.15.66:0/2325547198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e1148b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 -- 172.21.15.66:0/2325547198 >> 172.21.15.66:0/2325547198 conn(0x56551e10eba0 msgr2=0x56551e10ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 -- 172.21.15.66:0/2325547198 shutdown_connections 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.024+0000 7f34b7395740 1 -- 172.21.15.66:0/2325547198 wait complete. 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 -- start start 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 Processor -- start 2024-04-21T09:17:26.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e1144d0 0x56551e11d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e11a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56551e120970 con 0x56551e11da00 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56551e032500 con 0x56551e114e80 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56551e120af0 con 0x56551e1144d0 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b7395740 10 client.0 _ll_register_callbacks cb 0x56551e000b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b2c6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b2c6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56652/0 (socket says 172.21.15.66:56652) 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b2c6b640 1 -- 172.21.15.66:0/3297052003 learned_addr learned my addr 172.21.15.66:0/3297052003 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b1c69640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e11a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b2c6b640 1 -- 172.21.15.66:0/3297052003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e1144d0 msgr2=0x56551e11d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.025+0000 7f34b246a640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e1144d0 0x56551e11d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e1144d0 0x56551e11d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b2c6b640 1 -- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 msgr2=0x56551e11a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e11a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b2c6b640 1 -- 172.21.15.66:0/3297052003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551dd8b3f0 con 0x56551e11da00 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b1c69640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e11a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b246a640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e1144d0 0x56551e11d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11ab80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f34a4036000 tx=0x7f34a402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b24806f66626f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f3495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f3495ffb640 1 -- 172.21.15.66:0/3297052003 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4051060 con 0x56551e11da00 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f3495ffb640 1 -- 172.21.15.66:0/3297052003 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f34a4003d80 con 0x56551e11da00 2024-04-21T09:17:26.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349c0054d0 con 0x56551e11da00 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f3495ffb640 1 -- 172.21.15.66:0/3297052003 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34a4058050 con 0x56551e11da00 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b7395740 10 client.17913 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56551de557d0 con 0x56551e11da00 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f3495ffb640 1 -- 172.21.15.66:0/3297052003 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c0054d0 con 0x56551e11da00 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f3495ffb640 1 -- 172.21.15.66:0/3297052003 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f34a40416e0 con 0x56551e11da00 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b7395740 10 client.17913 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b7395740 10 client.17913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.026+0000 7f34b7395740 20 client.17913 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.027+0000 7f34b7395740 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e12dc50 0x56551e1300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.027+0000 7f34b7395740 4 client.17913 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.027+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56551e018930 con 0x56551e12dc50 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.027+0000 7f34b246a640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e12dc50 0x56551e1300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.027+0000 7f3495ffb640 10 client.17913 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:26.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.027+0000 7f34b246a640 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e12dc50 0x56551e1300d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8b93aa5ce9102b8 server_cookie=bd2b6b93e69ca403 in_seq=0 out_seq=0 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f3495ffb640 1 -- 172.21.15.66:0/3297052003 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56551e018930 con 0x56551e12dc50 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f3495ffb640 10 client.17913 handle_command_reply: tid=0 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 client.17913 shutdown 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 msgr2=0x56551e11ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11ab80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f34a4036000 tx=0x7f34a402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 shutdown_connections 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e12dc50 0x56551e1300d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11ab80 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f3495ffb640 0 client.17913 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e11a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 --2- 172.21.15.66:0/3297052003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e1144d0 0x56551e11d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 >> 172.21.15.66:0/3297052003 conn(0x56551e10eba0 msgr2=0x56551e1108f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.028+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 shutdown_connections 2024-04-21T09:17:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.029+0000 7f34b7395740 1 -- 172.21.15.66:0/3297052003 wait complete. 2024-04-21T09:17:26.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.029+0000 7f34b7395740 20 client.17913 trim_cache size 0 max 16384 2024-04-21T09:17:26.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 Processor -- start 2024-04-21T09:17:26.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 -- start start 2024-04-21T09:17:26.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e129830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 0x56551e10da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1b1b50 con 0x56551e11da00 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1af420 con 0x56551e1144d0 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b7395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1ad290 con 0x56551e114e80 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56656/0 (socket says 172.21.15.66:56656) 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 -- 172.21.15.66:0/820616754 learned_addr learned my addr 172.21.15.66:0/820616754 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b246a640 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e129830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 -- 172.21.15.66:0/820616754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 msgr2=0x56551e10da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 0x56551e10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 -- 172.21.15.66:0/820616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 msgr2=0x56551e129830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e129830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.042+0000 7f34b2c6b640 1 -- 172.21.15.66:0/820616754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551e114090 con 0x56551e11da00 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b2c6b640 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11cd70 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f34a402c630 tx=0x7f34a4009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10669dfec0837b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f3497fff640 1 -- 172.21.15.66:0/820616754 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4060020 con 0x56551e11da00 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f3497fff640 1 -- 172.21.15.66:0/820616754 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a403a040 con 0x56551e11da00 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f3497fff640 1 -- 172.21.15.66:0/820616754 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4051050 con 0x56551e11da00 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 -- 172.21.15.66:0/820616754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 msgr2=0x56551e11cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11cd70 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f34a402c630 tx=0x7f34a4009c50 comp rx=0 tx=0).stop 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 -- 172.21.15.66:0/820616754 shutdown_connections 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e11da00 0x56551e11cd70 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e114e80 0x56551e10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 --2- 172.21.15.66:0/820616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e1144d0 0x56551e129830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 -- 172.21.15.66:0/820616754 >> 172.21.15.66:0/820616754 conn(0x56551e10eba0 msgr2=0x56551e10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 -- 172.21.15.66:0/820616754 shutdown_connections 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.043+0000 7f34b7395740 1 -- 172.21.15.66:0/820616754 wait complete. 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 -- start start 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 Processor -- start 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e1144d0 0x56551e125f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e116eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e11da00 0x56551e1173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1ad8e0 con 0x56551e1144d0 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1aed50 con 0x56551e114e80 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1adfb0 con 0x56551e11da00 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b7395740 10 client.0 _ll_register_callbacks cb 0x7f34a4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e1144d0 0x56551e125f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e1144d0 0x56551e125f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56660/0 (socket says 172.21.15.66:56660) 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 -- 172.21.15.66:0/2253154880 learned_addr learned my addr 172.21.15.66:0/2253154880 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b1c69640 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e116eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 -- 172.21.15.66:0/2253154880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e11da00 msgr2=0x56551e1173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e11da00 0x56551e1173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 -- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 msgr2=0x56551e116eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e116eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 -- 172.21.15.66:0/2253154880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551e117930 con 0x56551e1144d0 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f34b246a640 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e1144d0 0x56551e125f30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f34a8098c40 tx=0x7f34a8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a052234060a20b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.044+0000 7f3495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:26.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f3495ffb640 1 -- 172.21.15.66:0/2253154880 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a80cf040 con 0x56551e1144d0 2024-04-21T09:17:26.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f3495ffb640 1 -- 172.21.15.66:0/2253154880 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f34a809a570 con 0x56551e1144d0 2024-04-21T09:17:26.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a403a040 con 0x56551e1144d0 2024-04-21T09:17:26.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f3495ffb640 1 -- 172.21.15.66:0/2253154880 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34a8020040 con 0x56551e1144d0 2024-04-21T09:17:26.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 10 client.17919 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:26.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56551e193f30 con 0x56551e1144d0 2024-04-21T09:17:26.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f3495ffb640 1 -- 172.21.15.66:0/2253154880 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a403a040 con 0x56551e1144d0 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f3495ffb640 1 -- 172.21.15.66:0/2253154880 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f34a80988e0 con 0x56551e1144d0 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 10 client.17919 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 10 client.17919 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 20 client.17919 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e194630 0x56551e196a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 4 client.17919 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56551e11af10 con 0x56551e194630 2024-04-21T09:17:26.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.045+0000 7f34b1c69640 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e194630 0x56551e196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f3495ffb640 10 client.17919 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:26.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b1c69640 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e194630 0x56551e196a50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a2e122024983b67 server_cookie=f6d6d62549b2e6ac in_seq=0 out_seq=0 2024-04-21T09:17:26.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f3495ffb640 1 -- 172.21.15.66:0/2253154880 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56551e11af10 con 0x56551e194630 2024-04-21T09:17:26.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f3495ffb640 10 client.17919 handle_command_reply: tid=0 2024-04-21T09:17:26.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 client.17919 shutdown 2024-04-21T09:17:26.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e1144d0 msgr2=0x56551e125f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e1144d0 0x56551e125f30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f34a8098c40 tx=0x7f34a8005750 comp rx=0 tx=0).stop 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 shutdown_connections 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56551e194630 0x56551e196a50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56551e11da00 0x56551e1173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56551e114e80 0x56551e116eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 --2- 172.21.15.66:0/2253154880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56551e1144d0 0x56551e125f30 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f3495ffb640 0 client.17919 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 >> 172.21.15.66:0/2253154880 conn(0x56551e10eba0 msgr2=0x56551e115af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 shutdown_connections 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 1 -- 172.21.15.66:0/2253154880 wait complete. 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.046+0000 7f34b7395740 20 client.17919 trim_cache size 0 max 16384 2024-04-21T09:17:26.057 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34980a3d20 msgr2=0x7f34980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34980a3d20 0x7f34980a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34a4004af0 tx=0x7f34a40095c0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac06a840 msgr2=0x7f34ac1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac06a840 0x7f34ac1986b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f34a800d770 tx=0x7f34a800dc30 comp rx=0 tx=0).stop 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 shutdown_connections 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f34980a3d20 0x7f34980a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ac19cf70 0x7f34ac19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34ac107270 0x7f34ac198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 --2- 172.21.15.66:0/3145400884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34ac06a840 0x7f34ac1986b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 >> 172.21.15.66:0/3145400884 conn(0x7f34ac077eb0 msgr2=0x7f34ac0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 shutdown_connections 2024-04-21T09:17:26.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.047+0000 7f34b46ce640 1 -- 172.21.15.66:0/3145400884 wait complete. 2024-04-21T09:17:26.060 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:17:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 -- 172.21.15.66:0/3118522813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc06a840 msgr2=0x7facdc06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 --2- 172.21.15.66:0/3118522813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc06a840 0x7facdc06ac20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7faccc009900 tx=0x7faccc02bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 -- 172.21.15.66:0/3118522813 shutdown_connections 2024-04-21T09:17:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 --2- 172.21.15.66:0/3118522813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc107270 0x7facdc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 --2- 172.21.15.66:0/3118522813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc06b1f0 0x7facdc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 --2- 172.21.15.66:0/3118522813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc06a840 0x7facdc06ac20 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 -- 172.21.15.66:0/3118522813 >> 172.21.15.66:0/3118522813 conn(0x7facdc077eb0 msgr2=0x7facdc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 -- 172.21.15.66:0/3118522813 shutdown_connections 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.164+0000 7face19ce640 1 -- 172.21.15.66:0/3118522813 wait complete. 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 Processor -- start 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 -- start start 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc06a840 0x7facdc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc107270 0x7facdc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc1985c0 0x7facdc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1121e0 con 0x7facdc107270 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facdc112060 con 0x7facdc1985c0 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7face19ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7facdc112360 con 0x7facdc06a840 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc107270 0x7facdc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc107270 0x7facdc19e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56678/0 (socket says 172.21.15.66:56678) 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 -- 172.21.15.66:0/1288896277 learned_addr learned my addr 172.21.15.66:0/1288896277 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facdb7fe640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc1985c0 0x7facdc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facdaffd640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc06a840 0x7facdc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 -- 172.21.15.66:0/1288896277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc06a840 msgr2=0x7facdc19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc06a840 0x7facdc19e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 -- 172.21.15.66:0/1288896277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc1985c0 msgr2=0x7facdc198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc1985c0 0x7facdc198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.165+0000 7facd3fff640 1 -- 172.21.15.66:0/1288896277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc1990e0 con 0x7facdc107270 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.166+0000 7facd3fff640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc107270 0x7facdc19e950 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7facc4005fd0 tx=0x7facc400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.166+0000 7face09cc640 1 -- 172.21.15.66:0/1288896277 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4017070 con 0x7facdc107270 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.166+0000 7face09cc640 1 -- 172.21.15.66:0/1288896277 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc4009c30 con 0x7facdc107270 2024-04-21T09:17:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.166+0000 7face09cc640 1 -- 172.21.15.66:0/1288896277 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4003d30 con 0x7facdc107270 2024-04-21T09:17:26.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.166+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc1993d0 con 0x7facdc107270 2024-04-21T09:17:26.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.166+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc1a5670 con 0x7facdc107270 2024-04-21T09:17:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face09cc640 1 -- 172.21.15.66:0/1288896277 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7facc4003ed0 con 0x7facdc107270 2024-04-21T09:17:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 Processor -- start 2024-04-21T09:17:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 -- start start 2024-04-21T09:17:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d7e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390d87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d89e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559390ce31f0 con 0x559390d7ee80 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559390c804b0 con 0x559390d87a00 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face45c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559390cdf1b0 con 0x559390d7e4d0 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7facdaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390d87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7facdaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390d87430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56684/0 (socket says 172.21.15.66:56684) 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7facdaffd640 1 -- 172.21.15.66:0/1958085891 learned_addr learned my addr 172.21.15.66:0/1958085891 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7face09cc640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7facc80a3da0 0x7facc80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7facd3fff640 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d89e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.169+0000 7facdb7fe640 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d7e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdb7fe640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7facc80a3da0 0x7facc80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdaffd640 1 -- 172.21.15.66:0/1958085891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 msgr2=0x559390d7e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdaffd640 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d7e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdaffd640 1 -- 172.21.15.66:0/1958085891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 msgr2=0x559390d89e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdaffd640 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d89e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdaffd640 1 -- 172.21.15.66:0/1958085891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559390c721b0 con 0x559390d7ee80 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdb7fe640 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d7e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7face09cc640 1 -- 172.21.15.66:0/1288896277 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7facc40d0850 con 0x7facdc107270 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdaffd640 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390d87430 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7faccc000c00 tx=0x7faccc02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa350e1779c9e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facdb7fe640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7facc80a3da0 0x7facc80a6260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7facd4009fd0 tx=0x7facd4005bd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facd1ffb640 1 -- 172.21.15.66:0/1958085891 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc043020 con 0x559390d7ee80 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facd1ffb640 1 -- 172.21.15.66:0/1958085891 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faccc02c7e0 con 0x559390d7ee80 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.170+0000 7facd1ffb640 1 -- 172.21.15.66:0/1958085891 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0055b0 con 0x559390d7ee80 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 -- 172.21.15.66:0/1958085891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 msgr2=0x559390d87430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390d87430 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7faccc000c00 tx=0x7faccc02c5d0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 -- 172.21.15.66:0/1958085891 shutdown_connections 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d89e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390d87430 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 --2- 172.21.15.66:0/1958085891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d7e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 -- 172.21.15.66:0/1958085891 >> 172.21.15.66:0/1958085891 conn(0x559390d78ba0 msgr2=0x559390d78f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 -- 172.21.15.66:0/1958085891 shutdown_connections 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 -- 172.21.15.66:0/1958085891 wait complete. 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 -- start start 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.171+0000 7face45c4740 1 Processor -- start 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d87290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7ee80 0x559390d84620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d87a00 0x559390d84b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559390d8a910 con 0x559390d87a00 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559390c9c500 con 0x559390d7ee80 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559390d8aa90 con 0x559390d7e4d0 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 10 client.0 _ll_register_callbacks cb 0x559390c6ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d87a00 0x559390d84b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d87a00 0x559390d84b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56686/0 (socket says 172.21.15.66:56686) 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 -- 172.21.15.66:0/2765503136 learned_addr learned my addr 172.21.15.66:0/2765503136 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facd3fff640 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7ee80 0x559390d84620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdaffd640 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d87290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 -- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 msgr2=0x559390d87290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d87290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 -- 172.21.15.66:0/2765503136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7ee80 msgr2=0x559390d84620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7ee80 0x559390d84620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 -- 172.21.15.66:0/2765503136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593909f53f0 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facdb7fe640 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d87a00 0x559390d84b60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7facd4009b40 tx=0x7facd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4870d298ac974fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facbaffd640 1 -- 172.21.15.66:0/2765503136 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd400fb50 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7facbaffd640 1 -- 172.21.15.66:0/2765503136 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7facd400fcf0 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.172+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faccc02c7e0 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7facbaffd640 1 -- 172.21.15.66:0/2765503136 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facd4027400 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 10 client.17931 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559390abf7d0 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7facbaffd640 1 -- 172.21.15.66:0/2765503136 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faccc02c7e0 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7facbaffd640 1 -- 172.21.15.66:0/2765503136 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facd401f5d0 con 0x559390d87a00 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 10 client.17931 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 10 client.17931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:17:26.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 20 client.17931 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390d97b80 0x559390d99fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 4 client.17931 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559390c82930 con 0x559390d97b80 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.173+0000 7facdaffd640 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390d97b80 0x559390d99fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.174+0000 7facbaffd640 10 client.17931 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.174+0000 7facdaffd640 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390d97b80 0x559390d99fb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebdf54bcf6eef9b7 server_cookie=c74bc11cdfbac4a7 in_seq=0 out_seq=0 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.174+0000 7facbaffd640 1 -- 172.21.15.66:0/2765503136 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559390c82930 con 0x559390d97b80 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.174+0000 7facbaffd640 10 client.17931 handle_command_reply: tid=0 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 client.17931 shutdown 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d87a00 msgr2=0x559390d84b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d87a00 0x559390d84b60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7facd4009b40 tx=0x7facd4004a40 comp rx=0 tx=0).stop 2024-04-21T09:17:26.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 shutdown_connections 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390d97b80 0x559390d99fb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d87a00 0x559390d84b60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7ee80 0x559390d84620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7facbaffd640 0 client.17931 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 --2- 172.21.15.66:0/2765503136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7e4d0 0x559390d87290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 >> 172.21.15.66:0/2765503136 conn(0x559390d78ba0 msgr2=0x559390d7a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 shutdown_connections 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 1 -- 172.21.15.66:0/2765503136 wait complete. 2024-04-21T09:17:26.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.175+0000 7face45c4740 20 client.17931 trim_cache size 0 max 16384 2024-04-21T09:17:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 Processor -- start 2024-04-21T09:17:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 -- start start 2024-04-21T09:17:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7e4d0 0x559390d937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7ee80 0x559390d900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d77a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1b1d00 con 0x559390d7e4d0 2024-04-21T09:17:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1af5d0 con 0x559390d87a00 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.188+0000 7face45c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ad440 con 0x559390d7ee80 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7e4d0 0x559390d937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7e4d0 0x559390d937d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56688/0 (socket says 172.21.15.66:56688) 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 -- 172.21.15.66:0/1711536419 learned_addr learned my addr 172.21.15.66:0/1711536419 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdb7fe640 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d77a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facd3fff640 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7ee80 0x559390d900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 -- 172.21.15.66:0/1711536419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7ee80 msgr2=0x559390d900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7ee80 0x559390d900c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 -- 172.21.15.66:0/1711536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 msgr2=0x559390d77a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d77a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 -- 172.21.15.66:0/1711536419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559390d94340 con 0x559390d7e4d0 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdb7fe640 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d77a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facdaffd640 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7e4d0 0x559390d937d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7faccc000c00 tx=0x7faccc005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adc551ca33fe598 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facd17fa640 1 -- 172.21.15.66:0/1711536419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc043040 con 0x559390d7e4d0 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facd17fa640 1 -- 172.21.15.66:0/1711536419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faccc052bd0 con 0x559390d7e4d0 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.189+0000 7facd17fa640 1 -- 172.21.15.66:0/1711536419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0043a0 con 0x559390d7e4d0 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 -- 172.21.15.66:0/1711536419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7e4d0 msgr2=0x559390d937d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7e4d0 0x559390d937d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7faccc000c00 tx=0x7faccc005150 comp rx=0 tx=0).stop 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 -- 172.21.15.66:0/1711536419 shutdown_connections 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d87a00 0x559390d77a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d7ee80 0x559390d900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 --2- 172.21.15.66:0/1711536419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7e4d0 0x559390d937d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 -- 172.21.15.66:0/1711536419 >> 172.21.15.66:0/1711536419 conn(0x559390d78ba0 msgr2=0x559390d77370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 -- 172.21.15.66:0/1711536419 shutdown_connections 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 -- 172.21.15.66:0/1711536419 wait complete. 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 -- start start 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.190+0000 7face45c4740 1 Processor -- start 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7e4d0 0x559390df9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390df9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d87a00 0x559390d865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ada90 con 0x559390d7ee80 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1aef00 con 0x559390d7e4d0 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ae160 con 0x559390d87a00 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 10 client.0 _ll_register_callbacks cb 0x7facd4010650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390df9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390df9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56690/0 (socket says 172.21.15.66:56690) 2024-04-21T09:17:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 -- 172.21.15.66:0/2043494492 learned_addr learned my addr 172.21.15.66:0/2043494492 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facdaffd640 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7e4d0 0x559390df9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facdb7fe640 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d87a00 0x559390d865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 -- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d87a00 msgr2=0x559390d865b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d87a00 0x559390d865b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 -- 172.21.15.66:0/2043494492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7e4d0 msgr2=0x559390df9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7e4d0 0x559390df9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 -- 172.21.15.66:0/2043494492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559390d86ca0 con 0x559390d7ee80 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facd3fff640 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390df9ea0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7facc4098df0 tx=0x7facc4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c74bac5f57a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facbaffd640 1 -- 172.21.15.66:0/2043494492 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc40a5090 con 0x559390d7ee80 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7facbaffd640 1 -- 172.21.15.66:0/2043494492 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7facc4098a90 con 0x559390d7ee80 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.191+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faccc052bd0 con 0x559390d7ee80 2024-04-21T09:17:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7facbaffd640 1 -- 172.21.15.66:0/2043494492 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facc40d0420 con 0x559390d7ee80 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 10 client.17937 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559390e022a0 con 0x559390d7ee80 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7facbaffd640 1 -- 172.21.15.66:0/2043494492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faccc052bd0 con 0x559390d7ee80 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7facbaffd640 1 -- 172.21.15.66:0/2043494492 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7facc40041c0 con 0x559390d7ee80 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 10 client.17937 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 10 client.17937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 20 client.17937 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390e02a40 0x559390e04e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 4 client.17937 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:17:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559390d84ef0 con 0x559390e02a40 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.192+0000 7facdaffd640 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390e02a40 0x559390e04e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7facbaffd640 10 client.17937 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7facdaffd640 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390e02a40 0x559390e04e60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16bed1d03886b0c1 server_cookie=ca8283a7531be37c in_seq=0 out_seq=0 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7facbaffd640 1 -- 172.21.15.66:0/2043494492 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559390d84ef0 con 0x559390e02a40 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7facbaffd640 10 client.17937 handle_command_reply: tid=0 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 client.17937 shutdown 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 msgr2=0x559390df9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390df9ea0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7facc4098df0 tx=0x7facc4009e60 comp rx=0 tx=0).stop 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 shutdown_connections 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559390d87a00 0x559390d865b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559390e02a40 0x559390e04e60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559390d7ee80 0x559390df9ea0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 --2- 172.21.15.66:0/2043494492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559390d7e4d0 0x559390df9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 >> 172.21.15.66:0/2043494492 conn(0x559390d78ba0 msgr2=0x559390d77c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7facbaffd640 0 client.17937 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 shutdown_connections 2024-04-21T09:17:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 1 -- 172.21.15.66:0/2043494492 wait complete. 2024-04-21T09:17:26.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.193+0000 7face45c4740 20 client.17937 trim_cache size 0 max 16384 2024-04-21T09:17:26.200 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:17:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7facc80a3da0 msgr2=0x7facc80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7facc80a3da0 0x7facc80a6260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7facd4009fd0 tx=0x7facd4005bd0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc107270 msgr2=0x7facdc19e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc107270 0x7facdc19e950 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7facc4005fd0 tx=0x7facc400dde0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 shutdown_connections 2024-04-21T09:17:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7facc80a3da0 0x7facc80a6260 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc1985c0 0x7facdc198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc107270 0x7facdc19e950 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 --2- 172.21.15.66:0/1288896277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc06a840 0x7facdc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 >> 172.21.15.66:0/1288896277 conn(0x7facdc077eb0 msgr2=0x7facdc0ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 shutdown_connections 2024-04-21T09:17:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.194+0000 7face19ce640 1 -- 172.21.15.66:0/1288896277 wait complete. 2024-04-21T09:17:26.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:17:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 -- 172.21.15.66:0/1840176408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4107030 msgr2=0x7f44c4107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 --2- 172.21.15.66:0/1840176408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4107030 0x7f44c4107410 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009930 tx=0x7f44b402bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 -- 172.21.15.66:0/1840176408 shutdown_connections 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 --2- 172.21.15.66:0/1840176408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c4101b40 0x7f44c410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 --2- 172.21.15.66:0/1840176408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41011a0 0x7f44c4101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 --2- 172.21.15.66:0/1840176408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4107030 0x7f44c4107410 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 -- 172.21.15.66:0/1840176408 >> 172.21.15.66:0/1840176408 conn(0x7f44c4076c50 msgr2=0x7f44c4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 -- 172.21.15.66:0/1840176408 shutdown_connections 2024-04-21T09:17:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.310+0000 7f44c8c81640 1 -- 172.21.15.66:0/1840176408 wait complete. 2024-04-21T09:17:26.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 Processor -- start 2024-04-21T09:17:26.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 -- start start 2024-04-21T09:17:26.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41011a0 0x7f44c41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4101b40 0x7f44c4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c4107030 0x7f44c419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41110e0 con 0x7f44c4101b40 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4110f60 con 0x7f44c4107030 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c8c81640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4111260 con 0x7f44c41011a0 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4101b40 0x7f44c4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4101b40 0x7f44c4198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56704/0 (socket says 172.21.15.66:56704) 2024-04-21T09:17:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 -- 172.21.15.66:0/2852844170 learned_addr learned my addr 172.21.15.66:0/2852844170 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c4107030 0x7f44c419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c259c640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41011a0 0x7f44c41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 -- 172.21.15.66:0/2852844170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41011a0 msgr2=0x7f44c41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41011a0 0x7f44c41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 -- 172.21.15.66:0/2852844170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c4107030 msgr2=0x7f44c419d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c4107030 0x7f44c419d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44bbfff640 1 -- 172.21.15.66:0/2852844170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c419d750 con 0x7f44c4101b40 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c259c640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41011a0 0x7f44c41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:26.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.311+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c4107030 0x7f44c419d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.312+0000 7f44bbfff640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4101b40 0x7f44c4198cf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00d770 tx=0x7f44ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.312+0000 7f44bb7fe640 1 -- 172.21.15.66:0/2852844170 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac017070 con 0x7f44c4101b40 2024-04-21T09:17:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.312+0000 7f44bb7fe640 1 -- 172.21.15.66:0/2852844170 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44ac0054d0 con 0x7f44c4101b40 2024-04-21T09:17:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.312+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c419da40 con 0x7f44c4101b40 2024-04-21T09:17:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.312+0000 7f44bb7fe640 1 -- 172.21.15.66:0/2852844170 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac004c30 con 0x7f44c4101b40 2024-04-21T09:17:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.312+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a52d0 con 0x7f44c4101b40 2024-04-21T09:17:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.314+0000 7f44bb7fe640 1 -- 172.21.15.66:0/2852844170 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f44ac004e10 con 0x7f44c4101b40 2024-04-21T09:17:26.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.314+0000 7f44cb873740 1 Processor -- start 2024-04-21T09:17:26.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44cb873740 1 -- start start 2024-04-21T09:17:26.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44cb873740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f6f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44cb873740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44cb873740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44cb873740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556135ed41f0 con 0x556135f6f3b0 2024-04-21T09:17:26.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44cb873740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556135e714b0 con 0x556135f78e60 2024-04-21T09:17:26.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44cb873740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556135ed01b0 con 0x556135f6fd60 2024-04-21T09:17:26.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44bb7fe640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44b00a3d20 0x7f44b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f6f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f6f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56718/0 (socket says 172.21.15.66:56718) 2024-04-21T09:17:26.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 -- 172.21.15.66:0/756471836 learned_addr learned my addr 172.21.15.66:0/756471836 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44b00a3d20 0x7f44b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c259c640 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44bb7fe640 1 -- 172.21.15.66:0/2852844170 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f44ac0d0750 con 0x7f44c4101b40 2024-04-21T09:17:26.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44bbfff640 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 -- 172.21.15.66:0/756471836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 msgr2=0x556135f78890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f78890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 -- 172.21.15.66:0/756471836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 msgr2=0x556135f792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c2d9d640 1 -- 172.21.15.66:0/756471836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556135e631b0 con 0x556135f6f3b0 2024-04-21T09:17:26.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44bbfff640 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:26.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.315+0000 7f44c259c640 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f78890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:26.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.316+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f6f790 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f44bc012d70 tx=0x7f44bc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7163b564cce3a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.316+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44b00a3d20 0x7f44b00a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f44bc010c40 tx=0x7f44bc017000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.316+0000 7f44b8ff9640 1 -- 172.21.15.66:0/756471836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc023070 con 0x556135f6f3b0 2024-04-21T09:17:26.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.316+0000 7f44b8ff9640 1 -- 172.21.15.66:0/756471836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44bc009d20 con 0x556135f6f3b0 2024-04-21T09:17:26.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.316+0000 7f44b8ff9640 1 -- 172.21.15.66:0/756471836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc005280 con 0x556135f6f3b0 2024-04-21T09:17:26.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- 172.21.15.66:0/756471836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 msgr2=0x556135f6f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f6f790 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f44bc012d70 tx=0x7f44bc010740 comp rx=0 tx=0).stop 2024-04-21T09:17:26.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- 172.21.15.66:0/756471836 shutdown_connections 2024-04-21T09:17:26.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f78890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 --2- 172.21.15.66:0/756471836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f6f790 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- 172.21.15.66:0/756471836 >> 172.21.15.66:0/756471836 conn(0x556135f69ba0 msgr2=0x556135f69f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- 172.21.15.66:0/756471836 shutdown_connections 2024-04-21T09:17:26.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- 172.21.15.66:0/756471836 wait complete. 2024-04-21T09:17:26.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- start start 2024-04-21T09:17:26.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 Processor -- start 2024-04-21T09:17:26.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f7f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6fd60 0x556135f7c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 0x556135f7ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556135f7fd40 con 0x556135f6f3b0 2024-04-21T09:17:26.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556135e8d500 con 0x556135f6fd60 2024-04-21T09:17:26.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556135f7fec0 con 0x556135f78e60 2024-04-21T09:17:26.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44cb873740 10 client.0 _ll_register_callbacks cb 0x556135e5bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:26.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.317+0000 7f44c259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f7f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f7f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56720/0 (socket says 172.21.15.66:56720) 2024-04-21T09:17:26.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 -- 172.21.15.66:0/1227932091 learned_addr learned my addr 172.21.15.66:0/1227932091 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44bbfff640 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6fd60 0x556135f7c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 -- 172.21.15.66:0/1227932091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 msgr2=0x556135f7ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 0x556135f7ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 -- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6fd60 msgr2=0x556135f7c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6fd60 0x556135f7c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 -- 172.21.15.66:0/1227932091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556135be63f0 con 0x556135f6f3b0 2024-04-21T09:17:26.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44c259c640 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f7f350 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009930 tx=0x7f44b402ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c816b0bef30431f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:26.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44a9ffb640 1 -- 172.21.15.66:0/1227932091 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4005450 con 0x556135f6f3b0 2024-04-21T09:17:26.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44a9ffb640 1 -- 172.21.15.66:0/1227932091 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f44b40046c0 con 0x556135f6f3b0 2024-04-21T09:17:26.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556135e73930 con 0x556135f6f3b0 2024-04-21T09:17:26.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44a9ffb640 1 -- 172.21.15.66:0/1227932091 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44b40055f0 con 0x556135f6f3b0 2024-04-21T09:17:26.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44cb873740 10 client.17949 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:26.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.318+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556135cb07d0 con 0x556135f6f3b0 2024-04-21T09:17:26.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44a9ffb640 1 -- 172.21.15.66:0/1227932091 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556135e73930 con 0x556135f6f3b0 2024-04-21T09:17:26.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44a9ffb640 1 -- 172.21.15.66:0/1227932091 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44b4003710 con 0x556135f6f3b0 2024-04-21T09:17:26.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44cb873740 10 client.17949 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:26.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44cb873740 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44cb873740 20 client.17949 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44cb873740 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135f8d010 0x556135f8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44cb873740 4 client.17949 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556135ee1c10 con 0x556135f8d010 2024-04-21T09:17:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44bbfff640 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135f8d010 0x556135f8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44bbfff640 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135f8d010 0x556135f8f400 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7362600eb5450a server_cookie=3774a7feb9b0c8e5 in_seq=0 out_seq=0 2024-04-21T09:17:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.319+0000 7f44a9ffb640 10 client.17949 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44a9ffb640 1 -- 172.21.15.66:0/1227932091 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556135ee1c10 con 0x556135f8d010 2024-04-21T09:17:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44a9ffb640 10 client.17949 handle_command_reply: tid=0 2024-04-21T09:17:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 client.17949 shutdown 2024-04-21T09:17:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 msgr2=0x556135f7f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f7f350 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009930 tx=0x7f44b402ca70 comp rx=0 tx=0).stop 2024-04-21T09:17:26.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 shutdown_connections 2024-04-21T09:17:26.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135f8d010 0x556135f8f400 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 0x556135f7ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44a9ffb640 0 client.17949 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:26.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6fd60 0x556135f7c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 --2- 172.21.15.66:0/1227932091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f7f350 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.320+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 >> 172.21.15.66:0/1227932091 conn(0x556135f69ba0 msgr2=0x556135f6b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.321+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 shutdown_connections 2024-04-21T09:17:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.321+0000 7f44cb873740 1 -- 172.21.15.66:0/1227932091 wait complete. 2024-04-21T09:17:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.321+0000 7f44cb873740 20 client.17949 trim_cache size 0 max 16384 2024-04-21T09:17:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 Processor -- start 2024-04-21T09:17:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 -- start start 2024-04-21T09:17:26.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f88c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f7a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41af9d0 con 0x556135f6f3b0 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41ad2a0 con 0x556135f78e60 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44cb873740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41ab110 con 0x556135f6fd60 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f88c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f88c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56728/0 (socket says 172.21.15.66:56728) 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 -- 172.21.15.66:0/3693766209 learned_addr learned my addr 172.21.15.66:0/3693766209 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 -- 172.21.15.66:0/3693766209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 msgr2=0x556135f7a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 --2- 172.21.15.66:0/3693766209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f7a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 -- 172.21.15.66:0/3693766209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 msgr2=0x556135f68a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 --2- 172.21.15.66:0/3693766209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.334+0000 7f44c259c640 1 -- 172.21.15.66:0/3693766209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556135f8f770 con 0x556135f6f3b0 2024-04-21T09:17:26.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44c259c640 1 --2- 172.21.15.66:0/3693766209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f88c40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009850 tx=0x7f44b402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a8aed5ea3c804d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44abfff640 1 -- 172.21.15.66:0/3693766209 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b402c5d0 con 0x556135f6f3b0 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44abfff640 1 -- 172.21.15.66:0/3693766209 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b4003d50 con 0x556135f6f3b0 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44abfff640 1 -- 172.21.15.66:0/3693766209 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4003750 con 0x556135f6f3b0 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 -- 172.21.15.66:0/3693766209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 msgr2=0x556135f88c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 --2- 172.21.15.66:0/3693766209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f88c40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009850 tx=0x7f44b402bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 -- 172.21.15.66:0/3693766209 shutdown_connections 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 --2- 172.21.15.66:0/3693766209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f78e60 0x556135f68a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 --2- 172.21.15.66:0/3693766209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f6fd60 0x556135f7a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 --2- 172.21.15.66:0/3693766209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6f3b0 0x556135f88c40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 -- 172.21.15.66:0/3693766209 >> 172.21.15.66:0/3693766209 conn(0x556135f69ba0 msgr2=0x556135f68370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 -- 172.21.15.66:0/3693766209 shutdown_connections 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.335+0000 7f44cb873740 1 -- 172.21.15.66:0/3693766209 wait complete. 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 -- start start 2024-04-21T09:17:26.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 Processor -- start 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6f3b0 0x556135f7e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6fd60 0x556135f7e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 0x556135f7eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41ab760 con 0x556135f6fd60 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41acbd0 con 0x556135f6f3b0 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41abe30 con 0x556135f78e60 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44cb873740 10 client.0 _ll_register_callbacks cb 0x7f44b4004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6fd60 0x556135f7e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6fd60 0x556135f7e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56738/0 (socket says 172.21.15.66:56738) 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 -- 172.21.15.66:0/2572160308 learned_addr learned my addr 172.21.15.66:0/2572160308 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44c259c640 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6f3b0 0x556135f7e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44c2d9d640 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 0x556135f7eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 -- 172.21.15.66:0/2572160308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 msgr2=0x556135f7eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 0x556135f7eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 -- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6f3b0 msgr2=0x556135f7e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6f3b0 0x556135f7e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.336+0000 7f44bbfff640 1 -- 172.21.15.66:0/2572160308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556135f836d0 con 0x556135f6fd60 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44bbfff640 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6fd60 0x556135f7e980 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f44ac09c020 tx=0x7f44ac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccd67b6ef7612e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44a9ffb640 1 -- 172.21.15.66:0/2572160308 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac020040 con 0x556135f6fd60 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44a9ffb640 1 -- 172.21.15.66:0/2572160308 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f44ac0cdd50 con 0x556135f6fd60 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b4003d50 con 0x556135f6fd60 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44a9ffb640 1 -- 172.21.15.66:0/2572160308 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44ac0a06d0 con 0x556135f6fd60 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 10 client.17955 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556135ff3990 con 0x556135f6fd60 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44a9ffb640 1 -- 172.21.15.66:0/2572160308 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b4003d50 con 0x556135f6fd60 2024-04-21T09:17:26.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44a9ffb640 1 -- 172.21.15.66:0/2572160308 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44ac098990 con 0x556135f6fd60 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 10 client.17955 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 10 client.17955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 20 client.17955 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135ff40a0 0x556135ff6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 4 client.17955 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.337+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556135edba80 con 0x556135ff40a0 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44c259c640 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135ff40a0 0x556135ff6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44a9ffb640 10 client.17955 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44c259c640 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135ff40a0 0x556135ff6490 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725ba301f2e8e220 server_cookie=a4a979a6523fd213 in_seq=0 out_seq=0 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44a9ffb640 1 -- 172.21.15.66:0/2572160308 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556135edba80 con 0x556135ff40a0 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44a9ffb640 10 client.17955 handle_command_reply: tid=0 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 client.17955 shutdown 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6fd60 msgr2=0x556135f7e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6fd60 0x556135f7e980 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f44ac09c020 tx=0x7f44ac005740 comp rx=0 tx=0).stop 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 shutdown_connections 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x556135ff40a0 0x556135ff6490 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556135f78e60 0x556135f7eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556135f6fd60 0x556135f7e980 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44a9ffb640 0 client.17955 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 --2- 172.21.15.66:0/2572160308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556135f6f3b0 0x556135f7e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 >> 172.21.15.66:0/2572160308 conn(0x556135f69ba0 msgr2=0x556135f68c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 shutdown_connections 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 1 -- 172.21.15.66:0/2572160308 wait complete. 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.338+0000 7f44cb873740 20 client.17955 trim_cache size 0 max 16384 2024-04-21T09:17:26.356 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:17:26.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44b00a3d20 msgr2=0x7f44b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44b00a3d20 0x7f44b00a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f44bc010c40 tx=0x7f44bc017000 comp rx=0 tx=0).stop 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4101b40 msgr2=0x7f44c4198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4101b40 0x7f44c4198cf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00d770 tx=0x7f44ac00dc30 comp rx=0 tx=0).stop 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 shutdown_connections 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44b00a3d20 0x7f44b00a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c4107030 0x7f44c419d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44c4101b40 0x7f44c4198cf0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 --2- 172.21.15.66:0/2852844170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41011a0 0x7f44c41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 >> 172.21.15.66:0/2852844170 conn(0x7f44c4076c50 msgr2=0x7f44c4105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 shutdown_connections 2024-04-21T09:17:26.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:26.339+0000 7f44c8c81640 1 -- 172.21.15.66:0/2852844170 wait complete. 2024-04-21T09:17:26.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:26 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2281197643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:17:26.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:26 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2281197643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:17:26.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:26 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2281197643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:17:27.361 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:17:27.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:17:27.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:17:27.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 -- 172.21.15.66:0/1053670851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f4107270 msgr2=0x7ff7f410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/1053670851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f4107270 0x7ff7f410ee40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009ca0 tx=0x7ff7ec02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 -- 172.21.15.66:0/1053670851 shutdown_connections 2024-04-21T09:17:27.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/1053670851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f4107270 0x7ff7f410ee40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/1053670851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f406b1f0 0x7ff7f4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/1053670851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f406a840 0x7ff7f406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 -- 172.21.15.66:0/1053670851 >> 172.21.15.66:0/1053670851 conn(0x7ff7f4077eb0 msgr2=0x7ff7f40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 -- 172.21.15.66:0/1053670851 shutdown_connections 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.481+0000 7ff7f8bce640 1 -- 172.21.15.66:0/1053670851 wait complete. 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 Processor -- start 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 -- start start 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f406a840 0x7ff7f41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f4107270 0x7ff7f4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f419ceb0 0x7ff7f419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41121b0 con 0x7ff7f406a840 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4112030 con 0x7ff7f4107270 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.482+0000 7ff7f8bce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4112330 con 0x7ff7f419ceb0 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f406a840 0x7ff7f41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f406a840 0x7ff7f41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56758/0 (socket says 172.21.15.66:56758) 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 -- 172.21.15.66:0/711332130 learned_addr learned my addr 172.21.15.66:0/711332130 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f4107270 0x7ff7f4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f419ceb0 0x7ff7f419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 -- 172.21.15.66:0/711332130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f419ceb0 msgr2=0x7ff7f419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f419ceb0 0x7ff7f419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 -- 172.21.15.66:0/711332130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f4107270 msgr2=0x7ff7f4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f4107270 0x7ff7f4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 -- 172.21.15.66:0/711332130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f419d9a0 con 0x7ff7f406a840 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f419ceb0 0x7ff7f419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f259c640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f406a840 0x7ff7f41985f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c920 tx=0x7ff7dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7eb7fe640 1 -- 172.21.15.66:0/711332130 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc007c70 con 0x7ff7f406a840 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f419dc90 con 0x7ff7f406a840 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7eb7fe640 1 -- 172.21.15.66:0/711332130 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7dc007e10 con 0x7ff7f406a840 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7eb7fe640 1 -- 172.21.15.66:0/711332130 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc018820 con 0x7ff7f406a840 2024-04-21T09:17:27.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.483+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41a5480 con 0x7ff7f406a840 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.485+0000 7ff7eb7fe640 1 -- 172.21.15.66:0/711332130 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff7dc00f450 con 0x7ff7f406a840 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 Processor -- start 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 -- start start 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621bfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7eb7fe640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7e00a3d20 0x7ff7e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c9250 0x55bc621c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bc62123f00 con 0x55bc621c00f0 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc620c1590 con 0x55bc621bf740 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bc6211fec0 con 0x55bc621c9250 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.486+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7e00a3d20 0x7ff7e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56774/0 (socket says 172.21.15.66:56774) 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 -- 172.21.15.66:0/1920787926 learned_addr learned my addr 172.21.15.66:0/1920787926 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7eb7fe640 1 -- 172.21.15.66:0/711332130 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7ff7dc0d0cf0 con 0x7ff7f406a840 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621bfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c9250 0x55bc621c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 -- 172.21.15.66:0/1920787926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c9250 msgr2=0x55bc621c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c9250 0x55bc621c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 -- 172.21.15.66:0/1920787926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 msgr2=0x55bc621bfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621bfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 -- 172.21.15.66:0/1920787926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc61efe7e0 con 0x55bc621c00f0 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7e00a3d20 0x7ff7e00a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff7f41085f0 tx=0x7ff7e400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7f259c640 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621c8c30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0a5000 tx=0x7ff7dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aba36308390e5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7e8ff9640 1 -- 172.21.15.66:0/1920787926 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc09c8c0 con 0x55bc621c00f0 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7e8ff9640 1 -- 172.21.15.66:0/1920787926 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7dc0a0c70 con 0x55bc621c00f0 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.487+0000 7ff7e8ff9640 1 -- 172.21.15.66:0/1920787926 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0971c0 con 0x55bc621c00f0 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/1920787926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 msgr2=0x55bc621c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621c8c30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0a5000 tx=0x7ff7dc000bc0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/1920787926 shutdown_connections 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c9250 0x55bc621c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621c8c30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/1920787926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/1920787926 >> 172.21.15.66:0/1920787926 conn(0x55bc621b9fb0 msgr2=0x55bc621ba390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/1920787926 shutdown_connections 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/1920787926 wait complete. 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.488+0000 7ff7fb7c0740 1 -- start start 2024-04-21T09:17:27.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 1 Processor -- start 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bc621d0170 con 0x55bc621c00f0 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc620dd580 con 0x55bc621c9250 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bc621d02f0 con 0x55bc621bf740 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7fb7c0740 10 client.0 _ll_register_callbacks cb 0x55bc620e3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56788/0 (socket says 172.21.15.66:56788) 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2442776443 learned_addr learned my addr 172.21.15.66:0/2442776443 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f259c640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2442776443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 msgr2=0x55bc621ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 msgr2=0x55bc621cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2442776443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc61f5ef70 con 0x55bc621c00f0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f259c640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621cd240 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40116c0 tx=0x7ff7e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a547e1180ef90faa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.489+0000 7ff7d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/2442776443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e401e830 con 0x55bc621c00f0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/2442776443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7ff7e401e9d0 con 0x55bc621c00f0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7dc0a0c70 con 0x55bc621c00f0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/2442776443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7e402f050 con 0x55bc621c00f0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 10 client.17967 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc61df6170 con 0x55bc621c00f0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/2442776443 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7dc0a0c70 con 0x55bc621c00f0 2024-04-21T09:17:27.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/2442776443 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7e401c3b0 con 0x55bc621c00f0 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 10 client.17967 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 20 client.17967 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc621dd780 0x55bc621dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 4 client.17967 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc62131920 con 0x55bc621dd780 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.490+0000 7ff7f259c640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc621dd780 0x55bc621dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.491+0000 7ff7f259c640 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc621dd780 0x55bc621dfbb0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257a065b4687110b server_cookie=6657bcfdfb144ca5 in_seq=0 out_seq=0 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.491+0000 7ff7d9ffb640 10 client.17967 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/2442776443 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc62131920 con 0x55bc621dd780 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7d9ffb640 10 client.17967 handle_command_reply: tid=0 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 client.17967 shutdown 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 msgr2=0x55bc621cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621cd240 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40116c0 tx=0x7ff7e4004730 comp rx=0 tx=0).stop 2024-04-21T09:17:27.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 shutdown_connections 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc621dd780 0x55bc621dfbb0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621cd240 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7d9ffb640 0 client.17967 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2442776443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 >> 172.21.15.66:0/2442776443 conn(0x55bc621b9fb0 msgr2=0x55bc621bbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 shutdown_connections 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2442776443 wait complete. 2024-04-21T09:17:27.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.492+0000 7ff7fb7c0740 20 client.17967 trim_cache size 0 max 16384 2024-04-21T09:17:27.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 Processor -- start 2024-04-21T09:17:27.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 -- start start 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41af940 con 0x55bc621c00f0 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41ad210 con 0x55bc621c9250 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41ab080 con 0x55bc621bf740 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621d9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56802/0 (socket says 172.21.15.66:56802) 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2568794929 learned_addr learned my addr 172.21.15.66:0/2568794929 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2568794929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 msgr2=0x55bc621cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2568794929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 msgr2=0x55bc621ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 -- 172.21.15.66:0/2568794929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc621c7430 con 0x55bc621c00f0 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621d9070 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4004700 tx=0x7ff7e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b248f70ee2e2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7dbfff640 1 -- 172.21.15.66:0/2568794929 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e402c040 con 0x55bc621c00f0 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.510+0000 7ff7dbfff640 1 -- 172.21.15.66:0/2568794929 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e4005020 con 0x55bc621c00f0 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7dbfff640 1 -- 172.21.15.66:0/2568794929 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e401cea0 con 0x55bc621c00f0 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2568794929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 msgr2=0x55bc621d9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621d9070 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4004700 tx=0x7ff7e40057f0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2568794929 shutdown_connections 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621c9250 0x55bc621ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c00f0 0x55bc621d9070 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/2568794929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621bf740 0x55bc621cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2568794929 >> 172.21.15.66:0/2568794929 conn(0x55bc621b9fb0 msgr2=0x55bc621cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2568794929 shutdown_connections 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/2568794929 wait complete. 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 -- start start 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.511+0000 7ff7fb7c0740 1 Processor -- start 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621d5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c00f0 0x55bc621d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7fb7c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c9250 0x55bc621d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41ab6d0 con 0x55bc621c9250 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41acb40 con 0x55bc621bf740 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7fb7c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41abda0 con 0x55bc621c00f0 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7fb7c0740 10 client.0 _ll_register_callbacks cb 0x7ff7e4010b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c9250 0x55bc621d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c9250 0x55bc621d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56806/0 (socket says 172.21.15.66:56806) 2024-04-21T09:17:27.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 -- 172.21.15.66:0/436836111 learned_addr learned my addr 172.21.15.66:0/436836111 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f259c640 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f1d9b640 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c00f0 0x55bc621d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 -- 172.21.15.66:0/436836111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c00f0 msgr2=0x55bc621d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c00f0 0x55bc621d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 -- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 msgr2=0x55bc621d5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621d5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 -- 172.21.15.66:0/436836111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc621d2e70 con 0x55bc621c9250 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7f2d9d640 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c9250 0x55bc621d2640 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec02c630 tx=0x7ff7ec0326f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a69bf417f85109 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.512+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/436836111 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec046070 con 0x55bc621c9250 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e4005020 con 0x55bc621c9250 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/436836111 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7ff7ec003b70 con 0x55bc621c9250 2024-04-21T09:17:27.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/436836111 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec039040 con 0x55bc621c9250 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 10 client.17973 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc622449f0 con 0x55bc621c9250 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/436836111 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e4005020 con 0x55bc621c9250 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/436836111 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7ec04b4f0 con 0x55bc621c9250 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 10 client.17973 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 20 client.17973 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc622450f0 0x55bc62247510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 4 client.17973 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc6212b790 con 0x55bc622450f0 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.513+0000 7ff7f259c640 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc622450f0 0x55bc62247510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.514+0000 7ff7d9ffb640 10 client.17973 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.514+0000 7ff7f259c640 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc622450f0 0x55bc62247510 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b0896c6d5a5208f server_cookie=4c0bb4ce78e943cb in_seq=0 out_seq=0 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7d9ffb640 1 -- 172.21.15.66:0/436836111 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc6212b790 con 0x55bc622450f0 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7d9ffb640 10 client.17973 handle_command_reply: tid=0 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 client.17973 shutdown 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c9250 msgr2=0x55bc621d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c9250 0x55bc621d2640 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec02c630 tx=0x7ff7ec0326f0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 shutdown_connections 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55bc622450f0 0x55bc62247510 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc621c9250 0x55bc621d2640 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc621c00f0 0x55bc621d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7d9ffb640 0 client.17973 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 --2- 172.21.15.66:0/436836111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc621bf740 0x55bc621d5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 >> 172.21.15.66:0/436836111 conn(0x55bc621b9fb0 msgr2=0x55bc621b9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 shutdown_connections 2024-04-21T09:17:27.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 1 -- 172.21.15.66:0/436836111 wait complete. 2024-04-21T09:17:27.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.515+0000 7ff7fb7c0740 20 client.17973 trim_cache size 0 max 16384 2024-04-21T09:17:27.521 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:17:27.521 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:17:27.521 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "e2003f76-0407-4649-aa7c-3271e1f51663", 2024-04-21T09:17:27.521 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:17:27.521 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:17:27.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7e00a3d20 msgr2=0x7ff7e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7e00a3d20 0x7ff7e00a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff7f41085f0 tx=0x7ff7e400b390 comp rx=0 tx=0).stop 2024-04-21T09:17:27.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f406a840 msgr2=0x7ff7f41985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f406a840 0x7ff7f41985f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c920 tx=0x7ff7dc00cde0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 shutdown_connections 2024-04-21T09:17:27.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7ff7e00a3d20 0x7ff7e00a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7f419ceb0 0x7ff7f419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7f4107270 0x7ff7f4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 --2- 172.21.15.66:0/711332130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f406a840 0x7ff7f41985f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 >> 172.21.15.66:0/711332130 conn(0x7ff7f4077eb0 msgr2=0x7ff7f40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 shutdown_connections 2024-04-21T09:17:27.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.516+0000 7ff7f8bce640 1 -- 172.21.15.66:0/711332130 wait complete. 2024-04-21T09:17:27.525 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:27 smithi066 ceph-mon[27953]: pgmap v286: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:27.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:17:27.526 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:17:27.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:27 smithi186 ceph-mon[31505]: pgmap v286: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:27.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:27 smithi083 ceph-mon[31413]: pgmap v286: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 -- 172.21.15.66:0/1606026543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406af70 msgr2=0x7f451410ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1606026543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406af70 0x7f451410ede0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f450c009cd0 tx=0x7f450c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 -- 172.21.15.66:0/1606026543 shutdown_connections 2024-04-21T09:17:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1606026543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406af70 0x7f451410ede0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1606026543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f451406a5b0 0x7f451406aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1606026543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4514108060 0x7f4514108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 -- 172.21.15.66:0/1606026543 >> 172.21.15.66:0/1606026543 conn(0x7f4514077c40 msgr2=0x7f4514078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 -- 172.21.15.66:0/1606026543 shutdown_connections 2024-04-21T09:17:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.631+0000 7f451a4ba640 1 -- 172.21.15.66:0/1606026543 wait complete. 2024-04-21T09:17:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 Processor -- start 2024-04-21T09:17:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 -- start start 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406a5b0 0x7f4514198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f451406af70 0x7f4514198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4514108060 0x7f451419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45141121a0 con 0x7f451406a5b0 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4514112020 con 0x7f4514108060 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f451a4ba640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4514112320 con 0x7f451406af70 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f4513fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406a5b0 0x7f4514198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f4513fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406a5b0 0x7f4514198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56820/0 (socket says 172.21.15.66:56820) 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.632+0000 7f4513fff640 1 -- 172.21.15.66:0/1978341755 learned_addr learned my addr 172.21.15.66:0/1978341755 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f45137fe640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f451406af70 0x7f4514198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f4518a57640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4514108060 0x7f451419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f4513fff640 1 -- 172.21.15.66:0/1978341755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f451406af70 msgr2=0x7f4514198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f4513fff640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f451406af70 0x7f4514198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f4513fff640 1 -- 172.21.15.66:0/1978341755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4514108060 msgr2=0x7f451419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f4513fff640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4514108060 0x7f451419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f4513fff640 1 -- 172.21.15.66:0/1978341755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451419d600 con 0x7f451406a5b0 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f4513fff640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406a5b0 0x7f4514198660 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f450400e9f0 tx=0x7f450400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f45117fa640 1 -- 172.21.15.66:0/1978341755 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450400cd80 con 0x7f451406a5b0 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451419d8f0 con 0x7f451406a5b0 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f45117fa640 1 -- 172.21.15.66:0/1978341755 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45040054d0 con 0x7f451406a5b0 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f45117fa640 1 -- 172.21.15.66:0/1978341755 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504010690 con 0x7f451406a5b0 2024-04-21T09:17:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.633+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45141a5180 con 0x7f451406a5b0 2024-04-21T09:17:27.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.635+0000 7f45117fa640 1 -- 172.21.15.66:0/1978341755 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4504004e90 con 0x7f451406a5b0 2024-04-21T09:17:27.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.635+0000 7f451d187740 1 Processor -- start 2024-04-21T09:17:27.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f451d187740 1 -- start start 2024-04-21T09:17:27.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f451d187740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d28690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f451d187740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d35950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f451d187740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 0x5562a0d38280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f451d187740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5562a0c8d1e0 con 0x5562a0d28bd0 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f451d187740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5562a0c2a390 con 0x5562a0d282b0 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f451d187740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5562a0c891a0 con 0x5562a0d35e90 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d35950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d35950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56832/0 (socket says 172.21.15.66:56832) 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 -- 172.21.15.66:0/3694291865 learned_addr learned my addr 172.21.15.66:0/3694291865 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f45117fa640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44fc0a3d20 0x7f44fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f45137fe640 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 0x5562a0d38280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4518a57640 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d28690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4518a57640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44fc0a3d20 0x7f44fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 -- 172.21.15.66:0/3694291865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 msgr2=0x5562a0d38280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 0x5562a0d38280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 -- 172.21.15.66:0/3694291865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 msgr2=0x5562a0d28690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d28690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f4513fff640 1 -- 172.21.15.66:0/3694291865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a0a697d0 con 0x5562a0d28bd0 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.636+0000 7f45117fa640 1 -- 172.21.15.66:0/1978341755 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f45040d1110 con 0x7f451406a5b0 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f4513fff640 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d35950 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f450409cc70 tx=0x7f450409cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f5fe726e51d77b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f4518a57640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44fc0a3d20 0x7f44fc0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f450c02c630 tx=0x7f450c0095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f44fa7fc640 1 -- 172.21.15.66:0/3694291865 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450409c640 con 0x5562a0d28bd0 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f44fa7fc640 1 -- 172.21.15.66:0/3694291865 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4504005680 con 0x5562a0d28bd0 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f44fa7fc640 1 -- 172.21.15.66:0/3694291865 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040071c0 con 0x5562a0d28bd0 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 -- 172.21.15.66:0/3694291865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 msgr2=0x5562a0d35950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d35950 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f450409cc70 tx=0x7f450409cbe0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 -- 172.21.15.66:0/3694291865 shutdown_connections 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 0x5562a0d38280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d35950 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 --2- 172.21.15.66:0/3694291865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d28690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 -- 172.21.15.66:0/3694291865 >> 172.21.15.66:0/3694291865 conn(0x5562a0d22cb0 msgr2=0x5562a0d23090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 -- 172.21.15.66:0/3694291865 shutdown_connections 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.637+0000 7f451d187740 1 -- 172.21.15.66:0/3694291865 wait complete. 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 -- start start 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 Processor -- start 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d28bd0 0x5562a0d33410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d35e90 0x5562a0d33950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5562a0d38b90 con 0x5562a0d35e90 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5562a0c462f0 con 0x5562a0d282b0 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5562a0d38d10 con 0x5562a0d28bd0 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f451d187740 10 client.0 _ll_register_callbacks cb 0x5562a0c53580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f4518a57640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d35e90 0x5562a0d33950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f4518a57640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d35e90 0x5562a0d33950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56846/0 (socket says 172.21.15.66:56846) 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f4518a57640 1 -- 172.21.15.66:0/3487849850 learned_addr learned my addr 172.21.15.66:0/3487849850 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.638+0000 7f45137fe640 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d28bd0 0x5562a0d33410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f4513fff640 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f4518a57640 1 -- 172.21.15.66:0/3487849850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d28bd0 msgr2=0x5562a0d33410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f4518a57640 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d28bd0 0x5562a0d33410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f4518a57640 1 -- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 msgr2=0x5562a0d32ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f4518a57640 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d32ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f4518a57640 1 -- 172.21.15.66:0/3487849850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a099f3f0 con 0x5562a0d35e90 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f4518a57640 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d35e90 0x5562a0d33950 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f450c004ae0 tx=0x7f450c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb73a2de5bc7bd49 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f44f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f44f37fe640 1 -- 172.21.15.66:0/3487849850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c052060 con 0x5562a0d35e90 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f44f37fe640 1 -- 172.21.15.66:0/3487849850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f450c003b80 con 0x5562a0d35e90 2024-04-21T09:17:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4504005680 con 0x5562a0d35e90 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f44f37fe640 1 -- 172.21.15.66:0/3487849850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f450c039040 con 0x5562a0d35e90 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f451d187740 10 client.17985 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562a095f0f0 con 0x5562a0d35e90 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.639+0000 7f44f37fe640 1 -- 172.21.15.66:0/3487849850 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4504005680 con 0x5562a0d35e90 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f44f37fe640 1 -- 172.21.15.66:0/3487849850 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f450c042450 con 0x5562a0d35e90 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f451d187740 10 client.17985 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f451d187740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f451d187740 20 client.17985 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f451d187740 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0d45e90 0x5562a0d48280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f451d187740 4 client.17985 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562a0c9ac00 con 0x5562a0d45e90 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f4513fff640 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0d45e90 0x5562a0d48280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f44f37fe640 10 client.17985 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.640+0000 7f4513fff640 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0d45e90 0x5562a0d48280 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faabfa1276b1941c server_cookie=add471d5f5ac295b in_seq=0 out_seq=0 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f44f37fe640 1 -- 172.21.15.66:0/3487849850 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562a0c9ac00 con 0x5562a0d45e90 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f44f37fe640 10 client.17985 handle_command_reply: tid=0 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 client.17985 shutdown 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d35e90 msgr2=0x5562a0d33950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d35e90 0x5562a0d33950 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f450c004ae0 tx=0x7f450c02c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 shutdown_connections 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0d45e90 0x5562a0d48280 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d35e90 0x5562a0d33950 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d28bd0 0x5562a0d33410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f44f37fe640 0 client.17985 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 --2- 172.21.15.66:0/3487849850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d282b0 0x5562a0d32ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 >> 172.21.15.66:0/3487849850 conn(0x5562a0d22cb0 msgr2=0x5562a0d24a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 shutdown_connections 2024-04-21T09:17:27.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 1 -- 172.21.15.66:0/3487849850 wait complete. 2024-04-21T09:17:27.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.641+0000 7f451d187740 20 client.17985 trim_cache size 0 max 16384 2024-04-21T09:17:27.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 Processor -- start 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 -- start start 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d282b0 0x5562a0d35600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d35e90 0x5562a0d41a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45141afa20 con 0x5562a0d28bd0 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45141ad2f0 con 0x5562a0d35e90 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f451d187740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45141ab160 con 0x5562a0d282b0 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d23090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56860/0 (socket says 172.21.15.66:56860) 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 -- 172.21.15.66:0/3028665652 learned_addr learned my addr 172.21.15.66:0/3028665652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f4513fff640 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d282b0 0x5562a0d35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f4518a57640 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d35e90 0x5562a0d41a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 -- 172.21.15.66:0/3028665652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d282b0 msgr2=0x5562a0d35600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d282b0 0x5562a0d35600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 -- 172.21.15.66:0/3028665652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d35e90 msgr2=0x5562a0d41a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d35e90 0x5562a0d41a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 -- 172.21.15.66:0/3028665652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a0d27ee0 con 0x5562a0d28bd0 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f4513fff640 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d282b0 0x5562a0d35600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f4518a57640 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d35e90 0x5562a0d41a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.659+0000 7f45137fe640 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d23090 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f450800bc60 tx=0x7f4508012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac457bc35aa42e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f44f9ffb640 1 -- 172.21.15.66:0/3028665652 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450800bd50 con 0x5562a0d28bd0 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f44f9ffb640 1 -- 172.21.15.66:0/3028665652 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45080035f0 con 0x5562a0d28bd0 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f44f9ffb640 1 -- 172.21.15.66:0/3028665652 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450801c680 con 0x5562a0d28bd0 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 -- 172.21.15.66:0/3028665652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 msgr2=0x5562a0d23090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d23090 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f450800bc60 tx=0x7f4508012820 comp rx=0 tx=0).stop 2024-04-21T09:17:27.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 -- 172.21.15.66:0/3028665652 shutdown_connections 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d35e90 0x5562a0d41a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d28bd0 0x5562a0d23090 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 --2- 172.21.15.66:0/3028665652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d282b0 0x5562a0d35600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 -- 172.21.15.66:0/3028665652 >> 172.21.15.66:0/3028665652 conn(0x5562a0d22cb0 msgr2=0x5562a0d32e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 -- 172.21.15.66:0/3028665652 shutdown_connections 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.660+0000 7f451d187740 1 -- 172.21.15.66:0/3028665652 wait complete. 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 -- start start 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 Processor -- start 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d282b0 0x5562a0d3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d28bd0 0x5562a0d39820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 0x5562a0d39f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45141ab7b0 con 0x5562a0d282b0 2024-04-21T09:17:27.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45141acc20 con 0x5562a0d28bd0 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f45141abe80 con 0x5562a0d35e90 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f451d187740 10 client.0 _ll_register_callbacks cb 0x7f450c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d282b0 0x5562a0d3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d282b0 0x5562a0d3e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56876/0 (socket says 172.21.15.66:56876) 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 -- 172.21.15.66:0/3293540729 learned_addr learned my addr 172.21.15.66:0/3293540729 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f45137fe640 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d28bd0 0x5562a0d39820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 -- 172.21.15.66:0/3293540729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 msgr2=0x5562a0d39f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 0x5562a0d39f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 -- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d28bd0 msgr2=0x5562a0d39820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d28bd0 0x5562a0d39820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.661+0000 7f4513fff640 1 -- 172.21.15.66:0/3293540729 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a0dab400 con 0x5562a0d282b0 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f4513fff640 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d282b0 0x5562a0d3e4b0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f450409c9e0 tx=0x7f4504004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4111312c9b50263 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f44f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:27.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f44f37fe640 1 -- 172.21.15.66:0/3293540729 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504005180 con 0x5562a0d282b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f44f37fe640 1 -- 172.21.15.66:0/3293540729 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f45040046c0 con 0x5562a0d282b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45080035f0 con 0x5562a0d282b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f44f37fe640 1 -- 172.21.15.66:0/3293540729 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4504005680 con 0x5562a0d282b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 10 client.17991 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562a0dac550 con 0x5562a0d282b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f44f37fe640 1 -- 172.21.15.66:0/3293540729 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45080035f0 con 0x5562a0d282b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f44f37fe640 1 -- 172.21.15.66:0/3293540729 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4504099350 con 0x5562a0d282b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 10 client.17991 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 10 client.17991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 20 client.17991 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0dac8b0 0x5562a0daeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 4 client.17991 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562a0c94a70 con 0x5562a0dac8b0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.662+0000 7f45137fe640 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0dac8b0 0x5562a0daeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:27.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f45137fe640 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0dac8b0 0x5562a0daeca0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7fbeccb1901a0f server_cookie=fbc1bf9deb13120c in_seq=0 out_seq=0 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f44f37fe640 10 client.17991 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f44f37fe640 1 -- 172.21.15.66:0/3293540729 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5562a0c94a70 con 0x5562a0dac8b0 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f44f37fe640 10 client.17991 handle_command_reply: tid=0 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 client.17991 shutdown 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d282b0 msgr2=0x5562a0d3e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d282b0 0x5562a0d3e4b0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f450409c9e0 tx=0x7f4504004a90 comp rx=0 tx=0).stop 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 shutdown_connections 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5562a0dac8b0 0x5562a0daeca0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562a0d35e90 0x5562a0d39f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562a0d28bd0 0x5562a0d39820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f44f37fe640 0 client.17991 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 --2- 172.21.15.66:0/3293540729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562a0d282b0 0x5562a0d3e4b0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 >> 172.21.15.66:0/3293540729 conn(0x5562a0d22cb0 msgr2=0x5562a0d21d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 shutdown_connections 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 1 -- 172.21.15.66:0/3293540729 wait complete. 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.663+0000 7f451d187740 20 client.17991 trim_cache size 0 max 16384 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stdout: "e2003f76-0407-4649-aa7c-3271e1f51663": { 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "e2003f76-0407-4649-aa7c-3271e1f51663", 2024-04-21T09:17:27.670 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:17:27.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:17:27.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:17:27.671 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44fc0a3d20 msgr2=0x7f44fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44fc0a3d20 0x7f44fc0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f450c02c630 tx=0x7f450c0095c0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406a5b0 msgr2=0x7f4514198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406a5b0 0x7f4514198660 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f450400e9f0 tx=0x7f450400edf0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 shutdown_connections 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f44fc0a3d20 0x7f44fc0a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4514108060 0x7f451419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f451406af70 0x7f4514198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 --2- 172.21.15.66:0/1978341755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406a5b0 0x7f4514198660 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 >> 172.21.15.66:0/1978341755 conn(0x7f4514077c40 msgr2=0x7f45140ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 shutdown_connections 2024-04-21T09:17:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:27.664+0000 7f451a4ba640 1 -- 172.21.15.66:0/1978341755 wait complete. 2024-04-21T09:17:27.674 INFO:tasks.cephfs.filesystem:scrub status for tag:e2003f76-0407-4649-aa7c-3271e1f51663 - {'path': '/', 'tag': 'e2003f76-0407-4649-aa7c-3271e1f51663', 'options': 'recursive,force'} 2024-04-21T09:17:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:28 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:17:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:28 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:17:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:28 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:17:28.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:28 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:17:28.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:28 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:17:28.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:28 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:17:28.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:28 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:17:28.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:28 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:17:28.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:28 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:17:28.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:28 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:17:28.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:28 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:17:28.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:28 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:17:29.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:29 smithi186 ceph-mon[31505]: pgmap v287: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:29.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:29 smithi083 ceph-mon[31413]: pgmap v287: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:29.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:29 smithi066 ceph-mon[27953]: pgmap v287: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:31.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:31 smithi186 ceph-mon[31505]: pgmap v288: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:31.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:31 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:17:31.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:31 smithi083 ceph-mon[31413]: pgmap v288: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:31.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:31 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:17:31.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:31 smithi066 ceph-mon[27953]: pgmap v288: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:31.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:31 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:17:33.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:33 smithi186 ceph-mon[31505]: pgmap v289: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:33.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:33 smithi083 ceph-mon[31413]: pgmap v289: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:33.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:33 smithi066 ceph-mon[27953]: pgmap v289: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:35.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:35 smithi186 ceph-mon[31505]: pgmap v290: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:35.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:35 smithi083 ceph-mon[31413]: pgmap v290: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:35.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:35 smithi066 ceph-mon[27953]: pgmap v290: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:37 smithi186 ceph-mon[31505]: pgmap v291: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:37.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:37 smithi083 ceph-mon[31413]: pgmap v291: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:37.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:37 smithi066 ceph-mon[27953]: pgmap v291: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-04-21T09:17:39.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:39 smithi186 ceph-mon[31505]: pgmap v292: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:39 smithi083 ceph-mon[31413]: pgmap v292: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:39.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:39 smithi066 ceph-mon[27953]: pgmap v292: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-21T09:17:41.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:40 smithi186 ceph-mon[31505]: pgmap v293: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:41.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:40 smithi083 ceph-mon[31413]: pgmap v293: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:41.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:40 smithi066 ceph-mon[27953]: pgmap v293: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:43.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:43 smithi186 ceph-mon[31505]: pgmap v294: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:43.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:43 smithi083 ceph-mon[31413]: pgmap v294: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:43.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:43 smithi066 ceph-mon[27953]: pgmap v294: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:45.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:45 smithi186 ceph-mon[31505]: pgmap v295: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:45.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:45 smithi083 ceph-mon[31413]: pgmap v295: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:45.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:45 smithi066 ceph-mon[27953]: pgmap v295: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:47 smithi186 ceph-mon[31505]: pgmap v296: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:47.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:47 smithi083 ceph-mon[31413]: pgmap v296: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:47.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:47 smithi066 ceph-mon[27953]: pgmap v296: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:49.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:49 smithi186 ceph-mon[31505]: pgmap v297: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:49.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:49 smithi083 ceph-mon[31413]: pgmap v297: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:49 smithi066 ceph-mon[27953]: pgmap v297: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:51 smithi186 ceph-mon[31505]: pgmap v298: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:51.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:51 smithi083 ceph-mon[31413]: pgmap v298: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:51.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:51 smithi066 ceph-mon[27953]: pgmap v298: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:53 smithi186 ceph-mon[31505]: pgmap v299: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:53.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:53 smithi083 ceph-mon[31413]: pgmap v299: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:53.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:53 smithi066 ceph-mon[27953]: pgmap v299: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:55.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:55 smithi186 ceph-mon[31505]: pgmap v300: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:55 smithi083 ceph-mon[31413]: pgmap v300: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:55.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:55 smithi066 ceph-mon[27953]: pgmap v300: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:57.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:57 smithi186 ceph-mon[31505]: pgmap v301: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:57.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:57 smithi083 ceph-mon[31413]: pgmap v301: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:57 smithi066 ceph-mon[27953]: pgmap v301: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:17:57.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:17:57.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:17:57.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 -- 172.21.15.66:0/3987463812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe784102b50 msgr2=0x7fe784106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 --2- 172.21.15.66:0/3987463812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe784102b50 0x7fe784106b90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe778009900 tx=0x7fe77802bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 -- 172.21.15.66:0/3987463812 shutdown_connections 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 --2- 172.21.15.66:0/3987463812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe784107270 0x7fe78410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 --2- 172.21.15.66:0/3987463812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe784102b50 0x7fe784106b90 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 --2- 172.21.15.66:0/3987463812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7841021a0 0x7fe784102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 -- 172.21.15.66:0/3987463812 >> 172.21.15.66:0/3987463812 conn(0x7fe7840fda40 msgr2=0x7fe7840ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 -- 172.21.15.66:0/3987463812 shutdown_connections 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.792+0000 7fe78a1db640 1 -- 172.21.15.66:0/3987463812 wait complete. 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 Processor -- start 2024-04-21T09:17:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 -- start start 2024-04-21T09:17:57.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7841021a0 0x7fe7841987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe784107270 0x7fe784198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78419d020 0x7fe78419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe784112190 con 0x7fe78419d020 2024-04-21T09:17:57.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe784112010 con 0x7fe784107270 2024-04-21T09:17:57.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.793+0000 7fe78a1db640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe784112310 con 0x7fe7841021a0 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78419d020 0x7fe78419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78419d020 0x7fe78419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41904/0 (socket says 172.21.15.66:41904) 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe784107270 0x7fe784198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 -- 172.21.15.66:0/2925845165 learned_addr learned my addr 172.21.15.66:0/2925845165 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe7837fe640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7841021a0 0x7fe7841987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 -- 172.21.15.66:0/2925845165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7841021a0 msgr2=0x7fe7841987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7841021a0 0x7fe7841987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 -- 172.21.15.66:0/2925845165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe784107270 msgr2=0x7fe784198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe784107270 0x7fe784198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 -- 172.21.15.66:0/2925845165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78419db10 con 0x7fe78419d020 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe782ffd640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe784107270 0x7fe784198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe783fff640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78419d020 0x7fe78419d4d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00cce0 tx=0x7fe77c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe780ff9640 1 -- 172.21.15.66:0/2925845165 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00f450 con 0x7fe78419d020 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe780ff9640 1 -- 172.21.15.66:0/2925845165 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c00faa0 con 0x7fe78419d020 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78419de00 con 0x7fe78419d020 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe780ff9640 1 -- 172.21.15.66:0/2925845165 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c018b70 con 0x7fe78419d020 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.794+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841a55c0 con 0x7fe78419d020 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe780ff9640 1 -- 172.21.15.66:0/2925845165 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe77c018d10 con 0x7fe78419d020 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 Processor -- start 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 -- start start 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e59300f1e0 con 0x55e5930b3a80 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e592fac390 con 0x55e5930aaea0 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e59300b1a0 con 0x55e5930aa580 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe780ff9640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe76c0a3da0 0x7fe76c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41906/0 (socket says 172.21.15.66:41906) 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe782ffd640 1 -- 172.21.15.66:0/3297696346 learned_addr learned my addr 172.21.15.66:0/3297696346 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe7837fe640 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe783fff640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe76c0a3da0 0x7fe76c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.797+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe780ff9640 1 -- 172.21.15.66:0/2925845165 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe77c0d0f40 con 0x7fe78419d020 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe782ffd640 1 -- 172.21.15.66:0/3297696346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 msgr2=0x55e5930aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe782ffd640 1 -- 172.21.15.66:0/3297696346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 msgr2=0x55e5930b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe782ffd640 1 -- 172.21.15.66:0/3297696346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e592deb7d0 con 0x55e5930b3a80 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930b5e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe778005ec0 tx=0x7fe778009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b2c29f0d455ace server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe783fff640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe76c0a3da0 0x7fe76c0a6260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0ce370 tx=0x7fe77c018610 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe769ffb640 1 -- 172.21.15.66:0/3297696346 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780044b0 con 0x55e5930b3a80 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe769ffb640 1 -- 172.21.15.66:0/3297696346 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77802cc90 con 0x55e5930b3a80 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.798+0000 7fe769ffb640 1 -- 172.21.15.66:0/3297696346 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778042470 con 0x55e5930b3a80 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3297696346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 msgr2=0x55e5930b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930b5e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe778005ec0 tx=0x7fe778009560 comp rx=0 tx=0).stop 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3297696346 shutdown_connections 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930b5e70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3297696346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3297696346 >> 172.21.15.66:0/3297696346 conn(0x55e5930a4cb0 msgr2=0x55e5930a5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3297696346 shutdown_connections 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3297696346 wait complete. 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.799+0000 7fe78cdc5740 1 -- start start 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 1 Processor -- start 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930b3a80 0x55e5930b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e5930b6810 con 0x55e5930aa580 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e592fc82f0 con 0x55e5930aaea0 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e5930b6990 con 0x55e5930b3a80 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe78cdc5740 10 client.0 _ll_register_callbacks cb 0x55e592fd5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41916/0 (socket says 172.21.15.66:41916) 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 -- 172.21.15.66:0/3766466872 learned_addr learned my addr 172.21.15.66:0/3766466872 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe7837fe640 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930b3a80 0x55e5930b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 -- 172.21.15.66:0/3766466872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930b3a80 msgr2=0x55e5930b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930b3a80 0x55e5930b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 -- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 msgr2=0x55e5930b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 -- 172.21.15.66:0/3766466872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e592d213f0 con 0x55e5930aa580 2024-04-21T09:17:57.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe782ffd640 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930b0ae0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe77802bc70 tx=0x7fe778009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dbb8ba550e4351 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.800+0000 7fe762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe762ffd640 1 -- 172.21.15.66:0/3766466872 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778057060 con 0x55e5930aa580 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe762ffd640 1 -- 172.21.15.66:0/3766466872 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe7780037f0 con 0x55e5930aa580 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77802cc90 con 0x55e5930aa580 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe762ffd640 1 -- 172.21.15.66:0/3766466872 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe77805e050 con 0x55e5930aa580 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 10 client.18003 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e592ce10f0 con 0x55e5930aa580 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe762ffd640 1 -- 172.21.15.66:0/3766466872 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77802cc90 con 0x55e5930aa580 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe762ffd640 1 -- 172.21.15.66:0/3766466872 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe778042c80 con 0x55e5930aa580 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 10 client.18003 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 20 client.18003 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5930c38d0 0x55e5930c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 4 client.18003 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.801+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e59301cc00 con 0x55e5930c38d0 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.802+0000 7fe7837fe640 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5930c38d0 0x55e5930c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.802+0000 7fe762ffd640 10 client.18003 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.802+0000 7fe7837fe640 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5930c38d0 0x55e5930c5cc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df5752d03eaa8dc server_cookie=e0185e43bf4b8203 in_seq=0 out_seq=0 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe762ffd640 1 -- 172.21.15.66:0/3766466872 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e59301cc00 con 0x55e5930c38d0 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe762ffd640 10 client.18003 handle_command_reply: tid=0 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 client.18003 shutdown 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 msgr2=0x55e5930b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930b0ae0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe77802bc70 tx=0x7fe778009230 comp rx=0 tx=0).stop 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 shutdown_connections 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5930c38d0 0x55e5930c5cc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930b3a80 0x55e5930b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe762ffd640 0 client.18003 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:57.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/3766466872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930b0ae0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 >> 172.21.15.66:0/3766466872 conn(0x55e5930a4cb0 msgr2=0x55e5930a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:57.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 shutdown_connections 2024-04-21T09:17:57.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 1 -- 172.21.15.66:0/3766466872 wait complete. 2024-04-21T09:17:57.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.803+0000 7fe78cdc5740 20 client.18003 trim_cache size 0 max 16384 2024-04-21T09:17:57.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 Processor -- start 2024-04-21T09:17:57.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 -- start start 2024-04-21T09:17:57.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841b1af0 con 0x55e5930b3a80 2024-04-21T09:17:57.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841af3c0 con 0x55e5930aaea0 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841ad230 con 0x55e5930aa580 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41932/0 (socket says 172.21.15.66:41932) 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 -- 172.21.15.66:0/43806826 learned_addr learned my addr 172.21.15.66:0/43806826 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe7837fe640 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 -- 172.21.15.66:0/43806826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 msgr2=0x55e5930b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 -- 172.21.15.66:0/43806826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 msgr2=0x55e5930bf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.821+0000 7fe782ffd640 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930bf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe782ffd640 1 -- 172.21.15.66:0/43806826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5930aacd0 con 0x55e5930b3a80 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe782ffd640 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930af990 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe7780095a0 tx=0x7fe7780041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bc0840f6966c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe7697fa640 1 -- 172.21.15.66:0/43806826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778003e50 con 0x55e5930b3a80 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe7697fa640 1 -- 172.21.15.66:0/43806826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77802caf0 con 0x55e5930b3a80 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe7697fa640 1 -- 172.21.15.66:0/43806826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778003b40 con 0x55e5930b3a80 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 -- 172.21.15.66:0/43806826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 msgr2=0x55e5930af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930af990 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe7780095a0 tx=0x7fe7780041b0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 -- 172.21.15.66:0/43806826 shutdown_connections 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930b3a80 0x55e5930af990 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930aaea0 0x55e5930bf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/43806826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aa580 0x55e5930b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 -- 172.21.15.66:0/43806826 >> 172.21.15.66:0/43806826 conn(0x55e5930a4cb0 msgr2=0x55e5930af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.822+0000 7fe78cdc5740 1 -- 172.21.15.66:0/43806826 shutdown_connections 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 -- 172.21.15.66:0/43806826 wait complete. 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 -- start start 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 Processor -- start 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aaea0 0x55e5930c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930b3a80 0x55e5930c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841ad880 con 0x55e5930aa580 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841aecf0 con 0x55e5930b3a80 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841adf50 con 0x55e5930aaea0 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe78cdc5740 10 client.0 _ll_register_callbacks cb 0x7fe774009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe782ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41942/0 (socket says 172.21.15.66:41942) 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.823+0000 7fe782ffd640 1 -- 172.21.15.66:0/2878572616 learned_addr learned my addr 172.21.15.66:0/2878572616 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe782ffd640 1 -- 172.21.15.66:0/2878572616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aaea0 msgr2=0x55e5930c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe7837fe640 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aaea0 0x55e5930c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe782ffd640 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aaea0 0x55e5930c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe782ffd640 1 -- 172.21.15.66:0/2878572616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930b3a80 msgr2=0x55e5930c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe782ffd640 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930b3a80 0x55e5930c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe782ffd640 1 -- 172.21.15.66:0/2878572616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5930c6f40 con 0x55e5930aa580 2024-04-21T09:17:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe7837fe640 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aaea0 0x55e5930c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe782ffd640 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930c8f80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe778034700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd455d8234e99e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe762ffd640 1 -- 172.21.15.66:0/2878572616 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778034c00 con 0x55e5930aa580 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe762ffd640 1 -- 172.21.15.66:0/2878572616 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe778034da0 con 0x55e5930aa580 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77802caf0 con 0x55e5930aa580 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe762ffd640 1 -- 172.21.15.66:0/2878572616 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe7780344e0 con 0x55e5930aa580 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe78cdc5740 10 client.18009 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e593129980 con 0x55e5930aa580 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.824+0000 7fe762ffd640 1 -- 172.21.15.66:0/2878572616 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77802caf0 con 0x55e5930aa580 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe762ffd640 1 -- 172.21.15.66:0/2878572616 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe778042770 con 0x55e5930aa580 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe78cdc5740 10 client.18009 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe78cdc5740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe78cdc5740 20 client.18009 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e593129df0 0x55e59312c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe78cdc5740 4 client.18009 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e593016a70 con 0x55e593129df0 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe7837fe640 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e593129df0 0x55e59312c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe7837fe640 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e593129df0 0x55e59312c1e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4adeaa569588d3e6 server_cookie=1f230cc36faa1c9a in_seq=0 out_seq=0 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.825+0000 7fe762ffd640 10 client.18009 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe762ffd640 1 -- 172.21.15.66:0/2878572616 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e593016a70 con 0x55e593129df0 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe762ffd640 10 client.18009 handle_command_reply: tid=0 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 client.18009 shutdown 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 msgr2=0x55e5930c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930c8f80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe778034700 comp rx=0 tx=0).stop 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 shutdown_connections 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e593129df0 0x55e59312c1e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5930b3a80 0x55e5930c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5930aaea0 0x55e5930c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe762ffd640 0 client.18009 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 --2- 172.21.15.66:0/2878572616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5930aa580 0x55e5930c8f80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 >> 172.21.15.66:0/2878572616 conn(0x55e5930a4cb0 msgr2=0x55e5930b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 shutdown_connections 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 1 -- 172.21.15.66:0/2878572616 wait complete. 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78cdc5740 20 client.18009 trim_cache size 0 max 16384 2024-04-21T09:17:57.836 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:17:57.837 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:17:57.838 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:17:57.838 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe76c0a3da0 msgr2=0x7fe76c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78a1db640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe76c0a3da0 0x7fe76c0a6260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0ce370 tx=0x7fe77c018610 comp rx=0 tx=0).stop 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78419d020 msgr2=0x7fe78419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.826+0000 7fe78a1db640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78419d020 0x7fe78419d4d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00cce0 tx=0x7fe77c00a620 comp rx=0 tx=0).stop 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 shutdown_connections 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fe76c0a3da0 0x7fe76c0a6260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78419d020 0x7fe78419d4d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe784107270 0x7fe784198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 --2- 172.21.15.66:0/2925845165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7841021a0 0x7fe7841987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 >> 172.21.15.66:0/2925845165 conn(0x7fe7840fda40 msgr2=0x7fe7840fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 shutdown_connections 2024-04-21T09:17:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.827+0000 7fe78a1db640 1 -- 172.21.15.66:0/2925845165 wait complete. 2024-04-21T09:17:57.842 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:17:57.842 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 -- 172.21.15.66:0/4024868973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc101b60 msgr2=0x7f51cc10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/4024868973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc101b60 0x7f51cc10dc90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009cd0 tx=0x7f51c402c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 -- 172.21.15.66:0/4024868973 shutdown_connections 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/4024868973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc101b60 0x7f51cc10dc90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/4024868973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51cc1011a0 0x7f51cc101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/4024868973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51cc105e30 0x7f51cc106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 -- 172.21.15.66:0/4024868973 >> 172.21.15.66:0/4024868973 conn(0x7f51cc077e30 msgr2=0x7f51cc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 -- 172.21.15.66:0/4024868973 shutdown_connections 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 -- 172.21.15.66:0/4024868973 wait complete. 2024-04-21T09:17:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 Processor -- start 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.945+0000 7f51d2ee0640 1 -- start start 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d2ee0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51cc1011a0 0x7f51cc1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d2ee0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51cc101b60 0x7f51cc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d2ee0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc105e30 0x7f51cc19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d2ee0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc1110c0 con 0x7f51cc105e30 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d2ee0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc110f40 con 0x7f51cc101b60 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d2ee0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc111240 con 0x7f51cc1011a0 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc105e30 0x7f51cc19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc105e30 0x7f51cc19be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41968/0 (socket says 172.21.15.66:41968) 2024-04-21T09:17:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 -- 172.21.15.66:0/3870576554 learned_addr learned my addr 172.21.15.66:0/3870576554 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51cbfff640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51cc101b60 0x7f51cc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d0c7c640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51cc1011a0 0x7f51cc1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 -- 172.21.15.66:0/3870576554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51cc1011a0 msgr2=0x7f51cc1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51cc1011a0 0x7f51cc1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 -- 172.21.15.66:0/3870576554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51cc101b60 msgr2=0x7f51cc197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51cc101b60 0x7f51cc197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 -- 172.21.15.66:0/3870576554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc19c5b0 con 0x7f51cc105e30 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.946+0000 7f51d147d640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc105e30 0x7f51cc19be80 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009ca0 tx=0x7f51c4032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.947+0000 7f51c9ffb640 1 -- 172.21.15.66:0/3870576554 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4034640 con 0x7f51cc105e30 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.947+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc19c840 con 0x7f51cc105e30 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.947+0000 7f51c9ffb640 1 -- 172.21.15.66:0/3870576554 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c40324c0 con 0x7f51cc105e30 2024-04-21T09:17:57.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.947+0000 7f51c9ffb640 1 -- 172.21.15.66:0/3870576554 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4004040 con 0x7f51cc105e30 2024-04-21T09:17:57.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.947+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc0690e0 con 0x7f51cc105e30 2024-04-21T09:17:57.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.948+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc102570 con 0x7f51cc105e30 2024-04-21T09:17:57.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.949+0000 7f51c9ffb640 1 -- 172.21.15.66:0/3870576554 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f51c40348f0 con 0x7f51cc105e30 2024-04-21T09:17:57.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.949+0000 7f51c9ffb640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51b80a3d20 0x7f51b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:57.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.950+0000 7f51c9ffb640 1 -- 172.21.15.66:0/3870576554 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f51c40f4660 con 0x7f51cc105e30 2024-04-21T09:17:57.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.950+0000 7f51d0c7c640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51b80a3d20 0x7f51b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:57.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.950+0000 7f51d0c7c640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51b80a3d20 0x7f51b80a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005d00 tx=0x7f51bc00b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:57.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:57.953+0000 7f51c9ffb640 1 -- 172.21.15.66:0/3870576554 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51c40c0a40 con 0x7f51cc105e30 2024-04-21T09:17:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.179+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51cc069b00 con 0x7f51cc105e30 2024-04-21T09:17:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.180+0000 7f51c9ffb640 1 -- 172.21.15.66:0/3870576554 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f51c40c0340 con 0x7f51cc105e30 2024-04-21T09:17:58.179 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:17:58.179 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:17:58.180 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:17:58.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51b80a3d20 msgr2=0x7f51b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51b80a3d20 0x7f51b80a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005d00 tx=0x7f51bc00b410 comp rx=0 tx=0).stop 2024-04-21T09:17:58.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc105e30 msgr2=0x7f51cc19be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc105e30 0x7f51cc19be80 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009ca0 tx=0x7f51c4032e40 comp rx=0 tx=0).stop 2024-04-21T09:17:58.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 shutdown_connections 2024-04-21T09:17:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f51b80a3d20 0x7f51b80a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51cc105e30 0x7f51cc19be80 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51cc101b60 0x7f51cc197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 --2- 172.21.15.66:0/3870576554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51cc1011a0 0x7f51cc1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 >> 172.21.15.66:0/3870576554 conn(0x7f51cc077e30 msgr2=0x7f51cc0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 shutdown_connections 2024-04-21T09:17:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.182+0000 7f51d2ee0640 1 -- 172.21.15.66:0/3870576554 wait complete. 2024-04-21T09:17:58.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:17:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 -- 172.21.15.66:0/2001106039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 msgr2=0x7f048010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 --2- 172.21.15.66:0/2001106039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f048010bfd0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0470009900 tx=0x7f047002bd90 comp rx=0 tx=0).stop 2024-04-21T09:17:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 -- 172.21.15.66:0/2001106039 shutdown_connections 2024-04-21T09:17:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 --2- 172.21.15.66:0/2001106039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f048010c510 0x7f048010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 --2- 172.21.15.66:0/2001106039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f048010bfd0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 --2- 172.21.15.66:0/2001106039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04801031d0 0x7f04801035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 -- 172.21.15.66:0/2001106039 >> 172.21.15.66:0/2001106039 conn(0x7f0480077f50 msgr2=0x7f0480078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 -- 172.21.15.66:0/2001106039 shutdown_connections 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 -- 172.21.15.66:0/2001106039 wait complete. 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.299+0000 7f0487e43640 1 Processor -- start 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0487e43640 1 -- start start 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0487e43640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04801031d0 0x7f04801a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0487e43640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f04801a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0487e43640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f048010c510 0x7f048019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0487e43640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0480101360 con 0x7f0480103af0 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0487e43640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04801011e0 con 0x7f04801031d0 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0487e43640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04801014e0 con 0x7f048010c510 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f04801a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f04801a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41972/0 (socket says 172.21.15.66:41972) 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 -- 172.21.15.66:0/2315429244 learned_addr learned my addr 172.21.15.66:0/2315429244 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f0485bdf640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04801031d0 0x7f04801a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 -- 172.21.15.66:0/2315429244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f048010c510 msgr2=0x7f048019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f048010c510 0x7f048019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 -- 172.21.15.66:0/2315429244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04801031d0 msgr2=0x7f04801a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04801031d0 0x7f04801a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 -- 172.21.15.66:0/2315429244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048019b530 con 0x7f0480103af0 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.300+0000 7f04853de640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f04801a0c70 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0470005ec0 tx=0x7f047002d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.301+0000 7f0476ffd640 1 -- 172.21.15.66:0/2315429244 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470043020 con 0x7f0480103af0 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.301+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048019b7c0 con 0x7f0480103af0 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.301+0000 7f0476ffd640 1 -- 172.21.15.66:0/2315429244 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047002ddd0 con 0x7f0480103af0 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.301+0000 7f0476ffd640 1 -- 172.21.15.66:0/2315429244 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470003bc0 con 0x7f0480103af0 2024-04-21T09:17:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.301+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04801a75f0 con 0x7f0480103af0 2024-04-21T09:17:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.303+0000 7f0476ffd640 1 -- 172.21.15.66:0/2315429244 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0470046020 con 0x7f0480103af0 2024-04-21T09:17:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.303+0000 7f048aa34740 1 Processor -- start 2024-04-21T09:17:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f048aa34740 1 -- start start 2024-04-21T09:17:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f048aa34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf195d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f048aa34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1966430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f048aa34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf1968e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f048aa34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558bf18c21f0 con 0x558bf195d4d0 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f048aa34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558bf185f4b0 con 0x558bf195de80 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f048aa34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558bf18be1b0 con 0x558bf1966a00 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f0476ffd640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f046c0a3d20 0x7f046c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf195d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf195d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41982/0 (socket says 172.21.15.66:41982) 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 -- 172.21.15.66:0/3505224121 learned_addr learned my addr 172.21.15.66:0/3505224121 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04853de640 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf1968e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f046c0a3d20 0x7f046c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f0485bdf640 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1966430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f0476ffd640 1 -- 172.21.15.66:0/2315429244 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f04700f2ec0 con 0x7f0480103af0 2024-04-21T09:17:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 -- 172.21.15.66:0/3505224121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 msgr2=0x558bf1968e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf1968e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 -- 172.21.15.66:0/3505224121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 msgr2=0x558bf1966430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1966430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.304+0000 7f04863e0640 1 -- 172.21.15.66:0/3505224121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf18511b0 con 0x558bf195d4d0 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.305+0000 7f04863e0640 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf195d8b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0478012780 tx=0x7f0478012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbc8173aebc3819 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.305+0000 7f04863e0640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f046c0a3d20 0x7f046c0a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04780145b0 tx=0x7f0478019000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.305+0000 7f046bfff640 1 -- 172.21.15.66:0/3505224121 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478014c50 con 0x558bf195d4d0 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.305+0000 7f046bfff640 1 -- 172.21.15.66:0/3505224121 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047800b8f0 con 0x558bf195d4d0 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.305+0000 7f046bfff640 1 -- 172.21.15.66:0/3505224121 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478006090 con 0x558bf195d4d0 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 -- 172.21.15.66:0/3505224121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 msgr2=0x558bf195d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf195d8b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0478012780 tx=0x7f0478012c40 comp rx=0 tx=0).stop 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 -- 172.21.15.66:0/3505224121 shutdown_connections 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf1968e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1966430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 --2- 172.21.15.66:0/3505224121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf195d8b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 -- 172.21.15.66:0/3505224121 >> 172.21.15.66:0/3505224121 conn(0x558bf1957ba0 msgr2=0x558bf1957f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 -- 172.21.15.66:0/3505224121 shutdown_connections 2024-04-21T09:17:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 -- 172.21.15.66:0/3505224121 wait complete. 2024-04-21T09:17:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 -- start start 2024-04-21T09:17:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.306+0000 7f048aa34740 1 Processor -- start 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1966310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf195de80 0x558bf19636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf1966a00 0x558bf1963c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558bf1969910 con 0x558bf195d4d0 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558bf187b500 con 0x558bf1966a00 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558bf1969a90 con 0x558bf195de80 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 10 client.0 _ll_register_callbacks cb 0x558bf1849b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1966310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1966310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41994/0 (socket says 172.21.15.66:41994) 2024-04-21T09:17:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 -- 172.21.15.66:0/2677360783 learned_addr learned my addr 172.21.15.66:0/2677360783 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f04863e0640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf1966a00 0x558bf1963c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f04853de640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf195de80 0x558bf19636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 -- 172.21.15.66:0/2677360783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf195de80 msgr2=0x558bf19636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf195de80 0x558bf19636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 -- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf1966a00 msgr2=0x558bf1963c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf1966a00 0x558bf1963c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 -- 172.21.15.66:0/2677360783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bf15d43f0 con 0x558bf195d4d0 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f04853de640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf195de80 0x558bf19636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f0485bdf640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1966310 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f047c010500 tx=0x7f047c0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4879e21dd1f1c5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f04697fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f04697fa640 1 -- 172.21.15.66:0/2677360783 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c01a020 con 0x558bf195d4d0 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f04697fa640 1 -- 172.21.15.66:0/2677360783 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f047c016750 con 0x558bf195d4d0 2024-04-21T09:17:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.307+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047800b8f0 con 0x558bf195d4d0 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f04697fa640 1 -- 172.21.15.66:0/2677360783 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f047c025050 con 0x558bf195d4d0 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 10 client.18027 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558bf169e7d0 con 0x558bf195d4d0 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f04697fa640 1 -- 172.21.15.66:0/2677360783 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047800b8f0 con 0x558bf195d4d0 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f04697fa640 1 -- 172.21.15.66:0/2677360783 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f047c014a50 con 0x558bf195d4d0 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 10 client.18027 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 10 client.18027 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 20 client.18027 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf1976c90 0x558bf19790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 4 client.18027 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bf1861930 con 0x558bf1976c90 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.308+0000 7f04853de640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf1976c90 0x558bf19790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.309+0000 7f04697fa640 10 client.18027 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.309+0000 7f04853de640 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf1976c90 0x558bf19790c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76554e8f5bdde59b server_cookie=8740c389bcca3886 in_seq=0 out_seq=0 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f04697fa640 1 -- 172.21.15.66:0/2677360783 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bf1861930 con 0x558bf1976c90 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f04697fa640 10 client.18027 handle_command_reply: tid=0 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 client.18027 shutdown 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 msgr2=0x558bf1966310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1966310 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f047c010500 tx=0x7f047c0109c0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 shutdown_connections 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf1976c90 0x558bf19790c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf1966a00 0x558bf1963c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f04697fa640 0 client.18027 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf195de80 0x558bf19636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 --2- 172.21.15.66:0/2677360783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1966310 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 >> 172.21.15.66:0/2677360783 conn(0x558bf1957ba0 msgr2=0x558bf19598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 shutdown_connections 2024-04-21T09:17:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 1 -- 172.21.15.66:0/2677360783 wait complete. 2024-04-21T09:17:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.310+0000 7f048aa34740 20 client.18027 trim_cache size 0 max 16384 2024-04-21T09:17:58.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 Processor -- start 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 -- start start 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1972800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1956a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf1965dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b3ef0 con 0x558bf195d4d0 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b17c0 con 0x558bf195de80 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f048aa34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04801af630 con 0x558bf1966a00 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f0485bdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1972800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f0485bdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1972800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42000/0 (socket says 172.21.15.66:42000) 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f0485bdf640 1 -- 172.21.15.66:0/3874587652 learned_addr learned my addr 172.21.15.66:0/3874587652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.323+0000 7f04853de640 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1956a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f0485bdf640 1 -- 172.21.15.66:0/3874587652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 msgr2=0x558bf1965dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f0485bdf640 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf1965dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f0485bdf640 1 -- 172.21.15.66:0/3874587652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 msgr2=0x558bf1956a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f0485bdf640 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1956a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f0485bdf640 1 -- 172.21.15.66:0/3874587652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf1962650 con 0x558bf195d4d0 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f0485bdf640 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1972800 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f047c00b2d0 tx=0x7f047c004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe428a5d8c0df01e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f046b7fe640 1 -- 172.21.15.66:0/3874587652 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c01a050 con 0x558bf195d4d0 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f046b7fe640 1 -- 172.21.15.66:0/3874587652 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c016c00 con 0x558bf195d4d0 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f046b7fe640 1 -- 172.21.15.66:0/3874587652 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c014a60 con 0x558bf195d4d0 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f048aa34740 1 -- 172.21.15.66:0/3874587652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 msgr2=0x558bf1972800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f048aa34740 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1972800 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f047c00b2d0 tx=0x7f047c004640 comp rx=0 tx=0).stop 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f048aa34740 1 -- 172.21.15.66:0/3874587652 shutdown_connections 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f048aa34740 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf1965dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f048aa34740 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf1956a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.324+0000 7f048aa34740 1 --2- 172.21.15.66:0/3874587652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf1972800 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 -- 172.21.15.66:0/3874587652 >> 172.21.15.66:0/3874587652 conn(0x558bf1957ba0 msgr2=0x558bf1956370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 -- 172.21.15.66:0/3874587652 shutdown_connections 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 -- 172.21.15.66:0/3874587652 wait complete. 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 -- start start 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 Processor -- start 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf196a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf196af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf196b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04801afc80 con 0x558bf195d4d0 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b10f0 con 0x558bf195de80 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b0350 con 0x558bf1966a00 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f048aa34740 10 client.0 _ll_register_callbacks cb 0x7f047c00bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f0485bdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf196a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f0485bdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf196a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42014/0 (socket says 172.21.15.66:42014) 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.325+0000 7f0485bdf640 1 -- 172.21.15.66:0/2011318327 learned_addr learned my addr 172.21.15.66:0/2011318327 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04853de640 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf196af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04863e0640 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf196b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f0485bdf640 1 -- 172.21.15.66:0/2011318327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 msgr2=0x558bf196b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f0485bdf640 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf196b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f0485bdf640 1 -- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 msgr2=0x558bf196af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f0485bdf640 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf196af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f0485bdf640 1 -- 172.21.15.66:0/2011318327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bf195dc20 con 0x558bf195d4d0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f0485bdf640 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf196a9d0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f047c014f60 tx=0x7f047c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ffa19ebf99b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04697fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04697fa640 1 -- 172.21.15.66:0/2011318327 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c037060 con 0x558bf195d4d0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04697fa640 1 -- 172.21.15.66:0/2011318327 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f047c0059c0 con 0x558bf195d4d0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047c016c00 con 0x558bf195d4d0 2024-04-21T09:17:58.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04697fa640 1 -- 172.21.15.66:0/2011318327 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f047c03e050 con 0x558bf195d4d0 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f048aa34740 10 client.18033 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558bf19dd5d0 con 0x558bf195d4d0 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04697fa640 1 -- 172.21.15.66:0/2011318327 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c016c00 con 0x558bf195d4d0 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.326+0000 7f04697fa640 1 -- 172.21.15.66:0/2011318327 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f047c0061f0 con 0x558bf195d4d0 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f048aa34740 10 client.18033 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f048aa34740 10 client.18033 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f048aa34740 20 client.18033 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f048aa34740 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf19ddc30 0x558bf19e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f048aa34740 4 client.18033 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bf18cfc10 con 0x558bf19ddc30 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f04853de640 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf19ddc30 0x558bf19e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f04697fa640 10 client.18033 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f04853de640 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf19ddc30 0x558bf19e0020 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80e2b57dc469fe86 server_cookie=b9462665f3f016b8 in_seq=0 out_seq=0 2024-04-21T09:17:58.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f04697fa640 1 -- 172.21.15.66:0/2011318327 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bf18cfc10 con 0x558bf19ddc30 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f04697fa640 10 client.18033 handle_command_reply: tid=0 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.327+0000 7f048aa34740 1 client.18033 shutdown 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 msgr2=0x558bf196a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf196a9d0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f047c014f60 tx=0x7f047c00ba40 comp rx=0 tx=0).stop 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 shutdown_connections 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x558bf19ddc30 0x558bf19e0020 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bf1966a00 0x558bf196b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f04697fa640 0 client.18033 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bf195de80 0x558bf196af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 --2- 172.21.15.66:0/2011318327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bf195d4d0 0x558bf196a9d0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 >> 172.21.15.66:0/2011318327 conn(0x558bf1957ba0 msgr2=0x558bf1959800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 shutdown_connections 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 1 -- 172.21.15.66:0/2011318327 wait complete. 2024-04-21T09:17:58.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f048aa34740 20 client.18033 trim_cache size 0 max 16384 2024-04-21T09:17:58.336 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f046c0a3d20 msgr2=0x7f046c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f046c0a3d20 0x7f046c0a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04780145b0 tx=0x7f0478019000 comp rx=0 tx=0).stop 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 msgr2=0x7f04801a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f04801a0c70 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0470005ec0 tx=0x7f047002d9d0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 shutdown_connections 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f046c0a3d20 0x7f046c0a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f048010c510 0x7f048019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0480103af0 0x7f04801a0c70 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 --2- 172.21.15.66:0/2315429244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04801031d0 0x7f04801a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 >> 172.21.15.66:0/2315429244 conn(0x7f0480077f50 msgr2=0x7f048010d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 shutdown_connections 2024-04-21T09:17:58.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.328+0000 7f0487e43640 1 -- 172.21.15.66:0/2315429244 wait complete. 2024-04-21T09:17:58.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:17:58.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 -- 172.21.15.66:0/106130504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e410c3d0 msgr2=0x7f29e410e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 --2- 172.21.15.66:0/106130504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e410c3d0 0x7f29e410e870 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009900 tx=0x7f29e002bcd0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 -- 172.21.15.66:0/106130504 shutdown_connections 2024-04-21T09:17:58.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 --2- 172.21.15.66:0/106130504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e410c3d0 0x7f29e410e870 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 --2- 172.21.15.66:0/106130504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29e41038d0 0x7f29e410bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 --2- 172.21.15.66:0/106130504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e4102fb0 0x7f29e4103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 -- 172.21.15.66:0/106130504 >> 172.21.15.66:0/106130504 conn(0x7f29e40fc7d0 msgr2=0x7f29e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 -- 172.21.15.66:0/106130504 shutdown_connections 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 -- 172.21.15.66:0/106130504 wait complete. 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 Processor -- start 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.443+0000 7f29ea99d640 1 -- start start 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e4102fb0 0x7f29e4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e41038d0 0x7f29e4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29e410c3d0 0x7f29e419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4112060 con 0x7f29e41038d0 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4111ee0 con 0x7f29e4102fb0 2024-04-21T09:17:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41121e0 con 0x7f29e410c3d0 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e41038d0 0x7f29e4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e41038d0 0x7f29e4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42032/0 (socket says 172.21.15.66:42032) 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 -- 172.21.15.66:0/2911357983 learned_addr learned my addr 172.21.15.66:0/2911357983 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea19c640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29e410c3d0 0x7f29e419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 -- 172.21.15.66:0/2911357983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29e410c3d0 msgr2=0x7f29e419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29e410c3d0 0x7f29e419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 -- 172.21.15.66:0/2911357983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e4102fb0 msgr2=0x7f29e4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e999b640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e4102fb0 0x7f29e4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e4102fb0 0x7f29e4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 -- 172.21.15.66:0/2911357983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e419d5a0 con 0x7f29e41038d0 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea19c640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29e410c3d0 0x7f29e419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29e919a640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e41038d0 0x7f29e4198b40 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d750 tx=0x7f29d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29daffd640 1 -- 172.21.15.66:0/2911357983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4017070 con 0x7f29e41038d0 2024-04-21T09:17:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29daffd640 1 -- 172.21.15.66:0/2911357983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4005020 con 0x7f29e41038d0 2024-04-21T09:17:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29daffd640 1 -- 172.21.15.66:0/2911357983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4004c10 con 0x7f29e41038d0 2024-04-21T09:17:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e419d890 con 0x7f29e41038d0 2024-04-21T09:17:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.444+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e41a5120 con 0x7f29e41038d0 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.446+0000 7f29daffd640 1 -- 172.21.15.66:0/2911357983 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f29d40051d0 con 0x7f29e41038d0 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 Processor -- start 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 -- start start 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f3302fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f33030020 0x559f3303ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3303d010 0x559f3303f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559f32f941f0 con 0x559f3302f670 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f32f314b0 con 0x559f33030020 2024-04-21T09:17:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559f32f901b0 con 0x559f3303d010 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29daffd640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f29d00a3da0 0x7f29d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ea19c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f3302fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.447+0000 7f29ea19c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f3302fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42046/0 (socket says 172.21.15.66:42046) 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 -- 172.21.15.66:0/1740007736 learned_addr learned my addr 172.21.15.66:0/1740007736 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29e999b640 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f33030020 0x559f3303ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f29d00a3da0 0x7f29d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29daffd640 1 -- 172.21.15.66:0/2911357983 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f29d40d04b0 con 0x7f29e41038d0 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29e919a640 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3303d010 0x559f3303f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 -- 172.21.15.66:0/1740007736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3303d010 msgr2=0x559f3303f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3303d010 0x559f3303f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 -- 172.21.15.66:0/1740007736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f33030020 msgr2=0x559f3303ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f33030020 0x559f3303ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 -- 172.21.15.66:0/1740007736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f32f231b0 con 0x559f3302f670 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29e999b640 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f33030020 0x559f3303ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:17:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29e919a640 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3303d010 0x559f3303f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f3302fa50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f29e002c2e0 tx=0x7f29e002ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d09d7caee47ae4e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29ea19c640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f29d00a3da0 0x7f29d00a6260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29e002c210 tx=0x7f29e0040000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29cffff640 1 -- 172.21.15.66:0/1740007736 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e002ce00 con 0x559f3302f670 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29cffff640 1 -- 172.21.15.66:0/1740007736 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e0050070 con 0x559f3302f670 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.448+0000 7f29cffff640 1 -- 172.21.15.66:0/1740007736 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e00035f0 con 0x559f3302f670 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1740007736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 msgr2=0x559f3302fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f3302fa50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f29e002c2e0 tx=0x7f29e002ca30 comp rx=0 tx=0).stop 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1740007736 shutdown_connections 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3303d010 0x559f3303f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f33030020 0x559f3303ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1740007736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f3302fa50 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1740007736 >> 172.21.15.66:0/1740007736 conn(0x559f33029da0 msgr2=0x559f3302a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1740007736 shutdown_connections 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1740007736 wait complete. 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.449+0000 7f29ee7f2740 1 -- start start 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 1 Processor -- start 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 0x559f3303c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f33039c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559f3303fee0 con 0x559f33030020 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f32f4d500 con 0x559f3303d010 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559f33040060 con 0x559f3302f670 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ee7f2740 10 client.0 _ll_register_callbacks cb 0x559f32f1bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f33039c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f33039c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42056/0 (socket says 172.21.15.66:42056) 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 -- 172.21.15.66:0/1011292299 learned_addr learned my addr 172.21.15.66:0/1011292299 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29ea19c640 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 -- 172.21.15.66:0/1011292299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 msgr2=0x559f3303c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 0x559f3303c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 -- 172.21.15.66:0/1011292299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 msgr2=0x559f3303a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.450+0000 7f29e919a640 1 -- 172.21.15.66:0/1011292299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f32ca63f0 con 0x559f33030020 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29e919a640 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f33039c90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f29d40cd950 tx=0x7f29d40a0590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a55a884720c363 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29cd7fa640 1 -- 172.21.15.66:0/1011292299 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d40b5060 con 0x559f33030020 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e0050070 con 0x559f33030020 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29cd7fa640 1 -- 172.21.15.66:0/1011292299 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f29d4098790 con 0x559f33030020 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29cd7fa640 1 -- 172.21.15.66:0/1011292299 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29d4020040 con 0x559f33030020 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 10 client.18045 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559f32d707d0 con 0x559f33030020 2024-04-21T09:17:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29cd7fa640 1 -- 172.21.15.66:0/1011292299 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e0050070 con 0x559f33030020 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29cd7fa640 1 -- 172.21.15.66:0/1011292299 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29d40054f0 con 0x559f33030020 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 10 client.18045 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 10 client.18045 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 20 client.18045 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f3304d110 0x559f3304f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 4 client.18045 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.451+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f32f33930 con 0x559f3304d110 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.452+0000 7f29e999b640 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f3304d110 0x559f3304f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.452+0000 7f29cd7fa640 10 client.18045 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.452+0000 7f29e999b640 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f3304d110 0x559f3304f540 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=454e9002bc6835e9 server_cookie=9211a79841dfdcf5 in_seq=0 out_seq=0 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29cd7fa640 1 -- 172.21.15.66:0/1011292299 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f32f33930 con 0x559f3304d110 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29cd7fa640 10 client.18045 handle_command_reply: tid=0 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 client.18045 shutdown 2024-04-21T09:17:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 msgr2=0x559f33039c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f33039c90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f29d40cd950 tx=0x7f29d40a0590 comp rx=0 tx=0).stop 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 shutdown_connections 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f3304d110 0x559f3304f540 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f33039c90 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29cd7fa640 0 client.18045 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1011292299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 0x559f3303c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 >> 172.21.15.66:0/1011292299 conn(0x559f33029da0 msgr2=0x559f3302baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 shutdown_connections 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1011292299 wait complete. 2024-04-21T09:17:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.453+0000 7f29ee7f2740 20 client.18045 trim_cache size 0 max 16384 2024-04-21T09:17:58.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 Processor -- start 2024-04-21T09:17:58.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 -- start start 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 0x559f33048de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f3303c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41af820 con 0x559f33030020 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41ad0f0 con 0x559f3303d010 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41aaf60 con 0x559f3302f670 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f3303c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f3303c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42060/0 (socket says 172.21.15.66:42060) 2024-04-21T09:17:58.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 -- 172.21.15.66:0/1377018639 learned_addr learned my addr 172.21.15.66:0/1377018639 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29ea19c640 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e999b640 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 0x559f33048de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 -- 172.21.15.66:0/1377018639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 msgr2=0x559f33048de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 0x559f33048de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 -- 172.21.15.66:0/1377018639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 msgr2=0x559f3303c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 -- 172.21.15.66:0/1377018639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3302fdc0 con 0x559f33030020 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.467+0000 7f29e919a640 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f3303c3c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f29d409c0b0 tx=0x7f29d409c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce20147b7fc4821f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29cf7fe640 1 -- 172.21.15.66:0/1377018639 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d409bc60 con 0x559f33030020 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29cf7fe640 1 -- 172.21.15.66:0/1377018639 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4098790 con 0x559f33030020 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29cf7fe640 1 -- 172.21.15.66:0/1377018639 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d409b570 con 0x559f33030020 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1377018639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 msgr2=0x559f3303c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f3303c3c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f29d409c0b0 tx=0x7f29d409c8f0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1377018639 shutdown_connections 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f3303c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f33030020 0x559f3303c3c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/1377018639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3302f670 0x559f33048de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1377018639 >> 172.21.15.66:0/1377018639 conn(0x559f33029da0 msgr2=0x559f33035270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1377018639 shutdown_connections 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 -- 172.21.15.66:0/1377018639 wait complete. 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.468+0000 7f29ee7f2740 1 -- start start 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 1 Processor -- start 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f33045760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33030020 0x559f33040b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f330410c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41ab5b0 con 0x559f3302f670 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41aca20 con 0x559f3303d010 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41abc80 con 0x559f33030020 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ee7f2740 10 client.0 _ll_register_callbacks cb 0x7f29d409c120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f33045760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f33045760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42068/0 (socket says 172.21.15.66:42068) 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 -- 172.21.15.66:0/3938716788 learned_addr learned my addr 172.21.15.66:0/3938716788 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29ea19c640 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f330410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e919a640 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33030020 0x559f33040b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 -- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33030020 msgr2=0x559f33040b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33030020 0x559f33040b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 -- 172.21.15.66:0/3938716788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 msgr2=0x559f330410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f330410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.469+0000 7f29e999b640 1 -- 172.21.15.66:0/3938716788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f330418f0 con 0x559f3302f670 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29e999b640 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f33045760 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009fd0 tx=0x7f29dc012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394c3e2893d96f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29cd7fa640 1 -- 172.21.15.66:0/3938716788 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc01b8f0 con 0x559f3302f670 2024-04-21T09:17:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d4098790 con 0x559f3302f670 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29cd7fa640 1 -- 172.21.15.66:0/3938716788 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f29dc004ab0 con 0x559f3302f670 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29cd7fa640 1 -- 172.21.15.66:0/3938716788 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29dc02b050 con 0x559f3302f670 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 10 client.18051 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559f330b37b0 con 0x559f3302f670 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29cd7fa640 1 -- 172.21.15.66:0/3938716788 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4098790 con 0x559f3302f670 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29cd7fa640 1 -- 172.21.15.66:0/3938716788 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29dc023400 con 0x559f3302f670 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 10 client.18051 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 10 client.18051 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 20 client.18051 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f330b3f20 0x559f330b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 4 client.18051 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.470+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f32fa1c10 con 0x559f330b3f20 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29e919a640 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f330b3f20 0x559f330b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29cd7fa640 10 client.18051 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29e919a640 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f330b3f20 0x559f330b6340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a930e802eefb89f server_cookie=ec76bb679c3be71b in_seq=0 out_seq=0 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29cd7fa640 1 -- 172.21.15.66:0/3938716788 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f32fa1c10 con 0x559f330b3f20 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29cd7fa640 10 client.18051 handle_command_reply: tid=0 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 client.18051 shutdown 2024-04-21T09:17:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 msgr2=0x559f33045760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f33045760 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009fd0 tx=0x7f29dc012a40 comp rx=0 tx=0).stop 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 shutdown_connections 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f3303d010 0x559f330410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33030020 0x559f33040b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x559f330b3f20 0x559f330b6340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.471+0000 7f29ee7f2740 1 --2- 172.21.15.66:0/3938716788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f3302f670 0x559f33045760 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29cd7fa640 0 client.18051 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 >> 172.21.15.66:0/3938716788 conn(0x559f33029da0 msgr2=0x559f3302b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 shutdown_connections 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ee7f2740 1 -- 172.21.15.66:0/3938716788 wait complete. 2024-04-21T09:17:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ee7f2740 20 client.18051 trim_cache size 0 max 16384 2024-04-21T09:17:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f29d00a3da0 msgr2=0x7f29d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f29d00a3da0 0x7f29d00a6260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29e002c210 tx=0x7f29e0040000 comp rx=0 tx=0).stop 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e41038d0 msgr2=0x7f29e4198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e41038d0 0x7f29e4198b40 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d750 tx=0x7f29d400dc10 comp rx=0 tx=0).stop 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 shutdown_connections 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f29d00a3da0 0x7f29d00a6260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29e410c3d0 0x7f29e419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29e41038d0 0x7f29e4198b40 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 --2- 172.21.15.66:0/2911357983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e4102fb0 0x7f29e4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 >> 172.21.15.66:0/2911357983 conn(0x7f29e40fc7d0 msgr2=0x7f29e40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 shutdown_connections 2024-04-21T09:17:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.472+0000 7f29ea99d640 1 -- 172.21.15.66:0/2911357983 wait complete. 2024-04-21T09:17:58.483 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:58 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3870576554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:17:58.484 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:17:58.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:58 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3870576554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:17:58.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.589+0000 7f4ac54f2640 1 -- 172.21.15.66:0/1063620894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0107260 msgr2=0x7f4ac010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.589+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/1063620894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0107260 0x7f4ac010ee30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009ca0 tx=0x7f4ab002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.589+0000 7f4ac54f2640 1 -- 172.21.15.66:0/1063620894 shutdown_connections 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/1063620894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0107260 0x7f4ac010ee30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/1063620894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac0102b40 0x7f4ac0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/1063620894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0102190 0x7f4ac0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 -- 172.21.15.66:0/1063620894 >> 172.21.15.66:0/1063620894 conn(0x7f4ac00fd990 msgr2=0x7f4ac00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 -- 172.21.15.66:0/1063620894 shutdown_connections 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 -- 172.21.15.66:0/1063620894 wait complete. 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 Processor -- start 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.590+0000 7f4ac54f2640 1 -- start start 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0102190 0x7f4ac0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac0107260 0x7f4ac0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0074cd0 0x7f4ac01a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01121d0 con 0x7f4ac0102190 2024-04-21T09:17:58.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0112050 con 0x7f4ac0074cd0 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0112350 con 0x7f4ac0107260 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0102190 0x7f4ac0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0102190 0x7f4ac0074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42084/0 (socket says 172.21.15.66:42084) 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 -- 172.21.15.66:0/3754200990 learned_addr learned my addr 172.21.15.66:0/3754200990 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0074cd0 0x7f4ac01a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 -- 172.21.15.66:0/3754200990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac0107260 msgr2=0x7f4ac0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac0107260 0x7f4ac0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 -- 172.21.15.66:0/3754200990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0074cd0 msgr2=0x7f4ac01a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0074cd0 0x7f4ac01a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 -- 172.21.15.66:0/3754200990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac01a7000 con 0x7f4ac0102190 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0102190 0x7f4ac0074250 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800e9f0 tx=0x7f4ab800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4aa7fff640 1 -- 172.21.15.66:0/3754200990 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800cd80 con 0x7f4ac0102190 2024-04-21T09:17:58.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4aa7fff640 1 -- 172.21.15.66:0/3754200990 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab80108e0 con 0x7f4ac0102190 2024-04-21T09:17:58.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4aa7fff640 1 -- 172.21.15.66:0/3754200990 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8018980 con 0x7f4ac0102190 2024-04-21T09:17:58.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac01a72f0 con 0x7f4ac0102190 2024-04-21T09:17:58.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.591+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac01a7890 con 0x7f4ac0102190 2024-04-21T09:17:58.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.593+0000 7f4aa7fff640 1 -- 172.21.15.66:0/3754200990 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4ab8010430 con 0x7f4ac0102190 2024-04-21T09:17:58.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 Processor -- start 2024-04-21T09:17:58.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 -- start start 2024-04-21T09:17:58.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c1e90551f0 con 0x55c1e90f9a00 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8ff24b0 con 0x55c1e90f04d0 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c1e90511b0 con 0x55c1e90f0e80 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4aa7fff640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4aa80a3dd0 0x7f4aa80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90fbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42092/0 (socket says 172.21.15.66:42092) 2024-04-21T09:17:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3099418610 learned_addr learned my addr 172.21.15.66:0/3099418610 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4aa80a3dd0 0x7f4aa80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4aa7fff640 1 -- 172.21.15.66:0/3754200990 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4ab80d0fc0 con 0x7f4ac0102190 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3099418610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 msgr2=0x55c1e90f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3099418610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 msgr2=0x55c1e90f08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3099418610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e8fe41b0 con 0x55c1e90f9a00 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.594+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.595+0000 7f4abe7fc640 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90fbe50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400ba20 tx=0x7f4ab400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c19f8bdc9f3b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.595+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4aa80a3dd0 0x7f4aa80a6290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0004af0 tx=0x7f4ab00095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.595+0000 7f4aa57fa640 1 -- 172.21.15.66:0/3099418610 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4016070 con 0x55c1e90f9a00 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.595+0000 7f4aa57fa640 1 -- 172.21.15.66:0/3099418610 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab40054d0 con 0x55c1e90f9a00 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.595+0000 7f4aa57fa640 1 -- 172.21.15.66:0/3099418610 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4004ce0 con 0x55c1e90f9a00 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3099418610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 msgr2=0x55c1e90fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90fbe50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400ba20 tx=0x7f4ab400bee0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3099418610 shutdown_connections 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90fbe50 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3099418610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3099418610 >> 172.21.15.66:0/3099418610 conn(0x55c1e90eaba0 msgr2=0x55c1e90eaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3099418610 shutdown_connections 2024-04-21T09:17:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3099418610 wait complete. 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- start start 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 Processor -- start 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c1e90fc970 con 0x55c1e90f9a00 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c1e900e500 con 0x55c1e90f04d0 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c1e90fcaf0 con 0x55c1e90f0e80 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4ac81c4740 10 client.0 _ll_register_callbacks cb 0x55c1e8fdcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4abf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4abf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42104/0 (socket says 172.21.15.66:42104) 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4abf7fe640 1 -- 172.21.15.66:0/246259450 learned_addr learned my addr 172.21.15.66:0/246259450 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4abeffd640 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.596+0000 7f4abe7fc640 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4abf7fe640 1 -- 172.21.15.66:0/246259450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 msgr2=0x55c1e90f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4abf7fe640 1 -- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 msgr2=0x55c1e90f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4abf7fe640 1 -- 172.21.15.66:0/246259450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e8d673f0 con 0x55c1e90f9a00 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f6b00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0036000 tx=0x7f4ab002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64e4c1cc398a62b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4aa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/246259450 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0050060 con 0x55c1e90f9a00 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/246259450 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4ab0004040 con 0x55c1e90f9a00 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab40054d0 con 0x55c1e90f9a00 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/246259450 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ab0039030 con 0x55c1e90f9a00 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4ac81c4740 10 client.18063 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1e8e317d0 con 0x55c1e90f9a00 2024-04-21T09:17:58.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.597+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/246259450 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab40054d0 con 0x55c1e90f9a00 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/246259450 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4ab003f070 con 0x55c1e90f9a00 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4ac81c4740 10 client.18063 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4ac81c4740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4ac81c4740 20 client.18063 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9109aa0 0x55c1e910bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4ac81c4740 4 client.18063 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1e8ff4930 con 0x55c1e9109aa0 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4abeffd640 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9109aa0 0x55c1e910bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4aa2ff5640 10 client.18063 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.598+0000 7f4abeffd640 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9109aa0 0x55c1e910bef0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148a66264247915a server_cookie=e8ec79d856b8817e in_seq=0 out_seq=0 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/246259450 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1e8ff4930 con 0x55c1e9109aa0 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4aa2ff5640 10 client.18063 handle_command_reply: tid=0 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 client.18063 shutdown 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 msgr2=0x55c1e90f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f6b00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0036000 tx=0x7f4ab002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 shutdown_connections 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9109aa0 0x55c1e910bef0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f6b00 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4aa2ff5640 0 client.18063 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f0e80 0x55c1e90f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/246259450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f04d0 0x55c1e90f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 >> 172.21.15.66:0/246259450 conn(0x55c1e90eaba0 msgr2=0x55c1e90ec8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 shutdown_connections 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 1 -- 172.21.15.66:0/246259450 wait complete. 2024-04-21T09:17:58.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.599+0000 7f4ac81c4740 20 client.18063 trim_cache size 0 max 16384 2024-04-21T09:17:58.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:58 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3870576554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.612+0000 7f4ac81c4740 1 Processor -- start 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.612+0000 7f4ac81c4740 1 -- start start 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9105830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f0e80 0x55c1e90f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01b3ef0 con 0x55c1e90f9a00 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01b17c0 con 0x55c1e90f0e80 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01af630 con 0x55c1e90f04d0 2024-04-21T09:17:58.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42108/0 (socket says 172.21.15.66:42108) 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 -- 172.21.15.66:0/3279615004 learned_addr learned my addr 172.21.15.66:0/3279615004 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9105830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 -- 172.21.15.66:0/3279615004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 msgr2=0x55c1e9105830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9105830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 -- 172.21.15.66:0/3279615004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f0e80 msgr2=0x55c1e90f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f0e80 0x55c1e90f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 -- 172.21.15.66:0/3279615004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e90ea9d0 con 0x55c1e90f9a00 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4abf7fe640 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f8cf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0034a20 tx=0x7f4ab0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebe2d44b423c622 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4aa4ff9640 1 -- 172.21.15.66:0/3279615004 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0050070 con 0x55c1e90f9a00 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4aa4ff9640 1 -- 172.21.15.66:0/3279615004 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab0047040 con 0x55c1e90f9a00 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.613+0000 7f4aa4ff9640 1 -- 172.21.15.66:0/3279615004 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0004750 con 0x55c1e90f9a00 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3279615004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 msgr2=0x55c1e90f8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f8cf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0034a20 tx=0x7f4ab0004a40 comp rx=0 tx=0).stop 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3279615004 shutdown_connections 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f9a00 0x55c1e90f8cf0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f0e80 0x55c1e90f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3279615004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3279615004 >> 172.21.15.66:0/3279615004 conn(0x55c1e90eaba0 msgr2=0x55c1e90f1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3279615004 shutdown_connections 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3279615004 wait complete. 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 -- start start 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 Processor -- start 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.614+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f0e80 0x55c1e90fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f9a00 0x55c1e90fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01afc80 con 0x55c1e90f0e80 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01b10f0 con 0x55c1e90f9a00 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01b0350 con 0x55c1e90f04d0 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 10 client.0 _ll_register_callbacks cb 0x7f4ab0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f0e80 0x55c1e90fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f0e80 0x55c1e90fd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42114/0 (socket says 172.21.15.66:42114) 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3186539182 learned_addr learned my addr 172.21.15.66:0/3186539182 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3186539182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 msgr2=0x55c1e9101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3186539182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f9a00 msgr2=0x55c1e90fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:58.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f9a00 0x55c1e90fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 -- 172.21.15.66:0/3186539182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e90fdf90 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4abe7fc640 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f0e80 0x55c1e90fd360 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4004950 tx=0x7f4ab400dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d150235685b7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4aa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/3186539182 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4029060 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/3186539182 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4ab4004040 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab0047040 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/3186539182 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ab4030050 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 10 client.18069 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.615+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1e916ff30 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/3186539182 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab0047040 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/3186539182 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4ab4003c30 con 0x55c1e90f0e80 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4ac81c4740 10 client.18069 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:58.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4ac81c4740 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4ac81c4740 20 client.18069 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9170370 0x55c1e9172760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4ac81c4740 4 client.18069 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1e90f6e90 con 0x55c1e9170370 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9170370 0x55c1e9172760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4abeffd640 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9170370 0x55c1e9172760 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b5911467b966f95 server_cookie=efaa36c0613c1d1e in_seq=0 out_seq=0 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.616+0000 7f4aa2ff5640 10 client.18069 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4aa2ff5640 1 -- 172.21.15.66:0/3186539182 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1e90f6e90 con 0x55c1e9170370 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4aa2ff5640 10 client.18069 handle_command_reply: tid=0 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 client.18069 shutdown 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f0e80 msgr2=0x55c1e90fd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f0e80 0x55c1e90fd360 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4004950 tx=0x7f4ab400dc70 comp rx=0 tx=0).stop 2024-04-21T09:17:58.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 shutdown_connections 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c1e9170370 0x55c1e9172760 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1e90f9a00 0x55c1e90fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1e90f0e80 0x55c1e90fd360 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4aa2ff5640 0 client.18069 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 --2- 172.21.15.66:0/3186539182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c1e90f04d0 0x55c1e9101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 >> 172.21.15.66:0/3186539182 conn(0x55c1e90eaba0 msgr2=0x55c1e90ec7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 shutdown_connections 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 1 -- 172.21.15.66:0/3186539182 wait complete. 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac81c4740 20 client.18069 trim_cache size 0 max 16384 2024-04-21T09:17:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:17:58.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4aa80a3dd0 msgr2=0x7f4aa80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4aa80a3dd0 0x7f4aa80a6290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0004af0 tx=0x7f4ab00095c0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0102190 msgr2=0x7f4ac0074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:58.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0102190 0x7f4ac0074250 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800e9f0 tx=0x7f4ab800eeb0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 shutdown_connections 2024-04-21T09:17:58.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f4aa80a3dd0 0x7f4aa80a6290 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0074cd0 0x7f4ac01a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac0107260 0x7f4ac0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 --2- 172.21.15.66:0/3754200990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac0102190 0x7f4ac0074250 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:58.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.617+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 >> 172.21.15.66:0/3754200990 conn(0x7f4ac00fd990 msgr2=0x7f4ac00ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:58.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.618+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 shutdown_connections 2024-04-21T09:17:58.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:58.618+0000 7f4ac54f2640 1 -- 172.21.15.66:0/3754200990 wait complete. 2024-04-21T09:17:59.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:17:59 smithi186 ceph-mon[31505]: pgmap v302: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:59.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:17:59 smithi083 ceph-mon[31413]: pgmap v302: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:17:59 smithi066 ceph-mon[27953]: pgmap v302: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:17:59.628 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T09:17:59.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:17:59.628 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T09:17:59.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 -- 172.21.15.66:0/3045699377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644406aa90 msgr2=0x7f644406aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 --2- 172.21.15.66:0/3045699377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644406aa90 0x7f644406aed0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f642c009900 tx=0x7f642c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 -- 172.21.15.66:0/3045699377 shutdown_connections 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 --2- 172.21.15.66:0/3045699377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f644410e810 0x7f6444110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 --2- 172.21.15.66:0/3045699377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644406aa90 0x7f644406aed0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 --2- 172.21.15.66:0/3045699377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6444102e30 0x7f644406a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 -- 172.21.15.66:0/3045699377 >> 172.21.15.66:0/3045699377 conn(0x7f6444076c50 msgr2=0x7f6444077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 -- 172.21.15.66:0/3045699377 shutdown_connections 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 -- 172.21.15.66:0/3045699377 wait complete. 2024-04-21T09:17:59.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.749+0000 7f6449706640 1 Processor -- start 2024-04-21T09:17:59.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6449706640 1 -- start start 2024-04-21T09:17:59.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6449706640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f644406aa90 0x7f64441a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6449706640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6444102e30 0x7f64441a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6449706640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644410e810 0x7f644419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6449706640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64441143a0 con 0x7f644410e810 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6449706640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6444114220 con 0x7f644406aa90 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6449706640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6444114520 con 0x7f6444102e30 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644410e810 0x7f644419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644410e810 0x7f644419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42122/0 (socket says 172.21.15.66:42122) 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 -- 172.21.15.66:0/3770204830 learned_addr learned my addr 172.21.15.66:0/3770204830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f6442ffd640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f644406aa90 0x7f64441a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64427fc640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6444102e30 0x7f64441a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 -- 172.21.15.66:0/3770204830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6444102e30 msgr2=0x7f64441a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6444102e30 0x7f64441a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 -- 172.21.15.66:0/3770204830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f644406aa90 msgr2=0x7f64441a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f644406aa90 0x7f64441a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64437fe640 1 -- 172.21.15.66:0/3770204830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644419afa0 con 0x7f644410e810 2024-04-21T09:17:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.750+0000 7f64427fc640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6444102e30 0x7f64441a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:59.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.751+0000 7f64437fe640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644410e810 0x7f644419a7d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f643400d720 tx=0x7f643400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.751+0000 7f643bfff640 1 -- 172.21.15.66:0/3770204830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434017070 con 0x7f644410e810 2024-04-21T09:17:59.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.751+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644419b290 con 0x7f644410e810 2024-04-21T09:17:59.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.751+0000 7f643bfff640 1 -- 172.21.15.66:0/3770204830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f643400de30 con 0x7f644410e810 2024-04-21T09:17:59.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.751+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64441a74b0 con 0x7f644410e810 2024-04-21T09:17:59.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.751+0000 7f643bfff640 1 -- 172.21.15.66:0/3770204830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434020060 con 0x7f644410e810 2024-04-21T09:17:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.753+0000 7f643bfff640 1 -- 172.21.15.66:0/3770204830 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6434004ca0 con 0x7f644410e810 2024-04-21T09:17:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 Processor -- start 2024-04-21T09:17:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 -- start start 2024-04-21T09:17:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f964dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9656840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f9659260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e5f95b1f00 con 0x55e5f9656e10 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5f954f590 con 0x55e5f964d8e0 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f644c3cb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e5f95adec0 con 0x55e5f964e290 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f64427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f9659260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f643bfff640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64300a3d20 0x7f64300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f64427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f9659260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42126/0 (socket says 172.21.15.66:42126) 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f64427fc640 1 -- 172.21.15.66:0/2115443683 learned_addr learned my addr 172.21.15.66:0/2115443683 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f64427fc640 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f964dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f64437fe640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64300a3d20 0x7f64300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.754+0000 7f6442ffd640 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9656840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f643bfff640 1 -- 172.21.15.66:0/3770204830 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f64340d07b0 con 0x7f644410e810 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64427fc640 1 -- 172.21.15.66:0/2115443683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 msgr2=0x55e5f9656840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64427fc640 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9656840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64427fc640 1 -- 172.21.15.66:0/2115443683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 msgr2=0x55e5f964dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64427fc640 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f964dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64427fc640 1 -- 172.21.15.66:0/2115443683 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f938c7e0 con 0x55e5f9656e10 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64427fc640 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f9659260 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f642c02c170 tx=0x7f642c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d289ebf4ef9ffc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64437fe640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64300a3d20 0x7f64300a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64340cdbe0 tx=0x7f6434000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64397fa640 1 -- 172.21.15.66:0/2115443683 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c049070 con 0x55e5f9656e10 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64397fa640 1 -- 172.21.15.66:0/2115443683 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642c005500 con 0x55e5f9656e10 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.755+0000 7f64397fa640 1 -- 172.21.15.66:0/2115443683 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c004920 con 0x55e5f9656e10 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 -- 172.21.15.66:0/2115443683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 msgr2=0x55e5f9659260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f9659260 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f642c02c170 tx=0x7f642c005b50 comp rx=0 tx=0).stop 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 -- 172.21.15.66:0/2115443683 shutdown_connections 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f9659260 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9656840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 --2- 172.21.15.66:0/2115443683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f964dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 -- 172.21.15.66:0/2115443683 >> 172.21.15.66:0/2115443683 conn(0x55e5f9647fb0 msgr2=0x55e5f9648390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 -- 172.21.15.66:0/2115443683 shutdown_connections 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 -- 172.21.15.66:0/2115443683 wait complete. 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 -- start start 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.756+0000 7f644c3cb740 1 Processor -- start 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f9653e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9654370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f96548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9659d20 con 0x55e5f9656e10 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5f956b580 con 0x55e5f964d8e0 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e5f9659ea0 con 0x55e5f964e290 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 10 client.0 _ll_register_callbacks cb 0x55e5f9571650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f96548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f96548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42128/0 (socket says 172.21.15.66:42128) 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 -- 172.21.15.66:0/3031917850 learned_addr learned my addr 172.21.15.66:0/3031917850 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f6442ffd640 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9654370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f9653e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 -- 172.21.15.66:0/3031917850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 msgr2=0x55e5f9654370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9654370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 -- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 msgr2=0x55e5f9653e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f9653e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 -- 172.21.15.66:0/3031917850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f93ecf70 con 0x55e5f9656e10 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f64427fc640 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f96548b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f642c009640 tx=0x7f642c0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca8b1405b77ffaa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f642a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f642a7fc640 1 -- 172.21.15.66:0/3031917850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c058060 con 0x55e5f9656e10 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f642a7fc640 1 -- 172.21.15.66:0/3031917850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f642c004ae0 con 0x55e5f9656e10 2024-04-21T09:17:59.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.757+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f642c005500 con 0x55e5f9656e10 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f642a7fc640 1 -- 172.21.15.66:0/3031917850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f642c05f050 con 0x55e5f9656e10 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 10 client.18081 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e5f9284170 con 0x55e5f9656e10 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f642a7fc640 1 -- 172.21.15.66:0/3031917850 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642c005500 con 0x55e5f9656e10 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f642a7fc640 1 -- 172.21.15.66:0/3031917850 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f642c040040 con 0x55e5f9656e10 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 10 client.18081 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 20 client.18081 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f9667200 0x55e5f96695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 4 client.18081 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f95bf920 con 0x55e5f9667200 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.758+0000 7f6442ffd640 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f9667200 0x55e5f96695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.759+0000 7f642a7fc640 10 client.18081 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.759+0000 7f6442ffd640 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f9667200 0x55e5f96695f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece122292d929ff7 server_cookie=752cde6454c776fd in_seq=0 out_seq=0 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.759+0000 7f642a7fc640 1 -- 172.21.15.66:0/3031917850 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5f95bf920 con 0x55e5f9667200 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.759+0000 7f642a7fc640 10 client.18081 handle_command_reply: tid=0 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.759+0000 7f644c3cb740 1 client.18081 shutdown 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 msgr2=0x55e5f96548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f96548b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f642c009640 tx=0x7f642c0043d0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 shutdown_connections 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f9667200 0x55e5f96695f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f9656e10 0x55e5f96548b0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9654370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f642a7fc640 0 client.18081 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 --2- 172.21.15.66:0/3031917850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f9653e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 >> 172.21.15.66:0/3031917850 conn(0x55e5f9647fb0 msgr2=0x55e5f9649d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 shutdown_connections 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 1 -- 172.21.15.66:0/3031917850 wait complete. 2024-04-21T09:17:59.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.760+0000 7f644c3cb740 20 client.18081 trim_cache size 0 max 16384 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 Processor -- start 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 -- start start 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964d8e0 0x55e5f9656560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9662830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f9656e10 0x55e5f966c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64441b3e40 con 0x55e5f964d8e0 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64441b1710 con 0x55e5f9656e10 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f644c3cb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f64441af580 con 0x55e5f964e290 2024-04-21T09:17:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964d8e0 0x55e5f9656560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964d8e0 0x55e5f9656560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42132/0 (socket says 172.21.15.66:42132) 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 -- 172.21.15.66:0/1923503548 learned_addr learned my addr 172.21.15.66:0/1923503548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f6442ffd640 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9662830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f9656e10 0x55e5f966c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 -- 172.21.15.66:0/1923503548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 msgr2=0x55e5f9662830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9662830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 -- 172.21.15.66:0/1923503548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f9656e10 msgr2=0x55e5f966c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f9656e10 0x55e5f966c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.778+0000 7f64427fc640 1 -- 172.21.15.66:0/1923503548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f964e030 con 0x55e5f964d8e0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f64427fc640 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964d8e0 0x55e5f9656560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f642c0092a0 tx=0x7f642c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698d68855f5e3b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f6438ff9640 1 -- 172.21.15.66:0/1923503548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c058070 con 0x55e5f964d8e0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f6438ff9640 1 -- 172.21.15.66:0/1923503548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642c02c5d0 con 0x55e5f964d8e0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f6438ff9640 1 -- 172.21.15.66:0/1923503548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c049070 con 0x55e5f964d8e0 2024-04-21T09:17:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f644c3cb740 1 -- 172.21.15.66:0/1923503548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964d8e0 msgr2=0x55e5f9656560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964d8e0 0x55e5f9656560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f642c0092a0 tx=0x7f642c009230 comp rx=0 tx=0).stop 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f644c3cb740 1 -- 172.21.15.66:0/1923503548 shutdown_connections 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f9656e10 0x55e5f966c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f964e290 0x55e5f9662830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.779+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1923503548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964d8e0 0x55e5f9656560 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 -- 172.21.15.66:0/1923503548 >> 172.21.15.66:0/1923503548 conn(0x55e5f9647fb0 msgr2=0x55e5f9662120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 -- 172.21.15.66:0/1923503548 shutdown_connections 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 -- 172.21.15.66:0/1923503548 wait complete. 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 -- start start 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 Processor -- start 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f966a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964e290 0x55e5f966a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f9656e10 0x55e5f966ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64441afbd0 con 0x55e5f964e290 2024-04-21T09:17:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64441b1040 con 0x55e5f964d8e0 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f64441b02a0 con 0x55e5f9656e10 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f644c3cb740 10 client.0 _ll_register_callbacks cb 0x7f642c047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f6442ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964e290 0x55e5f966a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f6442ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964e290 0x55e5f966a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42148/0 (socket says 172.21.15.66:42148) 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f6442ffd640 1 -- 172.21.15.66:0/1232175315 learned_addr learned my addr 172.21.15.66:0/1232175315 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f64427fc640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f966a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.780+0000 7f64427fc640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f9656e10 0x55e5f966ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f6442ffd640 1 -- 172.21.15.66:0/1232175315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f9656e10 msgr2=0x55e5f966ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f6442ffd640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f9656e10 0x55e5f966ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f6442ffd640 1 -- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 msgr2=0x55e5f966a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f6442ffd640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f966a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f6442ffd640 1 -- 172.21.15.66:0/1232175315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f96cc410 con 0x55e5f964e290 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f64427fc640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f966a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f6442ffd640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964e290 0x55e5f966a5d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f643c003a80 tx=0x7f643c01b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f498130e88a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f642a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f642a7fc640 1 -- 172.21.15.66:0/1232175315 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0142d0 con 0x55e5f964e290 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f642a7fc640 1 -- 172.21.15.66:0/1232175315 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f643c014920 con 0x55e5f964e290 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f642c02c5d0 con 0x55e5f964e290 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f642a7fc640 1 -- 172.21.15.66:0/1232175315 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f643c014b60 con 0x55e5f964e290 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 10 client.18087 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e5f96cd7c0 con 0x55e5f964e290 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f642a7fc640 1 -- 172.21.15.66:0/1232175315 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642c02c5d0 con 0x55e5f964e290 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f642a7fc640 1 -- 172.21.15.66:0/1232175315 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f643c022070 con 0x55e5f964e290 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 10 client.18087 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 10 client.18087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 20 client.18087 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f96cdfc0 0x55e5f96d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 4 client.18087 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T09:17:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.781+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e5f95b9790 con 0x55e5f96cdfc0 2024-04-21T09:17:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.782+0000 7f64427fc640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f96cdfc0 0x55e5f96d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.782+0000 7f64427fc640 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f96cdfc0 0x55e5f96d03e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44128060a50bc9d1 server_cookie=aca084788a32b298 in_seq=0 out_seq=0 2024-04-21T09:17:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.782+0000 7f642a7fc640 10 client.18087 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f642a7fc640 1 -- 172.21.15.66:0/1232175315 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e5f95b9790 con 0x55e5f96cdfc0 2024-04-21T09:17:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f642a7fc640 10 client.18087 handle_command_reply: tid=0 2024-04-21T09:17:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f644c3cb740 1 client.18087 shutdown 2024-04-21T09:17:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964e290 msgr2=0x55e5f966a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964e290 0x55e5f966a5d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f643c003a80 tx=0x7f643c01b660 comp rx=0 tx=0).stop 2024-04-21T09:17:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 shutdown_connections 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55e5f96cdfc0 0x55e5f96d03e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5f9656e10 0x55e5f966ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5f964e290 0x55e5f966a5d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.783+0000 7f642a7fc640 0 client.18087 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f644c3cb740 1 --2- 172.21.15.66:0/1232175315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5f964d8e0 0x55e5f966a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 >> 172.21.15.66:0/1232175315 conn(0x55e5f9647fb0 msgr2=0x55e5f96629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 shutdown_connections 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f644c3cb740 1 -- 172.21.15.66:0/1232175315 wait complete. 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f644c3cb740 20 client.18087 trim_cache size 0 max 16384 2024-04-21T09:17:59.789 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:17:59.790 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-04-21T09:17:59.790 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "79499f4d-1c93-43c4-84f8-54295a07a936", 2024-04-21T09:17:59.790 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-04-21T09:17:59.790 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:17:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64300a3d20 msgr2=0x7f64300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64300a3d20 0x7f64300a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64340cdbe0 tx=0x7f6434000bc0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644410e810 msgr2=0x7f644419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644410e810 0x7f644419a7d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f643400d720 tx=0x7f643400dbe0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 shutdown_connections 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f64300a3d20 0x7f64300a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f644410e810 0x7f644419a7d0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6444102e30 0x7f64441a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 --2- 172.21.15.66:0/3770204830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f644406aa90 0x7f64441a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 >> 172.21.15.66:0/3770204830 conn(0x7f6444076c50 msgr2=0x7f644410f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 shutdown_connections 2024-04-21T09:17:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.784+0000 7f6449706640 1 -- 172.21.15.66:0/3770204830 wait complete. 2024-04-21T09:17:59.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:17:59.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:17:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.899+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2189653183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df0104d50 msgr2=0x7f0df0108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.899+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2189653183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df0104d50 0x7f0df0108d90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009900 tx=0x7f0dd802bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2189653183 shutdown_connections 2024-04-21T09:17:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2189653183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df0109470 0x7f0df0111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2189653183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df0104d50 0x7f0df0108d90 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2189653183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df01043a0 0x7f0df0104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2189653183 >> 172.21.15.66:0/2189653183 conn(0x7f0df00ffc20 msgr2=0x7f0df0102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2189653183 shutdown_connections 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2189653183 wait complete. 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 Processor -- start 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.900+0000 7f0df4ad2640 1 -- start start 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df01043a0 0x7f0df01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df0109470 0x7f0df01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df019a6f0 0x7f0df019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0114390 con 0x7f0df019a6f0 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0114210 con 0x7f0df0109470 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0114510 con 0x7f0df01043a0 2024-04-21T09:17:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df019a6f0 0x7f0df019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df019a6f0 0x7f0df019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42172/0 (socket says 172.21.15.66:42172) 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 -- 172.21.15.66:0/2797018264 learned_addr learned my addr 172.21.15.66:0/2797018264 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0dee59c640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df01043a0 0x7f0df01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 -- 172.21.15.66:0/2797018264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df01043a0 msgr2=0x7f0df01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df01043a0 0x7f0df01a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df0109470 0x7f0df01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 -- 172.21.15.66:0/2797018264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df0109470 msgr2=0x7f0df01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df0109470 0x7f0df01a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 -- 172.21.15.66:0/2797018264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df019b2f0 con 0x7f0df019a6f0 2024-04-21T09:17:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0deed9d640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df019a6f0 0x7f0df019ab70 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009fd0 tx=0x7f0de000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0de77fe640 1 -- 172.21.15.66:0/2797018264 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0017070 con 0x7f0df019a6f0 2024-04-21T09:17:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0de77fe640 1 -- 172.21.15.66:0/2797018264 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de0003e20 con 0x7f0df019a6f0 2024-04-21T09:17:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df01a73d0 con 0x7f0df019a6f0 2024-04-21T09:17:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0de77fe640 1 -- 172.21.15.66:0/2797018264 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0010460 con 0x7f0df019a6f0 2024-04-21T09:17:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.901+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df01a7780 con 0x7f0df019a6f0 2024-04-21T09:17:59.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.903+0000 7f0de77fe640 1 -- 172.21.15.66:0/2797018264 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0de0003970 con 0x7f0df019a6f0 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 Processor -- start 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 -- start start 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b82960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b8b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b8de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f515ae71e0 con 0x55f515b8ba80 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f515a84390 con 0x55f515b82ea0 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.904+0000 7f0df77a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f515ae31a0 con 0x55f515b82580 2024-04-21T09:17:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0de77fe640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0ddc0a3d20 0x7f0ddc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b8de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b8de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42188/0 (socket says 172.21.15.66:42188) 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3029706372 learned_addr learned my addr 172.21.15.66:0/3029706372 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dee59c640 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b8b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0deed9d640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0ddc0a3d20 0x7f0ddc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b82960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3029706372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 msgr2=0x55f515b82960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b82960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3029706372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 msgr2=0x55f515b8b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0de77fe640 1 -- 172.21.15.66:0/2797018264 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f0de00d0a00 con 0x7f0df019a6f0 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b8b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3029706372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5158c37d0 con 0x55f515b8ba80 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b8de70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0df006b700 tx=0x7f0dd8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43d863b957d0c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0deed9d640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0ddc0a3d20 0x7f0ddc0a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0de009c270 tx=0x7f0de0098810 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0de4ff9640 1 -- 172.21.15.66:0/3029706372 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8004290 con 0x55f515b8ba80 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0de4ff9640 1 -- 172.21.15.66:0/3029706372 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd8047070 con 0x55f515b8ba80 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.905+0000 7f0de4ff9640 1 -- 172.21.15.66:0/3029706372 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8003d50 con 0x55f515b8ba80 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 -- 172.21.15.66:0/3029706372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 msgr2=0x55f515b8de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b8de70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0df006b700 tx=0x7f0dd8000bc0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 -- 172.21.15.66:0/3029706372 shutdown_connections 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b8de70 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b8b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3029706372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b82960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 -- 172.21.15.66:0/3029706372 >> 172.21.15.66:0/3029706372 conn(0x55f515b7ccb0 msgr2=0x55f515b7d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 -- 172.21.15.66:0/3029706372 shutdown_connections 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 -- 172.21.15.66:0/3029706372 wait complete. 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.906+0000 7f0df77a6740 1 -- start start 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 1 Processor -- start 2024-04-21T09:17:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82580 0x55f515b88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82ea0 0x55f515b88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b8ba80 0x55f515b89520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f515b8e810 con 0x55f515b82580 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f515aa02f0 con 0x55f515b8ba80 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f515b8e990 con 0x55f515b82ea0 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0df77a6740 10 client.0 _ll_register_callbacks cb 0x55f515aad580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82580 0x55f515b88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82580 0x55f515b88aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42194/0 (socket says 172.21.15.66:42194) 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 -- 172.21.15.66:0/128304961 learned_addr learned my addr 172.21.15.66:0/128304961 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b8ba80 0x55f515b89520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dee59c640 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82ea0 0x55f515b88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 -- 172.21.15.66:0/128304961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82ea0 msgr2=0x55f515b88fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82ea0 0x55f515b88fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 -- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b8ba80 msgr2=0x55f515b89520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b8ba80 0x55f515b89520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 -- 172.21.15.66:0/128304961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5157f93f0 con 0x55f515b82580 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82580 0x55f515b88aa0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0dd802c420 tx=0x7f0dd802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b6dcd17bdd6b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.907+0000 7f0dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/128304961 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8055060 con 0x55f515b82580 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/128304961 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f0dd802c9e0 con 0x55f515b82580 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd8047070 con 0x55f515b82580 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/128304961 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0dd805c050 con 0x55f515b82580 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 10 client.18099 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5157b90f0 con 0x55f515b82580 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/128304961 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd8047070 con 0x55f515b82580 2024-04-21T09:17:59.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/128304961 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0dd8034440 con 0x55f515b82580 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 10 client.18099 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 10 client.18099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 20 client.18099 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515b9b8b0 0x55f515b9dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 4 client.18099 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f515af4c00 con 0x55f515b9b8b0 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.908+0000 7f0dee59c640 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515b9b8b0 0x55f515b9dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.909+0000 7f0dd5ffb640 10 client.18099 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.909+0000 7f0dee59c640 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515b9b8b0 0x55f515b9dca0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c90f0b16d12d3f3 server_cookie=32051a0a11cd5b7f in_seq=0 out_seq=0 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.909+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/128304961 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f515af4c00 con 0x55f515b9b8b0 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.909+0000 7f0dd5ffb640 10 client.18099 handle_command_reply: tid=0 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.909+0000 7f0df77a6740 1 client.18099 shutdown 2024-04-21T09:17:59.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82580 msgr2=0x55f515b88aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82580 0x55f515b88aa0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0dd802c420 tx=0x7f0dd802bc30 comp rx=0 tx=0).stop 2024-04-21T09:17:59.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 shutdown_connections 2024-04-21T09:17:59.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515b9b8b0 0x55f515b9dca0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b8ba80 0x55f515b89520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82ea0 0x55f515b88fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0dd5ffb640 0 client.18099 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 --2- 172.21.15.66:0/128304961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82580 0x55f515b88aa0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 >> 172.21.15.66:0/128304961 conn(0x55f515b7ccb0 msgr2=0x55f515b7ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 shutdown_connections 2024-04-21T09:17:59.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 1 -- 172.21.15.66:0/128304961 wait complete. 2024-04-21T09:17:59.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.910+0000 7f0df77a6740 20 client.18099 trim_cache size 0 max 16384 2024-04-21T09:17:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 Processor -- start 2024-04-21T09:17:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 -- start start 2024-04-21T09:17:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b8b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b87c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b7e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b3e10 con 0x55f515b8ba80 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b16e0 con 0x55f515b82ea0 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0df77a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01af550 con 0x55f515b82580 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0dedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b7e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0dedd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b7e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42200/0 (socket says 172.21.15.66:42200) 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.927+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3473768608 learned_addr learned my addr 172.21.15.66:0/3473768608 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b8b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3473768608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 msgr2=0x55f515b8b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b8b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dee59c640 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b87c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3473768608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 msgr2=0x55f515b87c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b87c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dedd9b640 1 -- 172.21.15.66:0/3473768608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f515b7dee0 con 0x55f515b8ba80 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b7e810 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80059d0 tx=0x7f0dd8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b007c4085b91c5b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dd7fff640 1 -- 172.21.15.66:0/3473768608 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8055070 con 0x55f515b8ba80 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dd7fff640 1 -- 172.21.15.66:0/3473768608 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd8047040 con 0x55f515b8ba80 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0dd7fff640 1 -- 172.21.15.66:0/3473768608 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8034440 con 0x55f515b8ba80 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0df77a6740 1 -- 172.21.15.66:0/3473768608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 msgr2=0x55f515b7e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b7e810 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80059d0 tx=0x7f0dd8003f90 comp rx=0 tx=0).stop 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0df77a6740 1 -- 172.21.15.66:0/3473768608 shutdown_connections 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b8ba80 0x55f515b7e810 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82ea0 0x55f515b87c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.928+0000 7f0df77a6740 1 --2- 172.21.15.66:0/3473768608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b82580 0x55f515b8b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 -- 172.21.15.66:0/3473768608 >> 172.21.15.66:0/3473768608 conn(0x55f515b7ccb0 msgr2=0x55f515b7e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 -- 172.21.15.66:0/3473768608 shutdown_connections 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 -- 172.21.15.66:0/3473768608 wait complete. 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 -- start start 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 Processor -- start 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82580 0x55f515ba0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82ea0 0x55f515b9e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b8ba80 0x55f515b9e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01afba0 con 0x55f515b82ea0 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b1010 con 0x55f515b82580 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b0270 con 0x55f515b8ba80 2024-04-21T09:17:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0df77a6740 10 client.0 _ll_register_callbacks cb 0x7f0de800b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0dee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82ea0 0x55f515b9e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0dee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82ea0 0x55f515b9e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42206/0 (socket says 172.21.15.66:42206) 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0dee59c640 1 -- 172.21.15.66:0/1348065942 learned_addr learned my addr 172.21.15.66:0/1348065942 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82580 0x55f515ba0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.929+0000 7f0dee59c640 1 -- 172.21.15.66:0/1348065942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b8ba80 msgr2=0x55f515b9e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dee59c640 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b8ba80 0x55f515b9e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dee59c640 1 -- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82580 msgr2=0x55f515ba0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dee59c640 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82580 0x55f515ba0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dee59c640 1 -- 172.21.15.66:0/1348065942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f515b9eeb0 con 0x55f515b82ea0 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82580 0x55f515ba0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:17:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dee59c640 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82ea0 0x55f515b9e300 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x55f515ae72f0 tx=0x7f0de800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bc46742e7b12f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/1348065942 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8020020 con 0x55f515b82ea0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/1348065942 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f0de800e3b0 con 0x55f515b82ea0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd8047040 con 0x55f515b82ea0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/1348065942 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0de802b050 con 0x55f515b82ea0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 10 client.18105 fetch_fsmap learned FSMap version 25 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f515c019c0 con 0x55f515b82ea0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/1348065942 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd8047040 con 0x55f515b82ea0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/1348065942 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0de801b4c0 con 0x55f515b82ea0 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 10 client.18105 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 10 client.18105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 20 client.18105 populate_metadata read hostname 'smithi066' 2024-04-21T09:17:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515c02170 0x55f515c04590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:17:59.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 4 client.18105 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:17:59.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.930+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f515aeea70 con 0x55f515c02170 2024-04-21T09:17:59.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515c02170 0x55f515c04590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:17:59.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0dd5ffb640 10 client.18105 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0dedd9b640 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515c02170 0x55f515c04590 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27437d5662a5e86e server_cookie=cef63a0f52e61bc3 in_seq=0 out_seq=0 2024-04-21T09:17:59.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0dd5ffb640 1 -- 172.21.15.66:0/1348065942 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f515aeea70 con 0x55f515c02170 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0dd5ffb640 10 client.18105 handle_command_reply: tid=0 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 client.18105 shutdown 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82ea0 msgr2=0x55f515b9e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82ea0 0x55f515b9e300 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x55f515ae72f0 tx=0x7f0de800f7f0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 shutdown_connections 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55f515c02170 0x55f515c04590 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f515b8ba80 0x55f515b9e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f515b82ea0 0x55f515b9e300 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0dd5ffb640 0 client.18105 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 --2- 172.21.15.66:0/1348065942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f515b82580 0x55f515ba0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 >> 172.21.15.66:0/1348065942 conn(0x55f515b7ccb0 msgr2=0x55f515b7bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 shutdown_connections 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.931+0000 7f0df77a6740 1 -- 172.21.15.66:0/1348065942 wait complete. 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df77a6740 20 client.18105 trim_cache size 0 max 16384 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:17:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "79499f4d-1c93-43c4-84f8-54295a07a936": { 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "79499f4d-1c93-43c4-84f8-54295a07a936", 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:17:59.944 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:17:59.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0ddc0a3d20 msgr2=0x7f0ddc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0ddc0a3d20 0x7f0ddc0a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0de009c270 tx=0x7f0de0098810 comp rx=0 tx=0).stop 2024-04-21T09:17:59.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df019a6f0 msgr2=0x7f0df019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:17:59.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df019a6f0 0x7f0df019ab70 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009fd0 tx=0x7f0de000b260 comp rx=0 tx=0).stop 2024-04-21T09:17:59.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 shutdown_connections 2024-04-21T09:17:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0ddc0a3d20 0x7f0ddc0a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0df019a6f0 0x7f0df019ab70 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df0109470 0x7f0df01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 --2- 172.21.15.66:0/2797018264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df01043a0 0x7f0df01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:17:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 >> 172.21.15.66:0/2797018264 conn(0x7f0df00ffc20 msgr2=0x7f0df0101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:17:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 shutdown_connections 2024-04-21T09:17:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:17:59.932+0000 7f0df4ad2640 1 -- 172.21.15.66:0/2797018264 wait complete. 2024-04-21T09:17:59.947 INFO:tasks.cephfs.filesystem:scrub status for tag:79499f4d-1c93-43c4-84f8-54295a07a936 - {'path': '/', 'tag': '79499f4d-1c93-43c4-84f8-54295a07a936', 'options': 'recursive,force'} 2024-04-21T09:18:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:00 smithi186 ceph-mon[31505]: scrub queued for path: / 2024-04-21T09:18:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:00 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:18:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:00 smithi186 ceph-mon[31505]: scrub summary: active paths [/] 2024-04-21T09:18:00.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:00 smithi186 ceph-mon[31505]: scrub summary: idle+waiting paths [/] 2024-04-21T09:18:00.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:00 smithi083 ceph-mon[31413]: scrub queued for path: / 2024-04-21T09:18:00.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:00 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:18:00.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:00 smithi083 ceph-mon[31413]: scrub summary: active paths [/] 2024-04-21T09:18:00.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:00 smithi083 ceph-mon[31413]: scrub summary: idle+waiting paths [/] 2024-04-21T09:18:00.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:00 smithi066 ceph-mon[27953]: scrub queued for path: / 2024-04-21T09:18:00.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:00 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:18:00.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:00 smithi066 ceph-mon[27953]: scrub summary: active paths [/] 2024-04-21T09:18:00.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:00 smithi066 ceph-mon[27953]: scrub summary: idle+waiting paths [/] 2024-04-21T09:18:01.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:01 smithi186 ceph-mon[31505]: pgmap v303: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:01.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:01 smithi186 ceph-mon[31505]: scrub summary: idle 2024-04-21T09:18:01.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:01 smithi083 ceph-mon[31413]: pgmap v303: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:01.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:01 smithi083 ceph-mon[31413]: scrub summary: idle 2024-04-21T09:18:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:01 smithi066 ceph-mon[27953]: pgmap v303: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:01.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:01 smithi066 ceph-mon[27953]: scrub summary: idle 2024-04-21T09:18:03.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:03 smithi186 ceph-mon[31505]: pgmap v304: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:03.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:03 smithi083 ceph-mon[31413]: pgmap v304: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:03.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:03 smithi066 ceph-mon[27953]: pgmap v304: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:05.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:05 smithi186 ceph-mon[31505]: pgmap v305: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:18:05.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:05 smithi083 ceph-mon[31413]: pgmap v305: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:18:05.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:05 smithi066 ceph-mon[27953]: pgmap v305: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T09:18:07.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:07 smithi186 ceph-mon[31505]: pgmap v306: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T09:18:07.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:07 smithi186 ceph-mon[31505]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T09:18:07.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:07 smithi083 ceph-mon[31413]: pgmap v306: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T09:18:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:07 smithi083 ceph-mon[31413]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T09:18:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:07 smithi066 ceph-mon[27953]: pgmap v306: 129 pgs: 129 active+clean; 2.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T09:18:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:07 smithi066 ceph-mon[27953]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T09:18:09.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:09 smithi186 ceph-mon[31505]: pgmap v308: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-21T09:18:09.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:09 smithi083 ceph-mon[31413]: pgmap v308: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-21T09:18:09.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:09 smithi066 ceph-mon[27953]: pgmap v308: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-21T09:18:11.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:10 smithi186 ceph-mon[31505]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T09:18:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:10 smithi186 ceph-mon[31505]: pgmap v310: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-04-21T09:18:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:10 smithi083 ceph-mon[31413]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T09:18:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:10 smithi083 ceph-mon[31413]: pgmap v310: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-04-21T09:18:11.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:10 smithi066 ceph-mon[27953]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T09:18:11.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:10 smithi066 ceph-mon[27953]: pgmap v310: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-04-21T09:18:13.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:13 smithi186 ceph-mon[31505]: pgmap v311: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:18:13.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:13 smithi083 ceph-mon[31413]: pgmap v311: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:18:13.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:13 smithi066 ceph-mon[27953]: pgmap v311: 129 pgs: 129 active+clean; 2.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:18:15.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:15 smithi186 ceph-mon[31505]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:18:15.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:15 smithi083 ceph-mon[31413]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:18:15.621 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:15 smithi066 ceph-mon[27953]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T09:18:17.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:17 smithi186 ceph-mon[31505]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-21T09:18:17.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:17 smithi083 ceph-mon[31413]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-21T09:18:17.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:17 smithi066 ceph-mon[27953]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-21T09:18:19.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:19 smithi186 ceph-mon[31505]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:18:19.596 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:19 smithi066 ceph-mon[27953]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:18:19.598 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T09:18:19.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:19 smithi083 ceph-mon[31413]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T09:18:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 -- 172.21.15.66:0/684580485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 msgr2=0x7f72c41035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 --2- 172.21.15.66:0/684580485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c41035c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009900 tx=0x7f72b402bd60 comp rx=0 tx=0).stop 2024-04-21T09:18:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 -- 172.21.15.66:0/684580485 shutdown_connections 2024-04-21T09:18:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 --2- 172.21.15.66:0/684580485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c410c520 0x7f72c410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 --2- 172.21.15.66:0/684580485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c4103b00 0x7f72c410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 --2- 172.21.15.66:0/684580485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c41035c0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 -- 172.21.15.66:0/684580485 >> 172.21.15.66:0/684580485 conn(0x7f72c4077fd0 msgr2=0x7f72c40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 -- 172.21.15.66:0/684580485 shutdown_connections 2024-04-21T09:18:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 -- 172.21.15.66:0/684580485 wait complete. 2024-04-21T09:18:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.713+0000 7f72c9e81640 1 Processor -- start 2024-04-21T09:18:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c9e81640 1 -- start start 2024-04-21T09:18:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c9e81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c9e81640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c4103b00 0x7f72c4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c9e81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c410c520 0x7f72c419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c9e81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4112210 con 0x7f72c41031e0 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c9e81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4112090 con 0x7f72c410c520 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c9e81640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4112390 con 0x7f72c4103b00 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37054/0 (socket says 172.21.15.66:37054) 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 -- 172.21.15.66:0/292936564 learned_addr learned my addr 172.21.15.66:0/292936564 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c2ffd640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c4103b00 0x7f72c4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c3fff640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c410c520 0x7f72c419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 -- 172.21.15.66:0/292936564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c4103b00 msgr2=0x7f72c4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c4103b00 0x7f72c4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 -- 172.21.15.66:0/292936564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c410c520 msgr2=0x7f72c419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c410c520 0x7f72c419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c37fe640 1 -- 172.21.15.66:0/292936564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c419d720 con 0x7f72c41031e0 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c3fff640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c410c520 0x7f72c419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.714+0000 7f72c2ffd640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c4103b00 0x7f72c4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.715+0000 7f72c37fe640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c4198730 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f72b4005940 tx=0x7f72b402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.715+0000 7f72c0ff9640 1 -- 172.21.15.66:0/292936564 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4043020 con 0x7f72c41031e0 2024-04-21T09:18:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.715+0000 7f72c0ff9640 1 -- 172.21.15.66:0/292936564 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b402dd20 con 0x7f72c41031e0 2024-04-21T09:18:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.715+0000 7f72c0ff9640 1 -- 172.21.15.66:0/292936564 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4003b80 con 0x7f72c41031e0 2024-04-21T09:18:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.715+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c419d9b0 con 0x7f72c41031e0 2024-04-21T09:18:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.715+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c41a52a0 con 0x7f72c41031e0 2024-04-21T09:18:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.716+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c404fce0 con 0x7f72c41031e0 2024-04-21T09:18:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.717+0000 7f72c0ff9640 1 -- 172.21.15.66:0/292936564 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f72b4046020 con 0x7f72c41031e0 2024-04-21T09:18:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.718+0000 7f72c0ff9640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f72ac0a3e40 0x7f72ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.718+0000 7f72c0ff9640 1 -- 172.21.15.66:0/292936564 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f72b40f2cf0 con 0x7f72c41031e0 2024-04-21T09:18:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.718+0000 7f72c2ffd640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f72ac0a3e40 0x7f72ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.718+0000 7f72c2ffd640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f72ac0a3e40 0x7f72ac0a6300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72b80058c0 tx=0x7f72b8005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.723+0000 7f72c0ff9640 1 -- 172.21.15.66:0/292936564 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72b40f7070 con 0x7f72c41031e0 2024-04-21T09:18:19.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.930+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f72c4107710 con 0x7f72ac0a3e40 2024-04-21T09:18:19.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.939+0000 7f72c0ff9640 1 -- 172.21.15.66:0/292936564 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f72b800bca0 con 0x7f72ac0a3e40 2024-04-21T09:18:19.939 INFO:teuthology.orchestra.run.smithi066.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T09:08:17", "first": "2024-04-21T09:09:00", "last": "2024-04-21T09:18:00", "last_pruned": "2024-04-21T09:18:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-21T09:18:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f72ac0a3e40 msgr2=0x7f72ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f72ac0a3e40 0x7f72ac0a6300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72b80058c0 tx=0x7f72b8005850 comp rx=0 tx=0).stop 2024-04-21T09:18:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 msgr2=0x7f72c4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c4198730 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f72b4005940 tx=0x7f72b402d920 comp rx=0 tx=0).stop 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 shutdown_connections 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f72ac0a3e40 0x7f72ac0a6300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c410c520 0x7f72c419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c4103b00 0x7f72c4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 --2- 172.21.15.66:0/292936564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72c41031e0 0x7f72c4198730 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 >> 172.21.15.66:0/292936564 conn(0x7f72c4077fd0 msgr2=0x7f72c40fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 shutdown_connections 2024-04-21T09:18:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:19.941+0000 7f72c9e81640 1 -- 172.21.15.66:0/292936564 wait complete. 2024-04-21T09:18:19.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 -- 172.21.15.66:0/2231559623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70106170 msgr2=0x7fcb7010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 --2- 172.21.15.66:0/2231559623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70106170 0x7fcb7010dd40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009ca0 tx=0x7fcb6802c0f0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 -- 172.21.15.66:0/2231559623 shutdown_connections 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 --2- 172.21.15.66:0/2231559623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70106170 0x7fcb7010dd40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 --2- 172.21.15.66:0/2231559623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb70101aa0 0x7fcb70105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 --2- 172.21.15.66:0/2231559623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb701010f0 0x7fcb701014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 -- 172.21.15.66:0/2231559623 >> 172.21.15.66:0/2231559623 conn(0x7fcb70077fd0 msgr2=0x7fcb700783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 -- 172.21.15.66:0/2231559623 shutdown_connections 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 -- 172.21.15.66:0/2231559623 wait complete. 2024-04-21T09:18:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 Processor -- start 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.055+0000 7fcb763f4640 1 -- start start 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb763f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb701010f0 0x7fcb70199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb763f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb70106170 0x7fcb70199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb763f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb70199f80 0x7fcb701a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb763f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb701110e0 con 0x7fcb701010f0 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb763f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70110f60 con 0x7fcb70106170 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb763f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70111260 con 0x7fcb70199f80 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb701010f0 0x7fcb70199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb701010f0 0x7fcb70199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37080/0 (socket says 172.21.15.66:37080) 2024-04-21T09:18:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 -- 172.21.15.66:0/1337202290 learned_addr learned my addr 172.21.15.66:0/1337202290 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb74991640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb70199f80 0x7fcb701a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6f7fe640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb70106170 0x7fcb70199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 -- 172.21.15.66:0/1337202290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb70199f80 msgr2=0x7fcb701a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb70199f80 0x7fcb701a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 -- 172.21.15.66:0/1337202290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb70106170 msgr2=0x7fcb70199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb70106170 0x7fcb70199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 -- 172.21.15.66:0/1337202290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb701a4d20 con 0x7fcb701010f0 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6f7fe640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb70106170 0x7fcb70199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6ffff640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb701010f0 0x7fcb70199500 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d770 tx=0x7fcb5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.056+0000 7fcb6d7fa640 1 -- 172.21.15.66:0/1337202290 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58017070 con 0x7fcb701010f0 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.057+0000 7fcb6d7fa640 1 -- 172.21.15.66:0/1337202290 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb580054d0 con 0x7fcb701010f0 2024-04-21T09:18:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.057+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb701a4fb0 con 0x7fcb701010f0 2024-04-21T09:18:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.057+0000 7fcb6d7fa640 1 -- 172.21.15.66:0/1337202290 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58004c30 con 0x7fcb701010f0 2024-04-21T09:18:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.057+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb701a5480 con 0x7fcb701010f0 2024-04-21T09:18:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.058+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7004fce0 con 0x7fcb701010f0 2024-04-21T09:18:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.059+0000 7fcb6d7fa640 1 -- 172.21.15.66:0/1337202290 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fcb58004dd0 con 0x7fcb701010f0 2024-04-21T09:18:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.059+0000 7fcb6d7fa640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcb5c0a3df0 0x7fcb5c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.059+0000 7fcb6f7fe640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcb5c0a3df0 0x7fcb5c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.059+0000 7fcb6d7fa640 1 -- 172.21.15.66:0/1337202290 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fcb580d0660 con 0x7fcb701010f0 2024-04-21T09:18:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.060+0000 7fcb6f7fe640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcb5c0a3df0 0x7fcb5c0a62b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcb60005fd0 tx=0x7fcb60005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.064+0000 7fcb6d7fa640 1 -- 172.21.15.66:0/1337202290 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb5809c930 con 0x7fcb701010f0 2024-04-21T09:18:20.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.271+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 --> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fcb70101aa0 con 0x7fcb5c0a3df0 2024-04-21T09:18:20.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.279+0000 7fcb6d7fa640 1 -- 172.21.15.66:0/1337202290 <== mgr.15456 v2:172.21.15.66:6800/3952680321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fcb70101aa0 con 0x7fcb5c0a3df0 2024-04-21T09:18:20.279 INFO:teuthology.orchestra.run.smithi066.stdout:/ 1m 6m3h 2024-04-21T09:18:20.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcb5c0a3df0 msgr2=0x7fcb5c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:20.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcb5c0a3df0 0x7fcb5c0a62b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcb60005fd0 tx=0x7fcb60005850 comp rx=0 tx=0).stop 2024-04-21T09:18:20.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb701010f0 msgr2=0x7fcb70199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:20.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb701010f0 0x7fcb70199500 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d770 tx=0x7fcb5800dc30 comp rx=0 tx=0).stop 2024-04-21T09:18:20.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 shutdown_connections 2024-04-21T09:18:20.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fcb5c0a3df0 0x7fcb5c0a62b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb70199f80 0x7fcb701a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb70106170 0x7fcb70199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 --2- 172.21.15.66:0/1337202290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb701010f0 0x7fcb70199500 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:20.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 >> 172.21.15.66:0/1337202290 conn(0x7fcb70077fd0 msgr2=0x7fcb700ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:20.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 shutdown_connections 2024-04-21T09:18:20.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:20.281+0000 7fcb763f4640 1 -- 172.21.15.66:0/1337202290 wait complete. 2024-04-21T09:18:20.292 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-21T09:18:20.303 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-21T09:18:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:21 smithi186 ceph-mon[31505]: from='client.18111 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:18:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:21 smithi186 ceph-mon[31505]: pgmap v315: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:18:21.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:21 smithi186 ceph-mon[31505]: from='client.18117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:18:21.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:21 smithi083 ceph-mon[31413]: from='client.18111 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:18:21.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:21 smithi083 ceph-mon[31413]: pgmap v315: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:18:21.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:21 smithi083 ceph-mon[31413]: from='client.18117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:18:21.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:21 smithi066 ceph-mon[27953]: from='client.18111 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:18:21.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:21 smithi066 ceph-mon[27953]: pgmap v315: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 5 op/s 2024-04-21T09:18:21.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:21 smithi066 ceph-mon[27953]: from='client.18117 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T09:18:23.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:23 smithi186 ceph-mon[31505]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:18:23.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:23 smithi083 ceph-mon[31413]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:18:23.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:23 smithi066 ceph-mon[27953]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T09:18:25.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:25 smithi186 ceph-mon[31505]: pgmap v317: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T09:18:25.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:25 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:18:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:25 smithi083 ceph-mon[31413]: pgmap v317: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T09:18:25.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:25 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:18:25.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:25 smithi066 ceph-mon[27953]: pgmap v317: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T09:18:25.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:25 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T09:18:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: pgmap v318: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:28.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:18:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:27 smithi186 ceph-mon[31505]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: pgmap v318: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.112 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:18:28.113 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:18:28.113 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:27 smithi083 ceph-mon[31413]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: pgmap v318: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T09:18:28.122 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:27 smithi066 ceph-mon[27953]: from='mgr.15456 ' entity='mgr.x' 2024-04-21T09:18:29.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:28 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-21T09:18:29.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:28 smithi066 ceph-mon[27953]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T09:18:29.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:28 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-21T09:18:29.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:28 smithi066 ceph-mon[27953]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-21T09:18:29.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:28 smithi066 ceph-mon[27953]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-21T09:18:29.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:28 smithi066 ceph-mon[27953]: Unable to set osd_memory_target on smithi066 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-04-21T09:18:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:28 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-21T09:18:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:28 smithi186 ceph-mon[31505]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T09:18:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:28 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-21T09:18:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:28 smithi186 ceph-mon[31505]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-21T09:18:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:28 smithi186 ceph-mon[31505]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-21T09:18:29.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:28 smithi186 ceph-mon[31505]: Unable to set osd_memory_target on smithi066 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-04-21T09:18:29.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:28 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-21T09:18:29.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:28 smithi083 ceph-mon[31413]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T09:18:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:28 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-21T09:18:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:28 smithi083 ceph-mon[31413]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-21T09:18:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:28 smithi083 ceph-mon[31413]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-21T09:18:29.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:28 smithi083 ceph-mon[31413]: Unable to set osd_memory_target on smithi066 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-04-21T09:18:29.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T09:18:29.948 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T09:18:30.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.054+0000 7f0ba67b2640 1 -- 172.21.15.66:0/488387802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0101a30 msgr2=0x7f0ba0105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.054+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/488387802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0101a30 0x7f0ba0105a70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009900 tx=0x7f0b9002bc30 comp rx=0 tx=0).stop 2024-04-21T09:18:30.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.054+0000 7f0ba67b2640 1 -- 172.21.15.66:0/488387802 shutdown_connections 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.054+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/488387802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0106150 0x7f0ba010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.055+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/488387802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0101a30 0x7f0ba0105a70 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.055+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/488387802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba0101080 0x7f0ba0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.055+0000 7f0ba67b2640 1 -- 172.21.15.66:0/488387802 >> 172.21.15.66:0/488387802 conn(0x7f0ba0078000 msgr2=0x7f0ba0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.055+0000 7f0ba67b2640 1 -- 172.21.15.66:0/488387802 shutdown_connections 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.055+0000 7f0ba67b2640 1 -- 172.21.15.66:0/488387802 wait complete. 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.055+0000 7f0ba67b2640 1 Processor -- start 2024-04-21T09:18:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.055+0000 7f0ba67b2640 1 -- start start 2024-04-21T09:18:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0ba67b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0101080 0x7f0ba0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0ba67b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0106150 0x7f0ba0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0ba67b2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba019cfd0 0x7f0ba019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0ba67b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01110e0 con 0x7f0ba0106150 2024-04-21T09:18:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0ba67b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0110f60 con 0x7f0ba0101080 2024-04-21T09:18:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0ba67b2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0111260 con 0x7f0ba019cfd0 2024-04-21T09:18:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0106150 0x7f0ba0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0106150 0x7f0ba0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47044/0 (socket says 172.21.15.66:47044) 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3620413034 learned_addr learned my addr 172.21.15.66:0/3620413034 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0101080 0x7f0ba0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba019cfd0 0x7f0ba019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3620413034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba019cfd0 msgr2=0x7f0ba019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba019cfd0 0x7f0ba019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3620413034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0101080 msgr2=0x7f0ba0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0101080 0x7f0ba0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3620413034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba019dac0 con 0x7f0ba0106150 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0106150 0x7f0ba0198ca0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0b90000c00 tx=0x7f0b9002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9d7fa640 1 -- 172.21.15.66:0/3620413034 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90043020 con 0x7f0ba0106150 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.056+0000 7f0b9d7fa640 1 -- 172.21.15.66:0/3620413034 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b9002cc60 con 0x7f0ba0106150 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.057+0000 7f0b9d7fa640 1 -- 172.21.15.66:0/3620413034 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90003c30 con 0x7f0ba0106150 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.057+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba019dd50 con 0x7f0ba0106150 2024-04-21T09:18:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.057+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01a5520 con 0x7f0ba0106150 2024-04-21T09:18:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0b9d7fa640 1 -- 172.21.15.66:0/3620413034 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0b90046020 con 0x7f0ba0106150 2024-04-21T09:18:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 Processor -- start 2024-04-21T09:18:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 -- start start 2024-04-21T09:18:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b68960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b71540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b73e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c102acd1e0 con 0x55c102b71a80 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c102a6a390 con 0x55c102b68ea0 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c102ac91a0 con 0x55c102b68580 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b73e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b73e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47060/0 (socket says 172.21.15.66:47060) 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3696656959 learned_addr learned my addr 172.21.15.66:0/3696656959 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.059+0000 7f0b9d7fa640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0b880a3d20 0x7f0b880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b68960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b71540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0b880a3d20 0x7f0b880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9d7fa640 1 -- 172.21.15.66:0/3620413034 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f0b900f2cf0 con 0x7f0ba0106150 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3696656959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 msgr2=0x55c102b68960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b68960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3696656959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 msgr2=0x55c102b71540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b71540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/3696656959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1028a97d0 con 0x55c102b71a80 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b73e70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0b900be6d0 tx=0x7f0b900ba670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b6f326cf50ae3b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0b880a3d20 0x7f0b880a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005c40 tx=0x7f0b98009940 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b867fc640 1 -- 172.21.15.66:0/3696656959 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900be510 con 0x55c102b71a80 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b867fc640 1 -- 172.21.15.66:0/3696656959 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b900055b0 con 0x55c102b71a80 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.060+0000 7f0b867fc640 1 -- 172.21.15.66:0/3696656959 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900be330 con 0x55c102b71a80 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 -- 172.21.15.66:0/3696656959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 msgr2=0x55c102b73e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b73e70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0b900be6d0 tx=0x7f0b900ba670 comp rx=0 tx=0).stop 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 -- 172.21.15.66:0/3696656959 shutdown_connections 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b73e70 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b71540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/3696656959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b68960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 -- 172.21.15.66:0/3696656959 >> 172.21.15.66:0/3696656959 conn(0x55c102b62cb0 msgr2=0x55c102b63090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 -- 172.21.15.66:0/3696656959 shutdown_connections 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 -- 172.21.15.66:0/3696656959 wait complete. 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.061+0000 7f0ba93a2740 1 -- start start 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 1 Processor -- start 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b6ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b6ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b6f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c102acd1e0 con 0x55c102b71a80 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c102a862f0 con 0x55c102b68ea0 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c102b74630 con 0x55c102b68580 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba93a2740 10 client.0 _ll_register_callbacks cb 0x55c102a93580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b6f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b6f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47068/0 (socket says 172.21.15.66:47068) 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 -- 172.21.15.66:0/987888833 learned_addr learned my addr 172.21.15.66:0/987888833 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b6ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b6ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 -- 172.21.15.66:0/987888833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 msgr2=0x55c102b6ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b6ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 -- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 msgr2=0x55c102b6ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b6ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0ba4d4f640 1 -- 172.21.15.66:0/987888833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1027df3f0 con 0x55c102b71a80 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.062+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b6ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b6f4c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0b98012390 tx=0x7f0b980096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28387a1706cdbc34 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0b7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/987888833 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b98011b70 con 0x55c102b71a80 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/987888833 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f0b98011d10 con 0x55c102b71a80 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c102adac00 con 0x55c102b71a80 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/987888833 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b98032050 con 0x55c102b71a80 2024-04-21T09:18:30.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 10 client.18129 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c10279f0f0 con 0x55c102b71a80 2024-04-21T09:18:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/987888833 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c102adac00 con 0x55c102b71a80 2024-04-21T09:18:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/987888833 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0b9801f070 con 0x55c102b71a80 2024-04-21T09:18:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 10 client.18129 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:18:30.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 20 client.18129 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102b81740 0x55c102b83ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 4 client.18129 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.063+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c102ad4a70 con 0x55c102b81740 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.064+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102b81740 0x55c102b83ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.064+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102b81740 0x55c102b83ba0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6be473fd1797334 server_cookie=6ec6286e6bf9f550 in_seq=0 out_seq=0 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.064+0000 7f0b7f7fe640 10 client.18129 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/987888833 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c102ad4a70 con 0x55c102b81740 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0b7f7fe640 10 client.18129 handle_command_reply: tid=0 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 client.18129 shutdown 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 msgr2=0x55c102b6f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b6f4c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0b98012390 tx=0x7f0b980096c0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 shutdown_connections 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102b81740 0x55c102b83ba0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b71a80 0x55c102b6f4c0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0b7f7fe640 0 client.18129 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68ea0 0x55c102b6ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/987888833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68580 0x55c102b6ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 >> 172.21.15.66:0/987888833 conn(0x55c102b62cb0 msgr2=0x55c102b64a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 shutdown_connections 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 1 -- 172.21.15.66:0/987888833 wait complete. 2024-04-21T09:18:30.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.065+0000 7f0ba93a2740 20 client.18129 trim_cache size 0 max 16384 2024-04-21T09:18:30.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:29 smithi186 ceph-mon[31505]: pgmap v319: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T09:18:30.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 Processor -- start 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 -- start start 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68580 0x55c102b71170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68ea0 0x55c102b7d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b71a80 0x55c102b6dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b1c00 con 0x55c102b68ea0 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01af4d0 con 0x55c102b68580 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ad340 con 0x55c102b71a80 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68ea0 0x55c102b7d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68ea0 0x55c102b7d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47070/0 (socket says 172.21.15.66:47070) 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/801797098 learned_addr learned my addr 172.21.15.66:0/801797098 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68580 0x55c102b71170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/801797098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b71a80 msgr2=0x55c102b6dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b71a80 0x55c102b6dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/801797098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68580 msgr2=0x55c102b71170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68580 0x55c102b71170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.083+0000 7f0b9f7fe640 1 -- 172.21.15.66:0/801797098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c102b61bf0 con 0x55c102b68ea0 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.084+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68ea0 0x55c102b7d370 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0b900bb3a0 tx=0x7f0b900092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f72675a01f6461b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.084+0000 7f0b85ffb640 1 -- 172.21.15.66:0/801797098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900d7020 con 0x55c102b68ea0 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.084+0000 7f0b85ffb640 1 -- 172.21.15.66:0/801797098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b900bf0d0 con 0x55c102b68ea0 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.084+0000 7f0b85ffb640 1 -- 172.21.15.66:0/801797098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900f0260 con 0x55c102b68ea0 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.084+0000 7f0ba93a2740 1 -- 172.21.15.66:0/801797098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68ea0 msgr2=0x55c102b7d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.084+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68ea0 0x55c102b7d370 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0b900bb3a0 tx=0x7f0b900092e0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- 172.21.15.66:0/801797098 shutdown_connections 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b71a80 0x55c102b6dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68ea0 0x55c102b7d370 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/801797098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b68580 0x55c102b71170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- 172.21.15.66:0/801797098 >> 172.21.15.66:0/801797098 conn(0x55c102b62cb0 msgr2=0x55c102b614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- 172.21.15.66:0/801797098 shutdown_connections 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- 172.21.15.66:0/801797098 wait complete. 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- start start 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 Processor -- start 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68580 0x55c102b79ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68ea0 0x55c102b74e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b71a80 0x55c102b75360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ad990 con 0x55c102b68580 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01aee00 con 0x55c102b71a80 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ae060 con 0x55c102b68ea0 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.085+0000 7f0ba93a2740 10 client.0 _ll_register_callbacks cb 0x7f0b98012030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68580 0x55c102b79ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68580 0x55c102b79ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47084/0 (socket says 172.21.15.66:47084) 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 -- 172.21.15.66:0/875771723 learned_addr learned my addr 172.21.15.66:0/875771723 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68ea0 0x55c102b74e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0ba4d4f640 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b71a80 0x55c102b75360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 -- 172.21.15.66:0/875771723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68ea0 msgr2=0x55c102b74e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68ea0 0x55c102b74e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 -- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b71a80 msgr2=0x55c102b75360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b71a80 0x55c102b75360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 -- 172.21.15.66:0/875771723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c102b75920 con 0x55c102b68580 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b9ffff640 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68580 0x55c102b79ad0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0b94004700 tx=0x7f0b94010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27746b1ffab3fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/875771723 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94004220 con 0x55c102b68580 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/875771723 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f0b940043c0 con 0x55c102b68580 2024-04-21T09:18:30.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b900bf0d0 con 0x55c102b68580 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/875771723 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b94030050 con 0x55c102b68580 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0ba93a2740 10 client.18135 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.086+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c102be7b40 con 0x55c102b68580 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/875771723 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b900bf0d0 con 0x55c102b68580 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/875771723 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0b940035f0 con 0x55c102b68580 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0ba93a2740 10 client.18135 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0ba93a2740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0ba93a2740 20 client.18135 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102be82c0 0x55c102bea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0ba93a2740 4 client.18135 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c102a50590 con 0x55c102be82c0 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102be82c0 0x55c102bea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0b7f7fe640 10 client.18135 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.087+0000 7f0b9f7fe640 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102be82c0 0x55c102bea6e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4373b531b560929 server_cookie=a59ec6fd1fbbb335 in_seq=0 out_seq=0 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0b7f7fe640 1 -- 172.21.15.66:0/875771723 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c102a50590 con 0x55c102be82c0 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0b7f7fe640 10 client.18135 handle_command_reply: tid=0 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 client.18135 shutdown 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68580 msgr2=0x55c102b79ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68580 0x55c102b79ad0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0b94004700 tx=0x7f0b94010ad0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 shutdown_connections 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x55c102be82c0 0x55c102bea6e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c102b71a80 0x55c102b75360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c102b68ea0 0x55c102b74e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0b7f7fe640 0 client.18135 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 --2- 172.21.15.66:0/875771723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c102b68580 0x55c102b79ad0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 >> 172.21.15.66:0/875771723 conn(0x55c102b62cb0 msgr2=0x55c102b6dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 shutdown_connections 2024-04-21T09:18:30.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 1 -- 172.21.15.66:0/875771723 wait complete. 2024-04-21T09:18:30.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.088+0000 7f0ba93a2740 20 client.18135 trim_cache size 0 max 16384 2024-04-21T09:18:30.096 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:18:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-04-21T09:18:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-04-21T09:18:30.096 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0b880a3d20 msgr2=0x7f0b880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0b880a3d20 0x7f0b880a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005c40 tx=0x7f0b98009940 comp rx=0 tx=0).stop 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0106150 msgr2=0x7f0ba0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0106150 0x7f0ba0198ca0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0b90000c00 tx=0x7f0b9002c860 comp rx=0 tx=0).stop 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 shutdown_connections 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0b880a3d20 0x7f0b880a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba019cfd0 0x7f0ba019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0106150 0x7f0ba0198ca0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 --2- 172.21.15.66:0/3620413034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0101080 0x7f0ba0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 >> 172.21.15.66:0/3620413034 conn(0x7f0ba0078000 msgr2=0x7f0ba0069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 shutdown_connections 2024-04-21T09:18:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.089+0000 7f0ba67b2640 1 -- 172.21.15.66:0/3620413034 wait complete. 2024-04-21T09:18:30.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:29 smithi066 ceph-mon[27953]: pgmap v319: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T09:18:30.100 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T09:18:30.100 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:18:30.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:29 smithi083 ceph-mon[31413]: pgmap v319: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T09:18:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/3716953651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d41011a0 msgr2=0x7fd7d4101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/3716953651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d41011a0 0x7fd7d4101620 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009930 tx=0x7fd7c402bc30 comp rx=0 tx=0).stop 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/3716953651 shutdown_connections 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/3716953651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d4101b60 0x7fd7d410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/3716953651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d41011a0 0x7fd7d4101620 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/3716953651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7d4105e30 0x7fd7d4106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/3716953651 >> 172.21.15.66:0/3716953651 conn(0x7fd7d4077e30 msgr2=0x7fd7d4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/3716953651 shutdown_connections 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.202+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/3716953651 wait complete. 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 Processor -- start 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 -- start start 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d41011a0 0x7fd7d4102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7d4101b60 0x7fd7d4103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d4105e30 0x7fd7d4103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41110c0 con 0x7fd7d4105e30 2024-04-21T09:18:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4110f40 con 0x7fd7d4101b60 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d8ec1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4111240 con 0x7fd7d41011a0 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d4105e30 0x7fd7d4103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d4105e30 0x7fd7d4103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47102/0 (socket says 172.21.15.66:47102) 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 -- 172.21.15.66:0/2946146916 learned_addr learned my addr 172.21.15.66:0/2946146916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d1d9b640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7d4101b60 0x7fd7d4103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d259c640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d41011a0 0x7fd7d4102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 -- 172.21.15.66:0/2946146916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d41011a0 msgr2=0x7fd7d4102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d41011a0 0x7fd7d4102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 -- 172.21.15.66:0/2946146916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7d4101b60 msgr2=0x7fd7d4103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7d4101b60 0x7fd7d4103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d2d9d640 1 -- 172.21.15.66:0/2946146916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d41a6b70 con 0x7fd7d4105e30 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.203+0000 7fd7d1d9b640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7d4101b60 0x7fd7d4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.204+0000 7fd7d2d9d640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d4105e30 0x7fd7d4103550 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00cce0 tx=0x7fd7cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.204+0000 7fd7cb7fe640 1 -- 172.21.15.66:0/2946146916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc00f450 con 0x7fd7d4105e30 2024-04-21T09:18:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.204+0000 7fd7cb7fe640 1 -- 172.21.15.66:0/2946146916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc00faa0 con 0x7fd7d4105e30 2024-04-21T09:18:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.204+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d41a6e60 con 0x7fd7d4105e30 2024-04-21T09:18:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.204+0000 7fd7cb7fe640 1 -- 172.21.15.66:0/2946146916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc018b70 con 0x7fd7d4105e30 2024-04-21T09:18:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.204+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d41a7330 con 0x7fd7d4105e30 2024-04-21T09:18:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.205+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d4068de0 con 0x7fd7d4105e30 2024-04-21T09:18:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.206+0000 7fd7cb7fe640 1 -- 172.21.15.66:0/2946146916 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd7cc018d10 con 0x7fd7d4105e30 2024-04-21T09:18:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.207+0000 7fd7cb7fe640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd7c00a3d50 0x7fd7c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.207+0000 7fd7cb7fe640 1 -- 172.21.15.66:0/2946146916 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fd7cc0d0f10 con 0x7fd7d4105e30 2024-04-21T09:18:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.207+0000 7fd7d259c640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd7c00a3d50 0x7fd7c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.207+0000 7fd7d259c640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd7c00a3d50 0x7fd7c00a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00bb60 tx=0x7fd7bc00b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.212+0000 7fd7cb7fe640 1 -- 172.21.15.66:0/2946146916 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7cc014030 con 0x7fd7d4105e30 2024-04-21T09:18:30.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.440+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7d41a75c0 con 0x7fd7d4105e30 2024-04-21T09:18:30.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.440+0000 7fd7cb7fe640 1 -- 172.21.15.66:0/2946146916 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7fd7cc09d1e0 con 0x7fd7d4105e30 2024-04-21T09:18:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:18:30.441 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:18:30.441 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd7c00a3d50 msgr2=0x7fd7c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd7c00a3d50 0x7fd7c00a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00bb60 tx=0x7fd7bc00b390 comp rx=0 tx=0).stop 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d4105e30 msgr2=0x7fd7d4103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d4105e30 0x7fd7d4103550 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00cce0 tx=0x7fd7cc00a620 comp rx=0 tx=0).stop 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 shutdown_connections 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fd7c00a3d50 0x7fd7c00a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7d4105e30 0x7fd7d4103550 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7d4101b60 0x7fd7d4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 --2- 172.21.15.66:0/2946146916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d41011a0 0x7fd7d4102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.442+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 >> 172.21.15.66:0/2946146916 conn(0x7fd7d4077e30 msgr2=0x7fd7d4104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.443+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 shutdown_connections 2024-04-21T09:18:30.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.443+0000 7fd7d8ec1640 1 -- 172.21.15.66:0/2946146916 wait complete. 2024-04-21T09:18:30.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T09:18:30.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.557+0000 7f3d1ef96640 1 -- 172.21.15.66:0/2991409240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d1810c510 msgr2=0x7f3d1810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.557+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/2991409240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d1810c510 0x7f3d1810e9b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009ca0 tx=0x7f3d1002c230 comp rx=0 tx=0).stop 2024-04-21T09:18:30.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- 172.21.15.66:0/2991409240 shutdown_connections 2024-04-21T09:18:30.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/2991409240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d1810c510 0x7f3d1810e9b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/2991409240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d18103af0 0x7f3d1810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/2991409240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d181031d0 0x7f3d181035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- 172.21.15.66:0/2991409240 >> 172.21.15.66:0/2991409240 conn(0x7f3d18077f50 msgr2=0x7f3d18078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- 172.21.15.66:0/2991409240 shutdown_connections 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- 172.21.15.66:0/2991409240 wait complete. 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 Processor -- start 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- start start 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181031d0 0x7f3d181986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d18103af0 0x7f3d18198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1810c510 0x7f3d1819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18112180 con 0x7f3d181031d0 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18112000 con 0x7f3d1810c510 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1ef96640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18112300 con 0x7f3d18103af0 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1cd32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181031d0 0x7f3d181986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1cd32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181031d0 0x7f3d181986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47118/0 (socket says 172.21.15.66:47118) 2024-04-21T09:18:30.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.558+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3355565393 learned_addr learned my addr 172.21.15.66:0/3355565393 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d18103af0 0x7f3d18198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1d533640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1810c510 0x7f3d1819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3355565393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d18103af0 msgr2=0x7f3d18198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d18103af0 0x7f3d18198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3355565393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1810c510 msgr2=0x7f3d1819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1810c510 0x7f3d1819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3355565393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1819d6d0 con 0x7f3d181031d0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d18103af0 0x7f3d18198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181031d0 0x7f3d181986e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800d950 tx=0x7f3d0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d15ffb640 1 -- 172.21.15.66:0/3355565393 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08017030 con 0x7f3d181031d0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d15ffb640 1 -- 172.21.15.66:0/3355565393 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d08004020 con 0x7f3d181031d0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d15ffb640 1 -- 172.21.15.66:0/3355565393 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d080047a0 con 0x7f3d181031d0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1819d9c0 con 0x7f3d181031d0 2024-04-21T09:18:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.559+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d181a5250 con 0x7f3d181031d0 2024-04-21T09:18:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.561+0000 7f3d15ffb640 1 -- 172.21.15.66:0/3355565393 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3d08009a30 con 0x7f3d181031d0 2024-04-21T09:18:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 Processor -- start 2024-04-21T09:18:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 -- start start 2024-04-21T09:18:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc17790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc20890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56170cb7c1f0 con 0x56170cc173b0 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56170cb194b0 con 0x56170cc20e60 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d21b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56170cb781b0 con 0x56170cc17d60 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d15ffb640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f3d000a3d20 0x7f3d000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc17790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc17790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47130/0 (socket says 172.21.15.66:47130) 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 -- 172.21.15.66:0/2788483317 learned_addr learned my addr 172.21.15.66:0/2788483317 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d17fff640 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f3d000a3d20 0x7f3d000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d15ffb640 1 -- 172.21.15.66:0/3355565393 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f3d080d06e0 con 0x7f3d181031d0 2024-04-21T09:18:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc20890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 -- 172.21.15.66:0/2788483317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 msgr2=0x56170cc20890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc20890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 -- 172.21.15.66:0/2788483317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 msgr2=0x56170cc212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.562+0000 7f3d1d533640 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.563+0000 7f3d1d533640 1 -- 172.21.15.66:0/2788483317 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170cb0b1b0 con 0x56170cc173b0 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.563+0000 7f3d17fff640 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.563+0000 7f3d1d533640 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc17790 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f3d10004af0 tx=0x7f3d10003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc507c2adefd737 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.563+0000 7f3d1d533640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f3d000a3d20 0x7f3d000a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009a60 tx=0x7f3d10041000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.563+0000 7f3cfeffd640 1 -- 172.21.15.66:0/2788483317 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10051070 con 0x56170cc173b0 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.563+0000 7f3cfeffd640 1 -- 172.21.15.66:0/2788483317 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d10048070 con 0x56170cc173b0 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.563+0000 7f3cfeffd640 1 -- 172.21.15.66:0/2788483317 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10039040 con 0x56170cc173b0 2024-04-21T09:18:30.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 -- 172.21.15.66:0/2788483317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 msgr2=0x56170cc17790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc17790 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f3d10004af0 tx=0x7f3d10003ec0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 -- 172.21.15.66:0/2788483317 shutdown_connections 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc20890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 --2- 172.21.15.66:0/2788483317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc17790 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 -- 172.21.15.66:0/2788483317 >> 172.21.15.66:0/2788483317 conn(0x56170cc11ba0 msgr2=0x56170cc11f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 -- 172.21.15.66:0/2788483317 shutdown_connections 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 -- 172.21.15.66:0/2788483317 wait complete. 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 -- start start 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.564+0000 7f3d21b86740 1 Processor -- start 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc27250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc24b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56170cc27d40 con 0x56170cc173b0 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56170cb35500 con 0x56170cc20e60 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56170cc27ec0 con 0x56170cc17d60 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 10 client.0 _ll_register_callbacks cb 0x56170cb03b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc27250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc27250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47132/0 (socket says 172.21.15.66:47132) 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3958199214 learned_addr learned my addr 172.21.15.66:0/3958199214 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3958199214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 msgr2=0x56170cc245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1d533640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc24b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc245e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 msgr2=0x56170cc24b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc24b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3958199214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56170c88e3f0 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1d533640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc24b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc27250 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3d08098c80 tx=0x7f3d0809beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254afaf0261602ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3cf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3958199214 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d080b5060 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3958199214 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f3d080cdce0 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d10048070 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3958199214 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d0809c2b0 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 10 client.18153 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.565+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56170c9587d0 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3958199214 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d10048070 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3958199214 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3d08098920 con 0x56170cc173b0 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d21b86740 10 client.18153 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d21b86740 10 client.18153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:18:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d21b86740 20 client.18153 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc35020 0x56170cc37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d21b86740 4 client.18153 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56170cb1b930 con 0x56170cc35020 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc35020 0x56170cc37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3cf7fff640 10 client.18153 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.566+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc35020 0x56170cc37470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d5dbe72958f5d06 server_cookie=b00a4120a24d36a8 in_seq=0 out_seq=0 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3958199214 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56170cb1b930 con 0x56170cc35020 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3cf7fff640 10 client.18153 handle_command_reply: tid=0 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 client.18153 shutdown 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 msgr2=0x56170cc27250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc27250 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3d08098c80 tx=0x7f3d0809beb0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 shutdown_connections 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc35020 0x56170cc37470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc20e60 0x56170cc24b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3958199214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc27250 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3cf7fff640 0 client.18153 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:18:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.567+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 >> 172.21.15.66:0/3958199214 conn(0x56170cc11ba0 msgr2=0x56170cc138f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.568+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 shutdown_connections 2024-04-21T09:18:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.568+0000 7f3d21b86740 1 -- 172.21.15.66:0/3958199214 wait complete. 2024-04-21T09:18:30.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.568+0000 7f3d21b86740 20 client.18153 trim_cache size 0 max 16384 2024-04-21T09:18:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 Processor -- start 2024-04-21T09:18:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 -- start start 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc173b0 0x56170cc22250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc20e60 0x56170cc1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d181afb40 con 0x56170cc20e60 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d181ad410 con 0x56170cc173b0 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d21b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3d181ab280 con 0x56170cc17d60 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc20e60 0x56170cc1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc20e60 0x56170cc1f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47144/0 (socket says 172.21.15.66:47144) 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 -- 172.21.15.66:0/1922828153 learned_addr learned my addr 172.21.15.66:0/1922828153 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc173b0 0x56170cc22250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d17fff640 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 -- 172.21.15.66:0/1922828153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 msgr2=0x56170cc30c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc30c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 -- 172.21.15.66:0/1922828153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc173b0 msgr2=0x56170cc22250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc173b0 0x56170cc22250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 -- 172.21.15.66:0/1922828153 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170cc22b50 con 0x56170cc20e60 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.581+0000 7f3d1d533640 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc20e60 0x56170cc1f800 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d1002c770 tx=0x7f3d10004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69aa8fc8d2e56ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3cfe7fc640 1 -- 172.21.15.66:0/1922828153 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10051070 con 0x56170cc20e60 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3cfe7fc640 1 -- 172.21.15.66:0/1922828153 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d10048040 con 0x56170cc20e60 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3cfe7fc640 1 -- 172.21.15.66:0/1922828153 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10039050 con 0x56170cc20e60 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 -- 172.21.15.66:0/1922828153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc20e60 msgr2=0x56170cc1f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc20e60 0x56170cc1f800 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d1002c770 tx=0x7f3d10004cd0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 -- 172.21.15.66:0/1922828153 shutdown_connections 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc20e60 0x56170cc1f800 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc17d60 0x56170cc30c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 --2- 172.21.15.66:0/1922828153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc173b0 0x56170cc22250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 -- 172.21.15.66:0/1922828153 >> 172.21.15.66:0/1922828153 conn(0x56170cc11ba0 msgr2=0x56170cc245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 -- 172.21.15.66:0/1922828153 shutdown_connections 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 -- 172.21.15.66:0/1922828153 wait complete. 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.582+0000 7f3d21b86740 1 -- start start 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 1 Processor -- start 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc2d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc17d60 0x56170cc2d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc20e60 0x56170cc2a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d181ab8d0 con 0x56170cc173b0 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d181acd40 con 0x56170cc17d60 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3d181abfa0 con 0x56170cc20e60 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d21b86740 10 client.0 _ll_register_callbacks cb 0x7f3d0809c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc2d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc2d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47154/0 (socket says 172.21.15.66:47154) 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3831128861 learned_addr learned my addr 172.21.15.66:0/3831128861 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc17d60 0x56170cc2d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3831128861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc20e60 msgr2=0x56170cc2a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc20e60 0x56170cc2a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc17d60 msgr2=0x56170cc2d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc17d60 0x56170cc2d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 -- 172.21.15.66:0/3831128861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56170cc2a890 con 0x56170cc173b0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3d1cd32640 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc2d180 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005fd0 tx=0x7f3d08009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ca26c3fc48df1b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.583+0000 7f3cf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3831128861 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0807d020 con 0x56170cc173b0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3831128861 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f3d080a0170 con 0x56170cc173b0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d10048040 con 0x56170cc173b0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3831128861 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d08098550 con 0x56170cc173b0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 10 client.18159 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56170cc9bfc0 con 0x56170cc173b0 2024-04-21T09:18:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3831128861 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d10048040 con 0x56170cc173b0 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3831128861 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3d08098000 con 0x56170cc173b0 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 10 client.18159 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 10 client.18159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 20 client.18159 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc9c570 0x56170cc9e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 4 client.18159 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 --> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56170cb89c10 con 0x56170cc9c570 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.584+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc9c570 0x56170cc9e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3cf7fff640 10 client.18159 ms_handle_connect on v2:172.21.15.83:6836/1568837633 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d17fff640 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc9c570 0x56170cc9e990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b34bbac3949dfc server_cookie=a2918b28f84ae8cc in_seq=0 out_seq=0 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3cf7fff640 1 -- 172.21.15.66:0/3831128861 <== mds.2 v2:172.21.15.83:6836/1568837633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56170cb89c10 con 0x56170cc9c570 2024-04-21T09:18:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3cf7fff640 10 client.18159 handle_command_reply: tid=0 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 client.18159 shutdown 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 msgr2=0x56170cc2d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc2d180 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005fd0 tx=0x7f3d08009f30 comp rx=0 tx=0).stop 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 shutdown_connections 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:6836/1568837633,v1:172.21.15.83:6837/1568837633] conn(0x56170cc9c570 0x56170cc9e990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56170cc20e60 0x56170cc2a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56170cc17d60 0x56170cc2d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 --2- 172.21.15.66:0/3831128861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56170cc173b0 0x56170cc2d180 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3cf7fff640 0 client.18159 ms_handle_reset on v2:172.21.15.83:6836/1568837633 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 >> 172.21.15.66:0/3831128861 conn(0x56170cc11ba0 msgr2=0x56170cc10370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 shutdown_connections 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 1 -- 172.21.15.66:0/3831128861 wait complete. 2024-04-21T09:18:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.585+0000 7f3d21b86740 20 client.18159 trim_cache size 0 max 16384 2024-04-21T09:18:30.593 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:18:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f3d000a3d20 msgr2=0x7f3d000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f3d000a3d20 0x7f3d000a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009a60 tx=0x7f3d10041000 comp rx=0 tx=0).stop 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181031d0 msgr2=0x7f3d181986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181031d0 0x7f3d181986e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800d950 tx=0x7f3d0800de10 comp rx=0 tx=0).stop 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 shutdown_connections 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f3d000a3d20 0x7f3d000a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1810c510 0x7f3d1819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d18103af0 0x7f3d18198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 --2- 172.21.15.66:0/3355565393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d181031d0 0x7f3d181986e0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 >> 172.21.15.66:0/3355565393 conn(0x7f3d18077f50 msgr2=0x7f3d180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 shutdown_connections 2024-04-21T09:18:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.586+0000 7f3d1ef96640 1 -- 172.21.15.66:0/3355565393 wait complete. 2024-04-21T09:18:30.597 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T09:18:30.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 -- 172.21.15.66:0/2035018394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a406a840 msgr2=0x7f84a406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 --2- 172.21.15.66:0/2035018394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a406a840 0x7f84a406ac20 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f848c009900 tx=0x7f848c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 -- 172.21.15.66:0/2035018394 shutdown_connections 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 --2- 172.21.15.66:0/2035018394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a4107270 0x7f84a410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 --2- 172.21.15.66:0/2035018394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406b1f0 0x7f84a4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 --2- 172.21.15.66:0/2035018394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a406a840 0x7f84a406ac20 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 -- 172.21.15.66:0/2035018394 >> 172.21.15.66:0/2035018394 conn(0x7f84a4077eb0 msgr2=0x7f84a40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 -- 172.21.15.66:0/2035018394 shutdown_connections 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.700+0000 7f84aa732640 1 -- 172.21.15.66:0/2035018394 wait complete. 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 Processor -- start 2024-04-21T09:18:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 -- start start 2024-04-21T09:18:30.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a406a840 0x7f84a4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a4107270 0x7f84a4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a419d010 0x7f84a419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4112230 con 0x7f84a4107270 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41120b0 con 0x7f84a419d010 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.701+0000 7f84aa732640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41123b0 con 0x7f84a406a840 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a4107270 0x7f84a4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a4107270 0x7f84a4198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47174/0 (socket says 172.21.15.66:47174) 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 -- 172.21.15.66:0/3635076845 learned_addr learned my addr 172.21.15.66:0/3635076845 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a3fff640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a406a840 0x7f84a4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a419d010 0x7f84a419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 -- 172.21.15.66:0/3635076845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a406a840 msgr2=0x7f84a4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a406a840 0x7f84a4198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 -- 172.21.15.66:0/3635076845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a419d010 msgr2=0x7f84a419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a419d010 0x7f84a419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 -- 172.21.15.66:0/3635076845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a419db00 con 0x7f84a4107270 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a419d010 0x7f84a419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a37fe640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a4107270 0x7f84a4198c90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f849400d770 tx=0x7f849400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a17fa640 1 -- 172.21.15.66:0/3635076845 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494017070 con 0x7f84a4107270 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a17fa640 1 -- 172.21.15.66:0/3635076845 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8494009c30 con 0x7f84a4107270 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84a17fa640 1 -- 172.21.15.66:0/3635076845 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849400f620 con 0x7f84a4107270 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419ddf0 con 0x7f84a4107270 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.702+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a5630 con 0x7f84a4107270 2024-04-21T09:18:30.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.705+0000 7f84a17fa640 1 -- 172.21.15.66:0/3635076845 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f849401a070 con 0x7f84a4107270 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.705+0000 7f84ad401740 1 Processor -- start 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.705+0000 7f84ad401740 1 -- start start 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84ad401740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb49790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84ad401740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb52890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84ad401740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84ad401740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cf4baae1f0 con 0x55cf4bb493b0 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84ad401740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf4ba4b4b0 con 0x55cf4bb52e60 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84ad401740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cf4baaa1b0 con 0x55cf4bb49d60 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb49790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb49790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47190/0 (socket says 172.21.15.66:47190) 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 -- 172.21.15.66:0/669957122 learned_addr learned my addr 172.21.15.66:0/669957122 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a17fa640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f84900a3da0 0x7f84900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a3fff640 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb52890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a37fe640 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 -- 172.21.15.66:0/669957122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 msgr2=0x55cf4bb52890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb52890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 -- 172.21.15.66:0/669957122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 msgr2=0x55cf4bb532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 -- 172.21.15.66:0/669957122 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf4ba3d1b0 con 0x55cf4bb493b0 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a37fe640 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a3fff640 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb52890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a17fa640 1 -- 172.21.15.66:0/3635076845 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f84940d06c0 con 0x7f84a4107270 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f84900a3da0 0x7f84900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb49790 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c550 tx=0x7f849c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72c324f7087f226 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f849a7fc640 1 -- 172.21.15.66:0/669957122 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c00a600 con 0x55cf4bb493b0 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f849a7fc640 1 -- 172.21.15.66:0/669957122 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f849c00ac50 con 0x55cf4bb493b0 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.706+0000 7f849a7fc640 1 -- 172.21.15.66:0/669957122 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0179e0 con 0x55cf4bb493b0 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 -- 172.21.15.66:0/669957122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 msgr2=0x55cf4bb49790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb49790 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c550 tx=0x7f849c00ca10 comp rx=0 tx=0).stop 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f84900a3da0 0x7f84900a6260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f849c01f000 tx=0x7f849c018000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 -- 172.21.15.66:0/669957122 shutdown_connections 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb52890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 --2- 172.21.15.66:0/669957122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb49790 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 -- 172.21.15.66:0/669957122 >> 172.21.15.66:0/669957122 conn(0x55cf4bb43ba0 msgr2=0x55cf4bb43f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 -- 172.21.15.66:0/669957122 shutdown_connections 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 -- 172.21.15.66:0/669957122 wait complete. 2024-04-21T09:18:30.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.707+0000 7f84ad401740 1 -- start start 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 1 Processor -- start 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb493b0 0x55cf4bb59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb49d60 0x55cf4bb56600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb56b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cf4bb59da0 con 0x55cf4bb49d60 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf4ba67500 con 0x55cf4bb493b0 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cf4bb59f20 con 0x55cf4bb52e60 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84ad401740 10 client.0 _ll_register_callbacks cb 0x55cf4ba35b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb49d60 0x55cf4bb56600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb49d60 0x55cf4bb56600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47192/0 (socket says 172.21.15.66:47192) 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 -- 172.21.15.66:0/483654739 learned_addr learned my addr 172.21.15.66:0/483654739 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a3fff640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb493b0 0x55cf4bb59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb56b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 -- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 msgr2=0x55cf4bb56b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb56b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 -- 172.21.15.66:0/483654739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb493b0 msgr2=0x55cf4bb59270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb493b0 0x55cf4bb59270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a37fe640 1 -- 172.21.15.66:0/483654739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4b7c03f0 con 0x55cf4bb49d60 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.708+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb56b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84a37fe640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb49d60 0x55cf4bb56600 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8494009610 tx=0x7f849409c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4449734cd33cc641 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84837fe640 1 -- 172.21.15.66:0/483654739 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494021040 con 0x55cf4bb49d60 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84837fe640 1 -- 172.21.15.66:0/483654739 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8494098900 con 0x55cf4bb49d60 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849c00ac50 con 0x55cf4bb49d60 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84837fe640 1 -- 172.21.15.66:0/483654739 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84940a0640 con 0x55cf4bb49d60 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 10 client.18171 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf4b88a7d0 con 0x55cf4bb49d60 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84837fe640 1 -- 172.21.15.66:0/483654739 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f849c00ac50 con 0x55cf4bb49d60 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84837fe640 1 -- 172.21.15.66:0/483654739 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f84940d0b70 con 0x55cf4bb49d60 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 10 client.18171 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 10 client.18171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 20 client.18171 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bb66f30 0x55cf4bb69340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 4 client.18171 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.709+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf4ba4d930 con 0x55cf4bb66f30 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.710+0000 7f84a3fff640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bb66f30 0x55cf4bb69340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.710+0000 7f84837fe640 10 client.18171 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.710+0000 7f84a3fff640 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bb66f30 0x55cf4bb69340 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88bd95410a2fbf98 server_cookie=842979046cec5a43 in_seq=0 out_seq=0 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84837fe640 1 -- 172.21.15.66:0/483654739 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf4ba4d930 con 0x55cf4bb66f30 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84837fe640 10 client.18171 handle_command_reply: tid=0 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 client.18171 shutdown 2024-04-21T09:18:30.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb49d60 msgr2=0x55cf4bb56600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb49d60 0x55cf4bb56600 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8494009610 tx=0x7f849409c990 comp rx=0 tx=0).stop 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 shutdown_connections 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb56b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bb66f30 0x55cf4bb69340 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb49d60 0x55cf4bb56600 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 --2- 172.21.15.66:0/483654739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb493b0 0x55cf4bb59270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84837fe640 0 client.18171 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 >> 172.21.15.66:0/483654739 conn(0x55cf4bb43ba0 msgr2=0x55cf4bb458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 shutdown_connections 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 1 -- 172.21.15.66:0/483654739 wait complete. 2024-04-21T09:18:30.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.711+0000 7f84ad401740 20 client.18171 trim_cache size 0 max 16384 2024-04-21T09:18:30.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.724+0000 7f84ad401740 1 Processor -- start 2024-04-21T09:18:30.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.724+0000 7f84ad401740 1 -- start start 2024-04-21T09:18:30.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.724+0000 7f84ad401740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb62ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.724+0000 7f84ad401740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb49d60 0x55cf4bb524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84ad401740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84ad401740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41afa50 con 0x55cf4bb493b0 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84ad401740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41ad320 con 0x55cf4bb49d60 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84ad401740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41ab190 con 0x55cf4bb52e60 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb62ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb62ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47198/0 (socket says 172.21.15.66:47198) 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 -- 172.21.15.66:0/2871607577 learned_addr learned my addr 172.21.15.66:0/2871607577 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a37fe640 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb49d60 0x55cf4bb524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 -- 172.21.15.66:0/2871607577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 msgr2=0x55cf4bb58d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 -- 172.21.15.66:0/2871607577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb49d60 msgr2=0x55cf4bb524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb49d60 0x55cf4bb524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 -- 172.21.15.66:0/2871607577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf4bb436d0 con 0x55cf4bb493b0 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f84a3fff640 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb62ca0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f848c02c3a0 tx=0x7f848c02c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c4bdc5a5fb9080 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f8499ffb640 1 -- 172.21.15.66:0/2871607577 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c048050 con 0x55cf4bb493b0 2024-04-21T09:18:30.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f8499ffb640 1 -- 172.21.15.66:0/2871607577 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c02cc90 con 0x55cf4bb493b0 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.725+0000 7f8499ffb640 1 -- 172.21.15.66:0/2871607577 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c005400 con 0x55cf4bb493b0 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 -- 172.21.15.66:0/2871607577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 msgr2=0x55cf4bb62ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb62ca0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f848c02c3a0 tx=0x7f848c02c330 comp rx=0 tx=0).stop 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 -- 172.21.15.66:0/2871607577 shutdown_connections 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb49d60 0x55cf4bb524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 --2- 172.21.15.66:0/2871607577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb62ca0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 -- 172.21.15.66:0/2871607577 >> 172.21.15.66:0/2871607577 conn(0x55cf4bb43ba0 msgr2=0x55cf4bb51da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 -- 172.21.15.66:0/2871607577 shutdown_connections 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 -- 172.21.15.66:0/2871607577 wait complete. 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 -- start start 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.726+0000 7f84ad401740 1 Processor -- start 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb5f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41ab7e0 con 0x55cf4bb493b0 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41acc50 con 0x55cf4bb52e60 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41abeb0 con 0x55cf4bb49d60 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 10 client.0 _ll_register_callbacks cb 0x7f849409be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb5f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb5f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47204/0 (socket says 172.21.15.66:47204) 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 -- 172.21.15.66:0/1987652976 learned_addr learned my addr 172.21.15.66:0/1987652976 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 -- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 msgr2=0x55cf4bb57cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a37fe640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb57cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 -- 172.21.15.66:0/1987652976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 msgr2=0x55cf4bb58230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 -- 172.21.15.66:0/1987652976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4bb58870 con 0x55cf4bb493b0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a8ccf640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84a3fff640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb5f3a0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f848c02c170 tx=0x7f848c04adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5af97795f47bd2d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84837fe640 1 -- 172.21.15.66:0/1987652976 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c02c9f0 con 0x55cf4bb493b0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84837fe640 1 -- 172.21.15.66:0/1987652976 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f848c04f040 con 0x55cf4bb493b0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.727+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f848c02cc90 con 0x55cf4bb493b0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84837fe640 1 -- 172.21.15.66:0/1987652976 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f848c005280 con 0x55cf4bb493b0 2024-04-21T09:18:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 10 client.18177 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf4bbcd2b0 con 0x55cf4bb493b0 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84837fe640 1 -- 172.21.15.66:0/1987652976 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c02cc90 con 0x55cf4bb493b0 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84837fe640 1 -- 172.21.15.66:0/1987652976 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f848c003780 con 0x55cf4bb493b0 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 10 client.18177 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 10 client.18177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 20 client.18177 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bbcd960 0x55cf4bbcfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 4 client.18177 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 --> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf4bb56ed0 con 0x55cf4bbcd960 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.728+0000 7f84a37fe640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bbcd960 0x55cf4bbcfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84a37fe640 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bbcd960 0x55cf4bbcfd80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c720f3e76c5b1fc server_cookie=1cb1e1e3d20356fe in_seq=0 out_seq=0 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84837fe640 10 client.18177 ms_handle_connect on v2:172.21.15.186:6838/599772329 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84837fe640 1 -- 172.21.15.66:0/1987652976 <== mds.1 v2:172.21.15.186:6838/599772329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf4bb56ed0 con 0x55cf4bbcd960 2024-04-21T09:18:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84837fe640 10 client.18177 handle_command_reply: tid=0 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 client.18177 shutdown 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 msgr2=0x55cf4bb5f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb5f3a0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f848c02c170 tx=0x7f848c04adb0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 shutdown_connections 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4bb52e60 0x55cf4bb58230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf4bb49d60 0x55cf4bb57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.186:6838/599772329,v1:172.21.15.186:6839/599772329] conn(0x55cf4bbcd960 0x55cf4bbcfd80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 --2- 172.21.15.66:0/1987652976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4bb493b0 0x55cf4bb5f3a0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84837fe640 0 client.18177 ms_handle_reset on v2:172.21.15.186:6838/599772329 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 >> 172.21.15.66:0/1987652976 conn(0x55cf4bb43ba0 msgr2=0x55cf4bb51080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 shutdown_connections 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 1 -- 172.21.15.66:0/1987652976 wait complete. 2024-04-21T09:18:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.729+0000 7f84ad401740 20 client.18177 trim_cache size 0 max 16384 2024-04-21T09:18:30.738 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:18:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f84900a3da0 msgr2=0x7f84900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f84900a3da0 0x7f84900a6260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f849c01f000 tx=0x7f849c018000 comp rx=0 tx=0).stop 2024-04-21T09:18:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a4107270 msgr2=0x7f84a4198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a4107270 0x7f84a4198c90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f849400d770 tx=0x7f849400dc30 comp rx=0 tx=0).stop 2024-04-21T09:18:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 shutdown_connections 2024-04-21T09:18:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f84900a3da0 0x7f84900a6260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a419d010 0x7f84a419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84a4107270 0x7f84a4198c90 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 --2- 172.21.15.66:0/3635076845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a406a840 0x7f84a4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 >> 172.21.15.66:0/3635076845 conn(0x7f84a4077eb0 msgr2=0x7f84a4108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 shutdown_connections 2024-04-21T09:18:30.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.730+0000 7f84aa732640 1 -- 172.21.15.66:0/3635076845 wait complete. 2024-04-21T09:18:30.741 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2020692654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 msgr2=0x7fbc58101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2020692654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc58101a20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009900 tx=0x7fbc4802bc80 comp rx=0 tx=0).stop 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2020692654 shutdown_connections 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2020692654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc58101f60 0x7fbc5810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2020692654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc58101a20 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2020692654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc58105e20 0x7fbc58106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2020692654 >> 172.21.15.66:0/2020692654 conn(0x7fbc58077410 msgr2=0x7fbc580fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2020692654 shutdown_connections 2024-04-21T09:18:30.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.845+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2020692654 wait complete. 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 Processor -- start 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 -- start start 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc5819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc58101f60 0x7fbc5819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc58105e20 0x7fbc58198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581122f0 con 0x7fbc581015c0 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58112170 con 0x7fbc58105e20 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc5dd2b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58112470 con 0x7fbc58101f60 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc5819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc5819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47226/0 (socket says 172.21.15.66:47226) 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc577fe640 1 -- 172.21.15.66:0/2897330964 learned_addr learned my addr 172.21.15.66:0/2897330964 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.846+0000 7fbc57fff640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc58105e20 0x7fbc58198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc58101f60 0x7fbc5819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc577fe640 1 -- 172.21.15.66:0/2897330964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc58101f60 msgr2=0x7fbc5819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc577fe640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc58101f60 0x7fbc5819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc577fe640 1 -- 172.21.15.66:0/2897330964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc58105e20 msgr2=0x7fbc58198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc577fe640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc58105e20 0x7fbc58198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc577fe640 1 -- 172.21.15.66:0/2897330964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc58198d10 con 0x7fbc581015c0 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc58101f60 0x7fbc5819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc577fe640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc5819e3e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00e9f0 tx=0x7fbc4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc54ff9640 1 -- 172.21.15.66:0/2897330964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00cd80 con 0x7fbc581015c0 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc58199000 con 0x7fbc581015c0 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc54ff9640 1 -- 172.21.15.66:0/2897330964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc4c0108e0 con 0x7fbc581015c0 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc54ff9640 1 -- 172.21.15.66:0/2897330964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c018980 con 0x7fbc581015c0 2024-04-21T09:18:30.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.847+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc581a52a0 con 0x7fbc581015c0 2024-04-21T09:18:30.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.849+0000 7fbc54ff9640 1 -- 172.21.15.66:0/2897330964 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbc4c010430 con 0x7fbc581015c0 2024-04-21T09:18:30.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 Processor -- start 2024-04-21T09:18:30.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 -- start start 2024-04-21T09:18:30.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323498d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x563234996430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234998e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5632348f21f0 con 0x56323498de80 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56323488f4b0 con 0x563234996a00 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc609f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5632348ee1b0 con 0x56323498d4d0 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc54ff9640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc400a3d20 0x7fbc400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x563234996430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc57fff640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc400a3d20 0x7fbc400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.850+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234998e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x563234996430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47236/0 (socket says 172.21.15.66:47236) 2024-04-21T09:18:30.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234998e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.66:36854/0 (socket says 172.21.15.66:36854) 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc577fe640 1 -- 172.21.15.66:0/3584614484 learned_addr learned my addr 172.21.15.66:0/3584614484 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc57fff640 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323498d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc54ff9640 1 -- 172.21.15.66:0/2897330964 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fbc4c0d0fa0 con 0x7fbc581015c0 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc56ffd640 1 -- 172.21.15.66:0/3584614484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 msgr2=0x56323498d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323498d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc56ffd640 1 -- 172.21.15.66:0/3584614484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 msgr2=0x563234996430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x563234996430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc56ffd640 1 -- 172.21.15.66:0/3584614484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632348811b0 con 0x563234996a00 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc577fe640 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x563234996430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc57fff640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc400a3d20 0x7fbc400a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbc500077f0 tx=0x7fbc50007de0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234998e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ffa10 tx=0x7fbc4802c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0fa75565274f7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc3dffb640 1 -- 172.21.15.66:0/3584614484 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4802c770 con 0x563234996a00 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc3dffb640 1 -- 172.21.15.66:0/3584614484 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc4802c910 con 0x563234996a00 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.851+0000 7fbc3dffb640 1 -- 172.21.15.66:0/3584614484 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480046c0 con 0x563234996a00 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 -- 172.21.15.66:0/3584614484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 msgr2=0x563234998e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234998e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ffa10 tx=0x7fbc4802c6d0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 -- 172.21.15.66:0/3584614484 shutdown_connections 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234998e50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x563234996430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 --2- 172.21.15.66:0/3584614484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323498d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 -- 172.21.15.66:0/3584614484 >> 172.21.15.66:0/3584614484 conn(0x563234987ba0 msgr2=0x563234987f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 -- 172.21.15.66:0/3584614484 shutdown_connections 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 -- 172.21.15.66:0/3584614484 wait complete. 2024-04-21T09:18:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.852+0000 7fbc609f8740 1 -- start start 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 1 Processor -- start 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x5632349962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498de80 0x563234993680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x563234993c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563234999910 con 0x563234996a00 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5632348ab500 con 0x56323498de80 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563234999a90 con 0x56323498d4d0 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc609f8740 10 client.0 _ll_register_callbacks cb 0x563234879b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x563234993c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x563234993c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47246/0 (socket says 172.21.15.66:47246) 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 -- 172.21.15.66:0/1251078853 learned_addr learned my addr 172.21.15.66:0/1251078853 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498de80 0x563234993680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc577fe640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x5632349962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 -- 172.21.15.66:0/1251078853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 msgr2=0x5632349962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x5632349962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 -- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498de80 msgr2=0x563234993680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498de80 0x563234993680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc57fff640 1 -- 172.21.15.66:0/1251078853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632346043f0 con 0x563234996a00 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.853+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498de80 0x563234993680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc57fff640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x563234993c00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fbc50005ca0 tx=0x7fbc50007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0a35986a61db27 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1251078853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50026070 con 0x563234996a00 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1251078853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fbc50004d20 con 0x563234996a00 2024-04-21T09:18:30.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc4802c910 con 0x563234996a00 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1251078853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc50031050 con 0x563234996a00 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 10 client.18189 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632346ce7d0 con 0x563234996a00 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1251078853 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc4802c910 con 0x563234996a00 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1251078853 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc50005090 con 0x563234996a00 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 10 client.18189 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 20 client.18189 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5632349a6b80 0x5632349a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 4 client.18189 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.854+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563234891930 con 0x5632349a6b80 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.855+0000 7fbc577fe640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5632349a6b80 0x5632349a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.855+0000 7fbc577fe640 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5632349a6b80 0x5632349a8fd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b240740a8227469 server_cookie=b02719bbed7b6639 in_seq=0 out_seq=0 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.855+0000 7fbc36ffd640 10 client.18189 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1251078853 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563234891930 con 0x5632349a6b80 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc36ffd640 10 client.18189 handle_command_reply: tid=0 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 client.18189 shutdown 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 msgr2=0x563234993c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x563234993c00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fbc50005ca0 tx=0x7fbc50007520 comp rx=0 tx=0).stop 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 shutdown_connections 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x5632349a6b80 0x5632349a8fd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x563234993c00 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498de80 0x563234993680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc36ffd640 0 client.18189 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1251078853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x5632349962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 >> 172.21.15.66:0/1251078853 conn(0x563234987ba0 msgr2=0x5632349898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 shutdown_connections 2024-04-21T09:18:30.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 1 -- 172.21.15.66:0/1251078853 wait complete. 2024-04-21T09:18:30.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.856+0000 7fbc609f8740 20 client.18189 trim_cache size 0 max 16384 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.869+0000 7fbc609f8740 1 Processor -- start 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.869+0000 7fbc609f8740 1 -- start start 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.869+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498d4d0 0x5632349a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.869+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498de80 0x563234995db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.869+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x5632349962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc609f8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581b1c50 con 0x563234996a00 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc609f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581af520 con 0x56323498d4d0 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc609f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581ad390 con 0x56323498de80 2024-04-21T09:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x5632349962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x5632349962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47260/0 (socket says 172.21.15.66:47260) 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 -- 172.21.15.66:0/2173274310 learned_addr learned my addr 172.21.15.66:0/2173274310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc577fe640 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498d4d0 0x5632349a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498de80 0x563234995db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 -- 172.21.15.66:0/2173274310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498de80 msgr2=0x563234995db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498de80 0x563234995db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 -- 172.21.15.66:0/2173274310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498d4d0 msgr2=0x5632349a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498d4d0 0x5632349a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 -- 172.21.15.66:0/2173274310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323498f270 con 0x563234996a00 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc577fe640 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498d4d0 0x5632349a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc57fff640 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x5632349962f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fbc500077c0 tx=0x7fbc500058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd293fe809a3a468 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc3d7fa640 1 -- 172.21.15.66:0/2173274310 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5003a070 con 0x563234996a00 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc3d7fa640 1 -- 172.21.15.66:0/2173274310 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc50005a20 con 0x563234996a00 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.870+0000 7fbc3d7fa640 1 -- 172.21.15.66:0/2173274310 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50026050 con 0x563234996a00 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 -- 172.21.15.66:0/2173274310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 msgr2=0x5632349962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x5632349962f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fbc500077c0 tx=0x7fbc500058b0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 -- 172.21.15.66:0/2173274310 shutdown_connections 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563234996a00 0x5632349962f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498de80 0x563234995db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 --2- 172.21.15.66:0/2173274310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56323498d4d0 0x5632349a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 -- 172.21.15.66:0/2173274310 >> 172.21.15.66:0/2173274310 conn(0x563234987ba0 msgr2=0x563234993640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 -- 172.21.15.66:0/2173274310 shutdown_connections 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 -- 172.21.15.66:0/2173274310 wait complete. 2024-04-21T09:18:30.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 -- start start 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.871+0000 7fbc609f8740 1 Processor -- start 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323499eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x56323498fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234990430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581ad9e0 con 0x56323498de80 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581aee50 con 0x563234996a00 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581ae0b0 con 0x56323498d4d0 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 10 client.0 _ll_register_callbacks cb 0x7fbc50011e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x56323498fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x56323498fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47274/0 (socket says 172.21.15.66:47274) 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 -- 172.21.15.66:0/1534436936 learned_addr learned my addr 172.21.15.66:0/1534436936 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:30.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc577fe640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323499eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc57fff640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234990430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 -- 172.21.15.66:0/1534436936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 msgr2=0x56323499eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323499eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 -- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 msgr2=0x563234990430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234990430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 -- 172.21.15.66:0/1534436936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632349909a0 con 0x56323498de80 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc57fff640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234990430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc56ffd640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x56323498fef0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ffa10 tx=0x7fbc480056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1eb65c7635e410 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1534436936 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48004230 con 0x56323498de80 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc50005a20 con 0x56323498de80 2024-04-21T09:18:30.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.872+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1534436936 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fbc480043d0 con 0x56323498de80 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1534436936 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc4805b050 con 0x56323498de80 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 10 client.18195 fetch_fsmap learned FSMap version 25 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563234a0cf30 con 0x56323498de80 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1534436936 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc50005a20 con 0x56323498de80 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1534436936 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc48052630 con 0x56323498de80 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 10 client.18195 fetch_fsmap finished waiting for FSMap version 25 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 20 client.18195 populate_metadata read hostname 'smithi066' 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563234a0d290 0x563234a0f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 4 client.18195 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 --> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632348ffc10 con 0x563234a0d290 2024-04-21T09:18:30.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.873+0000 7fbc577fe640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563234a0d290 0x563234a0f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc36ffd640 10 client.18195 ms_handle_connect on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc577fe640 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563234a0d290 0x563234a0f680 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a877e19a0e745 server_cookie=43cbecd473247f11 in_seq=0 out_seq=0 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc36ffd640 1 -- 172.21.15.66:0/1534436936 <== mds.0 v2:172.21.15.83:6838/4140026829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632348ffc10 con 0x563234a0d290 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc36ffd640 10 client.18195 handle_command_reply: tid=0 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 client.18195 shutdown 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 msgr2=0x56323498fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x56323498fef0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ffa10 tx=0x7fbc480056d0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 shutdown_connections 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:6838/4140026829,v1:172.21.15.83:6839/4140026829] conn(0x563234a0d290 0x563234a0f680 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563234996a00 0x563234990430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56323498de80 0x56323498fef0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc36ffd640 0 client.18195 ms_handle_reset on v2:172.21.15.83:6838/4140026829 2024-04-21T09:18:30.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 --2- 172.21.15.66:0/1534436936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56323498d4d0 0x56323499eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 >> 172.21.15.66:0/1534436936 conn(0x563234987ba0 msgr2=0x563234986370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 shutdown_connections 2024-04-21T09:18:30.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 1 -- 172.21.15.66:0/1534436936 wait complete. 2024-04-21T09:18:30.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.874+0000 7fbc609f8740 20 client.18195 trim_cache size 0 max 16384 2024-04-21T09:18:30.886 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-04-21T09:18:30.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc400a3d20 msgr2=0x7fbc400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc400a3d20 0x7fbc400a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbc500077f0 tx=0x7fbc50007de0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 msgr2=0x7fbc5819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc5819e3e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00e9f0 tx=0x7fbc4c00eeb0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 shutdown_connections 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbc400a3d20 0x7fbc400a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc58105e20 0x7fbc58198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc58101f60 0x7fbc5819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 --2- 172.21.15.66:0/2897330964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc581015c0 0x7fbc5819e3e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 >> 172.21.15.66:0/2897330964 conn(0x7fbc58077410 msgr2=0x7fbc58103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 shutdown_connections 2024-04-21T09:18:30.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:30.875+0000 7fbc5dd2b640 1 -- 172.21.15.66:0/2897330964 wait complete. 2024-04-21T09:18:30.889 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/thrasher.py", line 43, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-04-21T09:18:30.890 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: 2024-04-21T09:18:30.890 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-21T09:18:30.903 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:18:31.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.004+0000 7f629bed2640 1 -- 172.21.15.66:0/3553479634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 msgr2=0x7f6294101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.004+0000 7f629bed2640 1 --2- 172.21.15.66:0/3553479634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f6294101490 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6290009900 tx=0x7f629002bc30 comp rx=0 tx=0).stop 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.004+0000 7f629bed2640 1 -- 172.21.15.66:0/3553479634 shutdown_connections 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.004+0000 7f629bed2640 1 --2- 172.21.15.66:0/3553479634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6294106180 0x7f629410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.004+0000 7f629bed2640 1 --2- 172.21.15.66:0/3553479634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6294101a60 0x7f6294105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.004+0000 7f629bed2640 1 --2- 172.21.15.66:0/3553479634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f6294101490 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.004+0000 7f629bed2640 1 -- 172.21.15.66:0/3553479634 >> 172.21.15.66:0/3553479634 conn(0x7f62940771d0 msgr2=0x7f62940775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 -- 172.21.15.66:0/3553479634 shutdown_connections 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 -- 172.21.15.66:0/3553479634 wait complete. 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 Processor -- start 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 -- start start 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f62941986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6294106180 0x7f6294198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f629419cfb0 0x7f629419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f62941110c0 con 0x7f62941010b0 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6294110f40 con 0x7f6294106180 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.005+0000 7f629bed2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6294111240 con 0x7f629419cfb0 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f62941986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f62941986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47296/0 (socket says 172.21.15.66:47296) 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 -- 172.21.15.66:0/1450662239 learned_addr learned my addr 172.21.15.66:0/1450662239 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f629a46f640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f629419cfb0 0x7f629419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f629946d640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6294106180 0x7f6294198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 -- 172.21.15.66:0/1450662239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f629419cfb0 msgr2=0x7f629419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f629419cfb0 0x7f629419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 -- 172.21.15.66:0/1450662239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6294106180 msgr2=0x7f6294198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6294106180 0x7f6294198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 -- 172.21.15.66:0/1450662239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629419daa0 con 0x7f62941010b0 2024-04-21T09:18:31.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f629a46f640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f629419cfb0 0x7f629419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f629946d640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6294106180 0x7f6294198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f6299c6e640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f62941986f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f6290005940 tx=0x7f62900045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f628affd640 1 -- 172.21.15.66:0/1450662239 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290043020 con 0x7f62941010b0 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f628affd640 1 -- 172.21.15.66:0/1450662239 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629002caf0 con 0x7f62941010b0 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629419dd30 con 0x7f62941010b0 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f628affd640 1 -- 172.21.15.66:0/1450662239 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290003af0 con 0x7f62941010b0 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.006+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62941a5550 con 0x7f62941010b0 2024-04-21T09:18:31.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.007+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6294069230 con 0x7f62941010b0 2024-04-21T09:18:31.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.009+0000 7f628affd640 1 -- 172.21.15.66:0/1450662239 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6290046020 con 0x7f62941010b0 2024-04-21T09:18:31.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.010+0000 7f628affd640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f62800a3d20 0x7f62800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.010+0000 7f628affd640 1 -- 172.21.15.66:0/1450662239 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f62900f3400 con 0x7f62941010b0 2024-04-21T09:18:31.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.010+0000 7f629946d640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f62800a3d20 0x7f62800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.010+0000 7f629946d640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f62800a3d20 0x7f62800a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62940fe5b0 tx=0x7f6284005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.014+0000 7f628affd640 1 -- 172.21.15.66:0/1450662239 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f629004d020 con 0x7f62941010b0 2024-04-21T09:18:31.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:30 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2946146916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:18:31.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:30 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2946146916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:18:31.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:30 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2946146916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:18:31.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.203+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6294199a40 con 0x7f62941010b0 2024-04-21T09:18:31.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.204+0000 7f628affd640 1 -- 172.21.15.66:0/1450662239 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+25272 (secure 0 0 0) 0x7f62900bf7e0 con 0x7f62941010b0 2024-04-21T09:18:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:18:31.205 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":104,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:18:09.796444+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:18:31.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f62800a3d20 msgr2=0x7f62800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f62800a3d20 0x7f62800a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62940fe5b0 tx=0x7f6284005f90 comp rx=0 tx=0).stop 2024-04-21T09:18:31.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 msgr2=0x7f62941986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f62941986f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f6290005940 tx=0x7f62900045e0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 shutdown_connections 2024-04-21T09:18:31.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f62800a3d20 0x7f62800a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f629419cfb0 0x7f629419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6294106180 0x7f6294198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 --2- 172.21.15.66:0/1450662239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62941010b0 0x7f62941986f0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 >> 172.21.15.66:0/1450662239 conn(0x7f62940771d0 msgr2=0x7f629406aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:31.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 shutdown_connections 2024-04-21T09:18:31.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.207+0000 7f629bed2640 1 -- 172.21.15.66:0/1450662239 wait complete. 2024-04-21T09:18:31.218 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:18:31.219 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:18:31.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.323+0000 7f57be0e4640 1 -- 172.21.15.66:0/3058706792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8107260 msgr2=0x7f57b810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.323+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3058706792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8107260 0x7f57b810ee30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ca0 tx=0x7f57b002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.323+0000 7f57be0e4640 1 -- 172.21.15.66:0/3058706792 shutdown_connections 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.323+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3058706792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8107260 0x7f57b810ee30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.323+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3058706792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b8102b40 0x7f57b8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.323+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3058706792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57b8102190 0x7f57b8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.323+0000 7f57be0e4640 1 -- 172.21.15.66:0/3058706792 >> 172.21.15.66:0/3058706792 conn(0x7f57b80fd990 msgr2=0x7f57b80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.324+0000 7f57be0e4640 1 -- 172.21.15.66:0/3058706792 shutdown_connections 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.324+0000 7f57be0e4640 1 -- 172.21.15.66:0/3058706792 wait complete. 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.324+0000 7f57be0e4640 1 Processor -- start 2024-04-21T09:18:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.324+0000 7f57be0e4640 1 -- start start 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.324+0000 7f57be0e4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8102190 0x7f57b8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57be0e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57b8107260 0x7f57b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57be0e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b8074cd0 0x7f57b81a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57be0e4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8112220 con 0x7f57b8102190 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57be0e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81120a0 con 0x7f57b8107260 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57be0e4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81123a0 con 0x7f57b8074cd0 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8102190 0x7f57b8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8102190 0x7f57b8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47310/0 (socket says 172.21.15.66:47310) 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 -- 172.21.15.66:0/3354678752 learned_addr learned my addr 172.21.15.66:0/3354678752 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b6ffd640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57b8107260 0x7f57b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b7fff640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b8074cd0 0x7f57b81a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 -- 172.21.15.66:0/3354678752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b8074cd0 msgr2=0x7f57b81a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b8074cd0 0x7f57b81a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 -- 172.21.15.66:0/3354678752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57b8107260 msgr2=0x7f57b8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57b8107260 0x7f57b8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 -- 172.21.15.66:0/3354678752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b81a7050 con 0x7f57b8102190 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b7fff640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b8074cd0 0x7f57b81a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b77fe640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8102190 0x7f57b8074250 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d950 tx=0x7f57a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b4ff9640 1 -- 172.21.15.66:0/3354678752 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8017070 con 0x7f57b8102190 2024-04-21T09:18:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b4ff9640 1 -- 172.21.15.66:0/3354678752 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a8009c30 con 0x7f57b8102190 2024-04-21T09:18:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b81a7340 con 0x7f57b8102190 2024-04-21T09:18:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57b4ff9640 1 -- 172.21.15.66:0/3354678752 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a800f620 con 0x7f57b8102190 2024-04-21T09:18:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.325+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81a7890 con 0x7f57b8102190 2024-04-21T09:18:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.326+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b80691c0 con 0x7f57b8102190 2024-04-21T09:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.327+0000 7f57b4ff9640 1 -- 172.21.15.66:0/3354678752 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f57a8010460 con 0x7f57b8102190 2024-04-21T09:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.328+0000 7f57b4ff9640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f57a00a3d50 0x7f57a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.328+0000 7f57b6ffd640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f57a00a3d50 0x7f57a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.328+0000 7f57b4ff9640 1 -- 172.21.15.66:0/3354678752 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f57a80d0890 con 0x7f57b8102190 2024-04-21T09:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.328+0000 7f57b6ffd640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f57a00a3d50 0x7f57a00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57b8075ce0 tx=0x7f57ac008040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.333+0000 7f57b4ff9640 1 -- 172.21.15.66:0/3354678752 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57a809cb60 con 0x7f57b8102190 2024-04-21T09:18:31.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.518+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57b810a490 con 0x7f57b8102190 2024-04-21T09:18:31.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.518+0000 7f57b4ff9640 1 -- 172.21.15.66:0/3354678752 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f57a809c460 con 0x7f57b8102190 2024-04-21T09:18:31.519 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f57a00a3d50 msgr2=0x7f57a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f57a00a3d50 0x7f57a00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57b8075ce0 tx=0x7f57ac008040 comp rx=0 tx=0).stop 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8102190 msgr2=0x7f57b8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8102190 0x7f57b8074250 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d950 tx=0x7f57a800de10 comp rx=0 tx=0).stop 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 shutdown_connections 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f57a00a3d50 0x7f57a00a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b8074cd0 0x7f57b81a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57b8107260 0x7f57b8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 --2- 172.21.15.66:0/3354678752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57b8102190 0x7f57b8074250 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 >> 172.21.15.66:0/3354678752 conn(0x7f57b80fd990 msgr2=0x7f57b80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 shutdown_connections 2024-04-21T09:18:31.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.520+0000 7f57be0e4640 1 -- 172.21.15.66:0/3354678752 wait complete. 2024-04-21T09:18:31.531 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:18:31.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 -- 172.21.15.66:0/2779116601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f196806b1f0 msgr2=0x7f1968106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 --2- 172.21.15.66:0/2779116601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f196806b1f0 0x7f1968106b90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1964009900 tx=0x7f196402bc30 comp rx=0 tx=0).stop 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 -- 172.21.15.66:0/2779116601 shutdown_connections 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 --2- 172.21.15.66:0/2779116601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968107270 0x7f196810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 --2- 172.21.15.66:0/2779116601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f196806b1f0 0x7f1968106b90 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 --2- 172.21.15.66:0/2779116601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196806a840 0x7f196806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 -- 172.21.15.66:0/2779116601 >> 172.21.15.66:0/2779116601 conn(0x7f1968077eb0 msgr2=0x7f19680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 -- 172.21.15.66:0/2779116601 shutdown_connections 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.636+0000 7f196fcba640 1 -- 172.21.15.66:0/2779116601 wait complete. 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 Processor -- start 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 -- start start 2024-04-21T09:18:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196806a840 0x7f1968100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968107270 0x7f19681013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1968101900 0x7f19681a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112210 con 0x7f1968101900 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112090 con 0x7f1968107270 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196fcba640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112390 con 0x7f196806a840 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1968101900 0x7f19681a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1968101900 0x7f19681a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47324/0 (socket says 172.21.15.66:47324) 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 -- 172.21.15.66:0/2645049388 learned_addr learned my addr 172.21.15.66:0/2645049388 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196da56640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196806a840 0x7f1968100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196d255640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968107270 0x7f19681013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 -- 172.21.15.66:0/2645049388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196806a840 msgr2=0x7f1968100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196806a840 0x7f1968100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 -- 172.21.15.66:0/2645049388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968107270 msgr2=0x7f19681013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968107270 0x7f19681013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.637+0000 7f196e257640 1 -- 172.21.15.66:0/2645049388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19681a70a0 con 0x7f1968101900 2024-04-21T09:18:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.638+0000 7f196e257640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1968101900 0x7f19681a69a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1960009fd0 tx=0x7f196000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.638+0000 7f195affd640 1 -- 172.21.15.66:0/2645049388 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960017070 con 0x7f1968101900 2024-04-21T09:18:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.638+0000 7f195affd640 1 -- 172.21.15.66:0/2645049388 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1960003e20 con 0x7f1968101900 2024-04-21T09:18:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.638+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19681a7390 con 0x7f1968101900 2024-04-21T09:18:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.638+0000 7f195affd640 1 -- 172.21.15.66:0/2645049388 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960010460 con 0x7f1968101900 2024-04-21T09:18:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.638+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19681a7860 con 0x7f1968101900 2024-04-21T09:18:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.639+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19680691c0 con 0x7f1968101900 2024-04-21T09:18:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.640+0000 7f195affd640 1 -- 172.21.15.66:0/2645049388 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1960003970 con 0x7f1968101900 2024-04-21T09:18:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.641+0000 7f195affd640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19540a3d50 0x7f19540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.641+0000 7f196da56640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19540a3d50 0x7f19540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.641+0000 7f195affd640 1 -- 172.21.15.66:0/2645049388 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f19600d09d0 con 0x7f1968101900 2024-04-21T09:18:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.641+0000 7f196da56640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19540a3d50 0x7f19540a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f195c009890 tx=0x7f195c008040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.644+0000 7f195affd640 1 -- 172.21.15.66:0/2645049388 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f196009cca0 con 0x7f1968101900 2024-04-21T09:18:31.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.831+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f196810a4a0 con 0x7f1968101900 2024-04-21T09:18:31.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.832+0000 7f195affd640 1 -- 172.21.15.66:0/2645049388 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7f196009c5a0 con 0x7f1968101900 2024-04-21T09:18:31.832 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:18:31.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19540a3d50 msgr2=0x7f19540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19540a3d50 0x7f19540a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f195c009890 tx=0x7f195c008040 comp rx=0 tx=0).stop 2024-04-21T09:18:31.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1968101900 msgr2=0x7f19681a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1968101900 0x7f19681a69a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1960009fd0 tx=0x7f196000b260 comp rx=0 tx=0).stop 2024-04-21T09:18:31.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 shutdown_connections 2024-04-21T09:18:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f19540a3d50 0x7f19540a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1968101900 0x7f19681a69a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968107270 0x7f19681013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 --2- 172.21.15.66:0/2645049388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196806a840 0x7f1968100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 >> 172.21.15.66:0/2645049388 conn(0x7f1968077eb0 msgr2=0x7f1968108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 shutdown_connections 2024-04-21T09:18:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.834+0000 7f196fcba640 1 -- 172.21.15.66:0/2645049388 wait complete. 2024-04-21T09:18:31.844 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:31 smithi066 ceph-mon[27953]: pgmap v320: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:31.844 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:31 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1450662239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:18:31.844 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:31 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3354678752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:18:31.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:18:31.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 -- 172.21.15.66:0/538542193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 msgr2=0x7f4038103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 --2- 172.21.15.66:0/538542193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f4038103590 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ca0 tx=0x7f402802c180 comp rx=0 tx=0).stop 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 -- 172.21.15.66:0/538542193 shutdown_connections 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 --2- 172.21.15.66:0/538542193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403810c580 0x7f403810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 --2- 172.21.15.66:0/538542193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4038103ad0 0x7f403810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 --2- 172.21.15.66:0/538542193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f4038103590 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 -- 172.21.15.66:0/538542193 >> 172.21.15.66:0/538542193 conn(0x7f40380771d0 msgr2=0x7f40380775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 -- 172.21.15.66:0/538542193 shutdown_connections 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.949+0000 7f403fdf4640 1 -- 172.21.15.66:0/538542193 wait complete. 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 Processor -- start 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 -- start start 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f403819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4038103ad0 0x7f403819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403810c580 0x7f40381985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40381121d0 con 0x7f40381031b0 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4038112050 con 0x7f4038103ad0 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403fdf4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4038112350 con 0x7f403810c580 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f403819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f403819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47338/0 (socket says 172.21.15.66:47338) 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 -- 172.21.15.66:0/2030931603 learned_addr learned my addr 172.21.15.66:0/2030931603 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403d38f640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4038103ad0 0x7f403819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 -- 172.21.15.66:0/2030931603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403810c580 msgr2=0x7f40381985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403810c580 0x7f40381985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 -- 172.21.15.66:0/2030931603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4038103ad0 msgr2=0x7f403819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4038103ad0 0x7f403819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403db90640 1 -- 172.21.15.66:0/2030931603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4038198d50 con 0x7f40381031b0 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.950+0000 7f403d38f640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4038103ad0 0x7f403819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:31.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.951+0000 7f403db90640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f403819e420 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4028005e30 tx=0x7f40280039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.951+0000 7f4036ffd640 1 -- 172.21.15.66:0/2030931603 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028039040 con 0x7f40381031b0 2024-04-21T09:18:31.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.951+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4038198fe0 con 0x7f40381031b0 2024-04-21T09:18:31.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.951+0000 7f4036ffd640 1 -- 172.21.15.66:0/2030931603 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4028044070 con 0x7f40381031b0 2024-04-21T09:18:31.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.951+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40381a52e0 con 0x7f40381031b0 2024-04-21T09:18:31.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.951+0000 7f4036ffd640 1 -- 172.21.15.66:0/2030931603 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028032900 con 0x7f40381031b0 2024-04-21T09:18:31.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.952+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403806ad70 con 0x7f40381031b0 2024-04-21T09:18:31.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.953+0000 7f4036ffd640 1 -- 172.21.15.66:0/2030931603 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4028004580 con 0x7f40381031b0 2024-04-21T09:18:31.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.954+0000 7f4036ffd640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f40200a3d20 0x7f40200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:31.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.954+0000 7f403d38f640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f40200a3d20 0x7f40200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:31.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.954+0000 7f4036ffd640 1 -- 172.21.15.66:0/2030931603 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f40280f3760 con 0x7f40381031b0 2024-04-21T09:18:31.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.954+0000 7f403d38f640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f40200a3d20 0x7f40200a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f402c0058c0 tx=0x7f402c005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:31.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:31.958+0000 7f4036ffd640 1 -- 172.21.15.66:0/2030931603 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40280bfb40 con 0x7f40381031b0 2024-04-21T09:18:32.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:31 smithi186 ceph-mon[31505]: pgmap v320: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:32.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:31 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1450662239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:18:32.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:31 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3354678752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:18:32.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:31 smithi083 ceph-mon[31413]: pgmap v320: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:32.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:31 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1450662239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:18:32.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:31 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3354678752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:18:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.142+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f403802d5c0 con 0x7f40381031b0 2024-04-21T09:18:32.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.143+0000 7f4036ffd640 1 -- 172.21.15.66:0/2030931603 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7f40280bf440 con 0x7f40381031b0 2024-04-21T09:18:32.143 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:18:32.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f40200a3d20 msgr2=0x7f40200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f40200a3d20 0x7f40200a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f402c0058c0 tx=0x7f402c005850 comp rx=0 tx=0).stop 2024-04-21T09:18:32.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 msgr2=0x7f403819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f403819e420 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4028005e30 tx=0x7f40280039f0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 shutdown_connections 2024-04-21T09:18:32.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f40200a3d20 0x7f40200a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403810c580 0x7f40381985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4038103ad0 0x7f403819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 --2- 172.21.15.66:0/2030931603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40381031b0 0x7f403819e420 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 >> 172.21.15.66:0/2030931603 conn(0x7f40380771d0 msgr2=0x7f40380fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:32.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 shutdown_connections 2024-04-21T09:18:32.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.145+0000 7f403fdf4640 1 -- 172.21.15.66:0/2030931603 wait complete. 2024-04-21T09:18:32.156 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:18:32.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 -- 172.21.15.66:0/3139419968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4101a50 msgr2=0x7fc7e4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 --2- 172.21.15.66:0/3139419968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4101a50 0x7fc7e4105a90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009900 tx=0x7fc7d802bc30 comp rx=0 tx=0).stop 2024-04-21T09:18:32.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 -- 172.21.15.66:0/3139419968 shutdown_connections 2024-04-21T09:18:32.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 --2- 172.21.15.66:0/3139419968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e4106170 0x7fc7e410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 --2- 172.21.15.66:0/3139419968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4101a50 0x7fc7e4105a90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 --2- 172.21.15.66:0/3139419968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e41010a0 0x7fc7e4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 -- 172.21.15.66:0/3139419968 >> 172.21.15.66:0/3139419968 conn(0x7fc7e4077eb0 msgr2=0x7fc7e40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 -- 172.21.15.66:0/3139419968 shutdown_connections 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.260+0000 7fc7ec152640 1 -- 172.21.15.66:0/3139419968 wait complete. 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 Processor -- start 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 -- start start 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e41010a0 0x7fc7e419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4106170 0x7fc7e419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e41985e0 0x7fc7e4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4111110 con 0x7fc7e4106170 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4110f90 con 0x7fc7e41985e0 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7ec152640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4111290 con 0x7fc7e41010a0 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7e96ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4106170 0x7fc7e419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7e96ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4106170 0x7fc7e419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47370/0 (socket says 172.21.15.66:47370) 2024-04-21T09:18:32.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.261+0000 7fc7e96ed640 1 -- 172.21.15.66:0/2489674301 learned_addr learned my addr 172.21.15.66:0/2489674301 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e9eee640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e41010a0 0x7fc7e419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7ea6ef640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e41985e0 0x7fc7e4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e96ed640 1 -- 172.21.15.66:0/2489674301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e41010a0 msgr2=0x7fc7e419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e96ed640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e41010a0 0x7fc7e419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e96ed640 1 -- 172.21.15.66:0/2489674301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e41985e0 msgr2=0x7fc7e4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e96ed640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e41985e0 0x7fc7e4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e96ed640 1 -- 172.21.15.66:0/2489674301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e4199100 con 0x7fc7e4106170 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e9eee640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e41010a0 0x7fc7e419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7ea6ef640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e41985e0 0x7fc7e4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e96ed640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4106170 0x7fc7e419e970 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8036040 tx=0x7fc7d802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e2ffd640 1 -- 172.21.15.66:0/2489674301 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8043020 con 0x7fc7e4106170 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e2ffd640 1 -- 172.21.15.66:0/2489674301 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d802cbe0 con 0x7fc7e4106170 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7e2ffd640 1 -- 172.21.15.66:0/2489674301 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8003bf0 con 0x7fc7e4106170 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e4199390 con 0x7fc7e4106170 2024-04-21T09:18:32.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.262+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e41a55e0 con 0x7fc7e4106170 2024-04-21T09:18:32.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.263+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e40691c0 con 0x7fc7e4106170 2024-04-21T09:18:32.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.264+0000 7fc7e2ffd640 1 -- 172.21.15.66:0/2489674301 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc7d8046020 con 0x7fc7e4106170 2024-04-21T09:18:32.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.265+0000 7fc7e2ffd640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7cc0a3d20 0x7fc7cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.265+0000 7fc7e9eee640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7cc0a3d20 0x7fc7cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:32.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.265+0000 7fc7e2ffd640 1 -- 172.21.15.66:0/2489674301 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fc7d80f2dc0 con 0x7fc7e4106170 2024-04-21T09:18:32.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.265+0000 7fc7e9eee640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7cc0a3d20 0x7fc7cc0a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc005fd0 tx=0x7fc7dc00b5f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:32.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.270+0000 7fc7e2ffd640 1 -- 172.21.15.66:0/2489674301 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7d804d020 con 0x7fc7e4106170 2024-04-21T09:18:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.456+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc7e4101a50 con 0x7fc7e4106170 2024-04-21T09:18:32.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.456+0000 7fc7e2ffd640 1 -- 172.21.15.66:0/2489674301 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v104) ==== 103+0+11 (secure 0 0 0) 0x7fc7d80bf1a0 con 0x7fc7e4106170 2024-04-21T09:18:32.457 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:18:32.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7cc0a3d20 msgr2=0x7fc7cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7cc0a3d20 0x7fc7cc0a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc005fd0 tx=0x7fc7dc00b5f0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4106170 msgr2=0x7fc7e419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4106170 0x7fc7e419e970 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8036040 tx=0x7fc7d802c7e0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 shutdown_connections 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fc7cc0a3d20 0x7fc7cc0a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e41985e0 0x7fc7e4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7e4106170 0x7fc7e419e970 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 --2- 172.21.15.66:0/2489674301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e41010a0 0x7fc7e419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 >> 172.21.15.66:0/2489674301 conn(0x7fc7e4077eb0 msgr2=0x7fc7e406a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 shutdown_connections 2024-04-21T09:18:32.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.458+0000 7fc7ec152640 1 -- 172.21.15.66:0/2489674301 wait complete. 2024-04-21T09:18:32.469 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-21T09:18:32.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 -- 172.21.15.66:0/2018504193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 msgr2=0x7fbf88107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 --2- 172.21.15.66:0/2018504193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf88107360 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009900 tx=0x7fbf7802bc30 comp rx=0 tx=0).stop 2024-04-21T09:18:32.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 -- 172.21.15.66:0/2018504193 shutdown_connections 2024-04-21T09:18:32.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 --2- 172.21.15.66:0/2018504193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf88101240 0x7fbf8810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 --2- 172.21.15.66:0/2018504193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88100880 0x7fbf88100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 --2- 172.21.15.66:0/2018504193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf88107360 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 -- 172.21.15.66:0/2018504193 >> 172.21.15.66:0/2018504193 conn(0x7fbf880761d0 msgr2=0x7fbf880765e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 -- 172.21.15.66:0/2018504193 shutdown_connections 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.571+0000 7fbf8d967640 1 -- 172.21.15.66:0/2018504193 wait complete. 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 Processor -- start 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 -- start start 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf88100880 0x7fbf88198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88101240 0x7fbf88198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf8819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88112190 con 0x7fbf88106f80 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88112010 con 0x7fbf88100880 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf8d967640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88112310 con 0x7fbf88101240 2024-04-21T09:18:32.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf877fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf8819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf877fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf8819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47398/0 (socket says 172.21.15.66:47398) 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf877fe640 1 -- 172.21.15.66:0/3848574155 learned_addr learned my addr 172.21.15.66:0/3848574155 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.572+0000 7fbf86ffd640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf88100880 0x7fbf88198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf877fe640 1 -- 172.21.15.66:0/3848574155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88101240 msgr2=0x7fbf88198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf877fe640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88101240 0x7fbf88198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf877fe640 1 -- 172.21.15.66:0/3848574155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf88100880 msgr2=0x7fbf88198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf877fe640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf88100880 0x7fbf88198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf877fe640 1 -- 172.21.15.66:0/3848574155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8819d700 con 0x7fbf88106f80 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf86ffd640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf88100880 0x7fbf88198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf877fe640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf8819cfd0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009fd0 tx=0x7fbf8000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf8c965640 1 -- 172.21.15.66:0/3848574155 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80017070 con 0x7fbf88106f80 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8819d9f0 con 0x7fbf88106f80 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf8c965640 1 -- 172.21.15.66:0/3848574155 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf80003e20 con 0x7fbf88106f80 2024-04-21T09:18:32.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf8c965640 1 -- 172.21.15.66:0/3848574155 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80010460 con 0x7fbf88106f80 2024-04-21T09:18:32.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.573+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881a5280 con 0x7fbf88106f80 2024-04-21T09:18:32.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.574+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf880691c0 con 0x7fbf88106f80 2024-04-21T09:18:32.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.575+0000 7fbf8c965640 1 -- 172.21.15.66:0/3848574155 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbf80003970 con 0x7fbf88106f80 2024-04-21T09:18:32.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.576+0000 7fbf8c965640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbf740a3d20 0x7fbf740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:18:32.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.576+0000 7fbf86ffd640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbf740a3d20 0x7fbf740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:18:32.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.576+0000 7fbf8c965640 1 -- 172.21.15.66:0/3848574155 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fbf800d0a50 con 0x7fbf88106f80 2024-04-21T09:18:32.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.577+0000 7fbf86ffd640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbf740a3d20 0x7fbf740a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000c00 tx=0x7fbf78005dd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:18:32.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.580+0000 7fbf8c965640 1 -- 172.21.15.66:0/3848574155 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf8009cd00 con 0x7fbf88106f80 2024-04-21T09:18:32.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.823+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fbf8802d5c0 con 0x7fbf88106f80 2024-04-21T09:18:32.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.831+0000 7fbf8c965640 1 -- 172.21.15.66:0/3848574155 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746062 (secure 0 0 0) 0x7fbf8009c600 con 0x7fbf88106f80 2024-04-21T09:18:32.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:18:32.842 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 25, 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "active_gid": 15456, 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "active_name": "x", 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "active_addrs": { 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:6800", 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3952680321 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.843 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:6801", 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3952680321 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "active_addr": "172.21.15.66:6801/3952680321", 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "active_change": "2024-04-21T09:08:12.028698+0000", 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "active_mgr_features": 4540701547738038271, 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "standbys": [ 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 25039, 2024-04-21T09:18:32.844 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "z", 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr_features": 4540701547738038271, 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:32.845 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.846 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.847 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.848 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.849 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.850 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.851 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2024-04-21T09:18:32.852 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.854 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.855 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.856 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.857 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T09:18:32.858 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.860 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.861 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2024-04-21T09:18:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2024-04-21T09:18:32.863 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2024-04-21T09:18:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:32.865 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2024-04-21T09:18:32.866 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2024-04-21T09:18:32.867 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:32.868 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.869 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.870 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.871 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.872 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.873 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.874 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2024-04-21T09:18:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "read", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap-read" 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.876 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.877 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.880 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.881 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.882 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2024-04-21T09:18:32.883 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.884 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T09:18:32.885 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.887 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm_log_destination": { 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm_log_destination", 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.888 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "file", 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "file,syslog", 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "syslog" 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2024-04-21T09:18:32.889 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2024-04-21T09:18:32.890 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2024-04-21T09:18:32.891 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2024-04-21T09:18:32.892 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2024-04-21T09:18:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.896 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.897 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.898 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.899 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.900 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.901 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.902 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.903 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_nvmeof": { 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_nvmeof", 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.904 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Nvme-of container image", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:32.905 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.906 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_samba": { 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_samba", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.907 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Samba/SMB container image", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2024-04-21T09:18:32.908 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.909 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.910 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.911 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.912 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.913 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.914 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.915 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "grafana_dashboards_path": { 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "grafana_dashboards_path", 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T09:18:32.916 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "hw_monitoring": { 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hw_monitoring", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.918 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "inventory_list_all": { 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "inventory_list_all", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.919 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.920 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.921 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.922 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.923 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:32.924 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.925 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.926 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.927 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.928 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.929 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.930 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2024-04-21T09:18:32.931 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.934 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.934 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.934 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.934 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.934 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.934 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.934 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "oob_default_addr": { 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "oob_default_addr", 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "169.254.1.1", 2024-04-21T09:18:32.936 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.937 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.938 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.939 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.940 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.942 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.943 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2024-04-21T09:18:32.944 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.945 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.946 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2024-04-21T09:18:32.946 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2024-04-21T09:18:32.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.946 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.946 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.948 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.949 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2024-04-21T09:18:32.951 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.952 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.953 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.954 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.955 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:32.956 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2024-04-21T09:18:32.957 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.958 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.959 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:32.960 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.961 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.962 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.964 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.965 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T09:18:32.966 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.967 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.969 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.970 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.971 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.972 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.973 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.974 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.976 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.977 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.978 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.979 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.980 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.981 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "[]", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.983 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "{}", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.984 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.985 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.986 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.988 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.989 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.990 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.991 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.992 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:32.993 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.994 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.996 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:32.998 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:32.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.000 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.002 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T09:18:33.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T09:18:33.004 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T09:18:33.005 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.006 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.008 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.009 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2024-04-21T09:18:33.010 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.011 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.012 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.013 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.014 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.016 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.017 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.018 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.019 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.020 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.021 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.022 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.023 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.024 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.026 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.027 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.028 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.029 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.030 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.031 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.032 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.035 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.036 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.037 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.038 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.039 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.041 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.042 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.043 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.044 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.045 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.046 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.048 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.049 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.050 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.052 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.054 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.055 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.056 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.057 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.058 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.059 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.060 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.061 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.062 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.063 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.064 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.066 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.067 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.068 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.070 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.071 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.072 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.073 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.074 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.075 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.076 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.077 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.078 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.079 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.080 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.081 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.082 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.083 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.084 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.086 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.087 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.088 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.089 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.090 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.094 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.096 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.097 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.098 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.099 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T09:18:33.100 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.101 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.102 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.103 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.104 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.105 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.106 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.107 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.108 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.109 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.110 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.111 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.112 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.114 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.115 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.116 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.117 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2024-04-21T09:18:33.118 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T09:18:33.119 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.121 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.122 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.123 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.124 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.125 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2024-04-21T09:18:33.126 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.127 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.128 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.129 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.130 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.131 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.132 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.133 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.135 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.136 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.137 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.138 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.139 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.141 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.142 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.144 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.145 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.147 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.148 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.149 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.150 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.151 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.152 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2024-04-21T09:18:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2024-04-21T09:18:33.154 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.155 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.156 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.157 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.158 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.159 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.160 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.161 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2024-04-21T09:18:33.162 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T09:18:33.163 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2024-04-21T09:18:33.164 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.169 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.171 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.172 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.173 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.174 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.175 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.176 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.177 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.178 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.179 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.180 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.181 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.182 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.183 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.184 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.185 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.186 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:33.187 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:33.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.190 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.191 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.192 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.193 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.194 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.195 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.196 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.197 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.201 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.202 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.205 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.206 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.207 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.208 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.210 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.211 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.212 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.213 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.214 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.215 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2024-04-21T09:18:33.216 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2024-04-21T09:18:33.217 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.218 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.219 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.220 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.221 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.222 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.223 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.226 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.227 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.229 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.230 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.231 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.232 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.233 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.234 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.235 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.236 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.237 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.238 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.239 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.240 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.241 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.242 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.243 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.244 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.245 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.246 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.248 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.249 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.250 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.251 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.252 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.254 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.255 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.256 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.257 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2024-04-21T09:18:33.258 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2024-04-21T09:18:33.259 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.260 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.261 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.262 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.264 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2024-04-21T09:18:33.265 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2024-04-21T09:18:33.266 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2024-04-21T09:18:33.267 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.269 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.270 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.271 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.274 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.275 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.276 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.277 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.278 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.279 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.280 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "periodic_async_work": { 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "periodic_async_work", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Periodically check for async work", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.281 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T09:18:33.282 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_no_wait": { 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T09:18:33.283 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.284 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.286 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.287 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.288 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.289 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.290 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.291 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2024-04-21T09:18:33.292 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2024-04-21T09:18:33.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2024-04-21T09:18:33.294 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.295 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 25118, 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "y", 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr_features": 4540701547738038271, 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.296 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.297 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.298 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.299 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.300 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.301 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2024-04-21T09:18:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2024-04-21T09:18:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2024-04-21T09:18:33.304 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2024-04-21T09:18:33.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.306 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.308 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.310 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.311 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2024-04-21T09:18:33.312 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.313 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.314 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T09:18:33.315 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2024-04-21T09:18:33.316 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2024-04-21T09:18:33.317 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.318 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.319 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.321 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.322 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.323 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.324 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.325 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "read", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap-read" 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.326 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.327 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.328 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T09:18:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.330 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.331 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.332 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.333 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.334 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.335 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.336 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm_log_destination": { 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm_log_destination", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.337 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "file", 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "file,syslog", 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "syslog" 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.338 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.339 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.340 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.342 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.343 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.344 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.345 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T09:18:33.346 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T09:18:33.347 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T09:18:33.348 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.349 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.350 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.352 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.353 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_nvmeof": { 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_nvmeof", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.354 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Nvme-of container image", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.355 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.356 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_samba": { 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_samba", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.357 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Samba/SMB container image", 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2024-04-21T09:18:33.358 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2024-04-21T09:18:33.359 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2024-04-21T09:18:33.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T09:18:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2024-04-21T09:18:33.362 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2024-04-21T09:18:33.363 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2024-04-21T09:18:33.364 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2024-04-21T09:18:33.365 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "grafana_dashboards_path": { 2024-04-21T09:18:33.366 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "grafana_dashboards_path", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2024-04-21T09:18:33.367 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "hw_monitoring": { 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hw_monitoring", 2024-04-21T09:18:33.368 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "inventory_list_all": { 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "inventory_list_all", 2024-04-21T09:18:33.369 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.370 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.371 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.372 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.373 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.374 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.376 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.377 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.382 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "oob_default_addr": { 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "oob_default_addr", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "169.254.1.1", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.383 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.384 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.385 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.386 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.387 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.388 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2024-04-21T09:18:33.390 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.391 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.392 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.393 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.394 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.395 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.396 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2024-04-21T09:18:33.397 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.398 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.399 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.400 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.402 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.403 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.404 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T09:18:33.405 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.406 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.407 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.409 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.410 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.411 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.412 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.413 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.414 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.415 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.416 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.417 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.418 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.419 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.420 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.421 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.422 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.423 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.424 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.426 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.427 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.428 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "[]", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.429 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "{}", 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.430 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.431 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.432 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T09:18:33.433 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T09:18:33.434 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.435 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.440 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.441 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.442 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.444 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.447 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.448 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.449 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.450 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.451 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.452 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.453 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.454 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2024-04-21T09:18:33.455 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.456 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.457 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.458 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.461 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.463 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.464 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.466 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.467 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.468 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.469 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.470 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.471 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.472 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.473 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.474 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.475 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.476 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.477 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.478 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.479 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.480 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.482 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.483 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.484 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.485 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.486 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.487 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.488 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.489 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.490 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.491 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.492 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.493 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.494 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2024-04-21T09:18:33.495 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.496 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.497 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.498 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.499 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.500 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.501 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.504 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.505 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.506 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.507 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2024-04-21T09:18:33.508 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.509 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.510 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.511 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.513 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.514 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.515 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.516 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.517 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.518 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.520 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.521 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.522 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.524 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.525 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.526 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.528 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.529 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.530 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.531 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.532 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.533 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.534 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.535 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.536 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2024-04-21T09:18:33.537 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2024-04-21T09:18:33.538 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T09:18:33.539 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.540 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.542 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.543 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.544 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.545 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.546 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.550 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.551 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2024-04-21T09:18:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.600 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.601 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.602 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.603 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.604 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.605 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.606 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.607 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.608 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.609 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.610 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.611 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.612 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.613 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.614 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.615 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.616 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.617 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.618 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2024-04-21T09:18:33.619 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2024-04-21T09:18:33.620 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.621 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.622 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.623 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.624 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.625 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.626 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.627 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.629 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.630 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.631 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.632 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.633 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.634 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.635 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.636 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.637 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.638 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.639 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.640 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.641 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.642 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2024-04-21T09:18:33.643 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.644 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.645 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.647 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.648 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.649 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.650 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2024-04-21T09:18:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.652 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.653 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.654 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.656 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.657 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2024-04-21T09:18:33.658 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2024-04-21T09:18:33.659 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2024-04-21T09:18:33.660 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2024-04-21T09:18:33.661 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.662 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.663 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.664 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.665 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.666 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.667 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.668 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.669 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2024-04-21T09:18:33.670 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.671 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.672 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.673 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.674 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.675 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.676 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.677 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.678 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.679 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.680 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.682 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.683 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.684 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.685 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.688 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2024-04-21T09:18:33.689 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.690 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.694 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.695 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.696 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.697 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.698 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.699 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.700 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.701 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.702 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.703 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.704 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.705 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.706 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.707 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.708 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.709 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.710 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.711 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.712 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.714 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.715 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.717 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.718 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.720 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.721 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "periodic_async_work": { 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "periodic_async_work", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.723 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Periodically check for async work", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.724 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_no_wait": { 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.725 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2024-04-21T09:18:33.726 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.727 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.728 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.729 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.730 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.733 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.734 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.735 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.736 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.737 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: "dashboard", 2024-04-21T09:18:33.738 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "restful", 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "snap_schedule" 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.739 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.740 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.741 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.742 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.743 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.744 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.745 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.746 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.747 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.748 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.749 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.750 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.751 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.752 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.753 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.754 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2024-04-21T09:18:33.755 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.756 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T09:18:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.758 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2024-04-21T09:18:33.759 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.761 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.762 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.763 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.764 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.765 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.766 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.767 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.768 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.769 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.770 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "read", 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap", 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap-read" 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2024-04-21T09:18:33.771 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.772 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2024-04-21T09:18:33.773 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2024-04-21T09:18:33.774 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.776 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.777 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.778 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.779 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.780 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.781 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm_log_destination": { 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm_log_destination", 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.782 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "file", 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "file,syslog", 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "syslog" 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.783 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.784 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.785 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2024-04-21T09:18:33.786 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2024-04-21T09:18:33.787 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2024-04-21T09:18:33.788 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2024-04-21T09:18:33.789 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.790 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.791 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.792 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.793 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.794 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2024-04-21T09:18:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.797 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_nvmeof": { 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_nvmeof", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Nvme-of container image", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.798 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.800 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_samba": { 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_samba", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Samba/SMB container image", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.803 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.804 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.805 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "grafana_dashboards_path": { 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "grafana_dashboards_path", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.811 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "hw_monitoring": { 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hw_monitoring", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.812 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inventory_list_all": { 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "inventory_list_all", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.813 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.814 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.814 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.814 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.814 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.814 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.814 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.816 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.817 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.818 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.819 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T09:18:33.820 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T09:18:33.821 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.822 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.823 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.824 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.825 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T09:18:33.826 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "oob_default_addr": { 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "oob_default_addr", 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "169.254.1.1", 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.827 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.828 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.829 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.830 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.831 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.832 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.833 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.835 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.836 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.837 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.838 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.839 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.840 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.841 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.842 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.843 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.844 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.845 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.846 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.847 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.848 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T09:18:33.849 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T09:18:33.850 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T09:18:33.851 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2024-04-21T09:18:33.852 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T09:18:33.853 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.855 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.856 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.858 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.859 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.860 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.861 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.862 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.863 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.864 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.865 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.866 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:33.867 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.868 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.869 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.870 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.871 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.872 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "[]", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.873 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "{}", 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T09:18:33.874 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T09:18:33.875 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.878 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.879 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.880 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.881 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.882 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.883 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.884 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.885 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.887 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.889 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.890 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.892 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.893 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.894 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.895 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.896 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.897 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.898 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.899 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2024-04-21T09:18:33.900 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.901 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.902 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.905 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.906 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.907 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.908 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2024-04-21T09:18:33.909 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.910 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.911 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.912 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.913 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2024-04-21T09:18:33.914 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2024-04-21T09:18:33.915 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.917 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.918 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.919 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.920 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.922 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.924 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.925 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2024-04-21T09:18:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2024-04-21T09:18:33.927 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.928 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.929 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.930 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.931 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.932 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.933 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.934 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.935 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.936 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.937 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.939 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T09:18:33.940 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T09:18:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.942 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.943 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.944 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.945 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.948 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.949 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2024-04-21T09:18:33.950 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.951 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.952 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.953 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2024-04-21T09:18:33.955 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.956 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.957 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.958 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.959 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.960 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.961 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.962 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.963 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.964 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.965 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2024-04-21T09:18:33.966 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.967 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.968 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.969 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.971 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.973 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2024-04-21T09:18:33.974 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.975 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.978 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.979 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.980 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.981 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.983 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.984 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2024-04-21T09:18:33.985 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.986 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.987 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.988 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.989 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.991 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.992 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.993 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.994 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.995 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:33.996 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:33.997 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.998 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.000 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.002 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.004 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.005 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.006 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.007 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2024-04-21T09:18:34.008 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.009 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.011 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.012 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.013 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.014 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2024-04-21T09:18:34.015 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.016 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.017 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.019 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.020 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.020 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.020 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.020 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.020 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.020 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.022 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.025 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.026 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:34.027 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2024-04-21T09:18:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.029 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.030 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.031 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.032 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.033 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.034 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.035 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.036 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2024-04-21T09:18:34.037 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2024-04-21T09:18:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2024-04-21T09:18:34.040 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.041 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.042 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.043 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.044 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.046 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.047 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.048 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.049 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T09:18:34.050 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T09:18:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2024-04-21T09:18:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2024-04-21T09:18:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2024-04-21T09:18:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2024-04-21T09:18:34.094 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2024-04-21T09:18:34.095 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.096 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.097 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2024-04-21T09:18:34.098 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2024-04-21T09:18:34.099 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.100 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.101 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.102 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.103 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.104 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.105 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.106 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.110 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.111 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.112 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.113 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.114 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.115 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.117 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.118 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.119 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.120 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.121 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.122 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T09:18:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.124 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.125 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.128 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.129 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.130 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.131 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.132 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.133 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.134 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.135 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.136 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.137 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.138 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.139 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.140 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.141 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.142 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.144 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.145 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.146 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.147 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.148 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.149 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.150 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.151 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.152 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.153 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.154 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.158 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.160 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.161 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.162 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.163 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.164 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.165 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.166 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.167 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "periodic_async_work": { 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "periodic_async_work", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Periodically check for async work", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.168 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.169 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_no_wait": { 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.170 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.171 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.172 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.173 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.174 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.175 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.176 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2024-04-21T09:18:34.177 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2024-04-21T09:18:34.178 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.179 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2024-04-21T09:18:34.180 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2024-04-21T09:18:34.181 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.182 INFO:teuthology.orchestra.run.smithi066.stdout: "services": { 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "dashboard": "https://172.21.15.66:8443/" 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "always_on_modules": { 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "octopus": [ 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2024-04-21T09:18:34.183 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "pacific": [ 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.184 INFO:teuthology.orchestra.run.smithi066.stdout: "quincy": [ 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "reef": [ 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2024-04-21T09:18:34.185 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "squid": [ 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2024-04-21T09:18:34.186 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "last_failure_osd_epoch": 96, 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "active_clients": [ 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2024-04-21T09:18:34.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1265654993 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "libcephsqlite", 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4187693870 2024-04-21T09:18:34.188 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3937663507 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.189 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3544846855 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2024-04-21T09:18:34.190 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3886909895 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 327820528 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:34.191 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-21T09:18:34.192 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:18:34.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:32 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2645049388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:32 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2030931603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.193 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:32 smithi083 ceph-mon[31413]: pgmap v321: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:34.194 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:32 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2489674301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.194 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:33 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3848574155' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T09:18:34.194 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:32 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2645049388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.194 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:32 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2030931603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.194 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:32 smithi186 ceph-mon[31505]: pgmap v321: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:34.194 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:32 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2489674301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.194 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:33 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3848574155' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbf740a3d20 msgr2=0x7fbf740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbf740a3d20 0x7fbf740a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000c00 tx=0x7fbf78005dd0 comp rx=0 tx=0).stop 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 msgr2=0x7fbf8819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf8819cfd0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009fd0 tx=0x7fbf8000b260 comp rx=0 tx=0).stop 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 shutdown_connections 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fbf740a3d20 0x7fbf740a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf88106f80 0x7fbf8819cfd0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:34.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88101240 0x7fbf88198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:34.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 --2- 172.21.15.66:0/3848574155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf88100880 0x7fbf88198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:18:34.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 >> 172.21.15.66:0/3848574155 conn(0x7fbf880761d0 msgr2=0x7fbf880fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:18:34.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 shutdown_connections 2024-04-21T09:18:34.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:18:32.837+0000 7fbf8d967640 1 -- 172.21.15.66:0/3848574155 wait complete. 2024-04-21T09:18:34.246 DEBUG:tasks.check_counter:Getting stats from a 2024-04-21T09:18:34.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-21T09:18:34.247 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:32 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2645049388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.247 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:32 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2030931603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.247 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:32 smithi066 ceph-mon[27953]: pgmap v321: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T09:18:34.247 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:32 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2489674301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:18:34.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:33 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3848574155' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T09:18:34.504 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:18:34.878 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:34 smithi066 ceph-mon[27953]: pgmap v322: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T09:18:35.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:34 smithi186 ceph-mon[31505]: pgmap v322: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T09:18:35.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:34 smithi083 ceph-mon[31413]: pgmap v322: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T09:18:36.364 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:18:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 4, 2024-04-21T09:18:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 4, 2024-04-21T09:18:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2944, 2024-04-21T09:18:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 448, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.001790690, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000040697, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000696425, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000788987, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000262995 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 219, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 209, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1450304, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 128544, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.040991736, 2024-04-21T09:18:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.014245213, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.029927681, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.004255423, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 208, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.011125691, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000053488 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1450304, 2024-04-21T09:18:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 128544 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 160, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 320, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 184144, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.013880054, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.009343951, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.002172432, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 157, 2024-04-21T09:18:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.016363802, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000104228 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 184144 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 149057, 2024-04-21T09:18:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 32, 2024-04-21T09:18:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-21T09:18:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-21T09:18:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-21T09:18:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 221, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1419437, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 221, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 221, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1419437, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:36.377 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:18:37.050 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-21T09:18:37.051 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-21T09:18:37.051 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-21T09:18:37.051 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-21T09:18:37.051 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-21T09:18:37.051 DEBUG:tasks.check_counter:Getting stats from d 2024-04-21T09:18:37.052 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-21T09:18:37.306 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:18:37.555 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:37 smithi066 ceph-mon[27953]: pgmap v323: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:37.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:37 smithi186 ceph-mon[31505]: pgmap v323: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:37.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:37 smithi083 ceph-mon[31413]: pgmap v323: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 226, 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 216, 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1452272, 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 132544, 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2024-04-21T09:18:39.002 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.042846830, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.014684378, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.030360490, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.004322581, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 215, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.010259565, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000047718 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.003 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1452272, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 132544 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 165, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 320, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 185024, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.012976706, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.008843225, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001820027, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:39.004 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 162, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.013887080, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000085722 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 185024 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:39.005 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 4, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 4, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2944, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 448, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.000678924, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000040607, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000627704, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000889569, 2024-04-21T09:18:39.006 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000296523 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-21T09:18:39.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:39.008 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:39.009 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 149017, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 31, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:39.010 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 228, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1420411, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 228, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-21T09:18:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 228, 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1420411, 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:39.013 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:18:39.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:39 smithi186 ceph-mon[31505]: pgmap v324: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:39.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:39 smithi083 ceph-mon[31413]: pgmap v324: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:39.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:39 smithi066 ceph-mon[27953]: pgmap v324: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:39.867 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-21T09:18:39.867 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-21T09:18:39.867 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-21T09:18:39.867 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-21T09:18:39.867 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-21T09:18:39.867 DEBUG:tasks.check_counter:Getting stats from g 2024-04-21T09:18:39.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-21T09:18:40.122 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:18:41.576 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:41 smithi066 ceph-mon[27953]: pgmap v325: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:41.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:41 smithi186 ceph-mon[31505]: pgmap v325: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:41.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:41 smithi083 ceph-mon[31413]: pgmap v325: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:41.848 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:18:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2024-04-21T09:18:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.001306487, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000176865, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 161, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 320, 2024-04-21T09:18:41.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 184320, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.014032006, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.009578824, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001982732, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 158, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.012961798, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000082036 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:41.851 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 184320 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 226, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 214, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1454128, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 130432, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.043411328, 2024-04-21T09:18:41.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.015100358, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.031758749, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.004454260, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 214, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.011030700, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000051545 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1454128, 2024-04-21T09:18:41.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 130432 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:41.854 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:41.855 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 149017, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 31, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:41.857 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-21T09:18:41.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 224, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1419891, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 224, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 224, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1419891, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:41.859 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:41.860 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:41.860 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:41.860 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:41.860 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:18:42.735 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-21T09:18:42.735 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-21T09:18:42.735 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-21T09:18:42.735 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-21T09:18:42.735 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-21T09:18:42.735 DEBUG:tasks.check_counter:Getting stats from j 2024-04-21T09:18:42.736 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-21T09:18:42.993 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T09:18:43.370 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:43 smithi066 ceph-mon[27953]: pgmap v326: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:43.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:43 smithi186 ceph-mon[31505]: pgmap v326: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:43.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:43 smithi083 ceph-mon[31413]: pgmap v326: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:44.805 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 232, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 220, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1455760, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 134272, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.046247827, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.014635252, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.035186910, 2024-04-21T09:18:44.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.004588820, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 220, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.010815529, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000049161 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1455760, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 134272 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.807 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 166, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 320, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 185200, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.014276856, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.010133578, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001644349, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 163, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.012655414, 2024-04-21T09:18:44.808 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000077640 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 185200 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2024-04-21T09:18:44.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.000263563, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000202315, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:44.810 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:44.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.811 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:44.811 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:44.811 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:44.811 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:44.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-21T09:18:44.812 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:44.813 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:44.814 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 148955, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 30, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-21T09:18:44.815 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:44.816 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 230, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1420671, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 230, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 230, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1420671, 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-21T09:18:44.817 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-21T09:18:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-21T09:18:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-21T09:18:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:44.818 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-21T09:18:44.818 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-21T09:18:45.544 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:45 smithi066 ceph-mon[27953]: pgmap v327: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:45.575 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-21T09:18:45.575 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-21T09:18:45.575 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-21T09:18:45.575 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-21T09:18:45.575 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-21T09:18:45.575 DEBUG:tasks.check_counter:Getting stats from b 2024-04-21T09:18:45.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-21T09:18:45.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:45 smithi186 ceph-mon[31505]: pgmap v327: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:45.602 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:45 smithi083 ceph-mon[31413]: pgmap v327: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:45.825 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:18:47.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:47 smithi083 ceph-mon[31413]: pgmap v328: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:47.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:47 smithi186 ceph-mon[31505]: pgmap v328: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:47.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:47 smithi066 ceph-mon[27953]: pgmap v328: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:47.983 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-21T09:18:47.983 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:47.983 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1396, 2024-04-21T09:18:47.983 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1130, 2024-04-21T09:18:47.983 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1966135, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 2167613, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 161, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.206175627, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.048097189, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.113603567, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.038091242, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1132, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.289042895, 2024-04-21T09:18:47.984 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000255338 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1167, 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000232651, 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000199 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1555488, 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 165696 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:47.985 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 942, 2024-04-21T09:18:47.986 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 897, 2024-04-21T09:18:47.986 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 333932, 2024-04-21T09:18:47.986 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 700740, 2024-04-21T09:18:47.986 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 11, 2024-04-21T09:18:47.986 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-21T09:18:47.986 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.107548462, 2024-04-21T09:18:47.986 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.039494394, 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.035744922, 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.028264795, 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 897, 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.133517562, 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000148849 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:47.987 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1087, 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000230245, 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000211 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 43472, 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 457520 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2311, 2024-04-21T09:18:47.988 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1109, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 888348, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1699076, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 39, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 9, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.268552541, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.062470113, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.098442181, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.103165098, 2024-04-21T09:18:47.989 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1102, 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.572174990, 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000519215 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2015, 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000354717, 2024-04-21T09:18:47.990 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000176 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 367296, 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 118288 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-21T09:18:47.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.992 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-21T09:18:47.992 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-21T09:18:47.992 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-21T09:18:47.992 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 698, 2024-04-21T09:18:47.992 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.189686899, 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000271757 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 87, 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003232776, 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000037158 2024-04-21T09:18:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 180, 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 180, 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 180, 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.158731472, 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000881841 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-21T09:18:47.994 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 60, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 39, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 7, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 32, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 29, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-21T09:18:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 3, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 3, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 254, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 242, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-21T09:18:47.996 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 222, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 4, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 234, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 6, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 222, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-21T09:18:47.997 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 138, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 19, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 119, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 84, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 1, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 15, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 34, 2024-04-21T09:18:47.998 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 100, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 80, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 3, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 28 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-21T09:18:47.999 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 3, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 2, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 6, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 4, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-21T09:18:48.000 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 19, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-21T09:18:48.001 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 242, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 160, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 160, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 82, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 24, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 10, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 9, 2024-04-21T09:18:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 9, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 1, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 1, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 1, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4989097, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 5128628, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 199, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.401539112, 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002017784 2024-04-21T09:18:48.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 29, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 52, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 23, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 20, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 20, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 39, 2024-04-21T09:18:48.004 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 43, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 4, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 3, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 25, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 22, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 58292, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 231800 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 227, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 180, 2024-04-21T09:18:48.005 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 36, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.005815453, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000726931 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 114, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.025457832, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000223314 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-21T09:18:48.007 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001063023, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000265755 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000135932, 2024-04-21T09:18:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000135932 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001837250, 2024-04-21T09:18:48.010 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000183725 2024-04-21T09:18:48.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.011 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-21T09:18:48.011 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8, 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.013442349, 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001680293 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-21T09:18:48.012 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.051720666, 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005172066 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000292350, 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000292350 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000911378, 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000455689 2024-04-21T09:18:48.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002005552, 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001002776 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.014 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-21T09:18:48.015 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.009474454, 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004737227 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.018875963, 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004718990 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002283098, 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001141549 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.017 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.018 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8, 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.020120641, 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002515080 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.005295531, 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001323882 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.020 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.021 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 6, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 4, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 27, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 13, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 610, 2024-04-21T09:18:48.022 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:48.023 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:48.024 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:48.025 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:48.026 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 343232, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 146, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 880, 2024-04-21T09:18:48.027 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 10, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 9512, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 182, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:48.028 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 134037, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 1185, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 1229, 2024-04-21T09:18:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 1033598, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 31, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 1229, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1229, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 21.179504700, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.017233120 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1229, 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1407 2024-04-21T09:18:48.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 1229, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 262, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 967, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 40, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 36, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 21, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 201, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 52, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-21T09:18:48.031 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 583, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 241, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 64, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-21T09:18:48.032 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-21T09:18:48.033 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-21T09:18:48.034 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 104, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 19, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 12, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 155, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 14 2024-04-21T09:18:48.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 65, 2024-04-21T09:18:48.036 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 3, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 71, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 4, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 4645, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 2704058, 2024-04-21T09:18:48.037 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 4645, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 4645, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2704058, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.038 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 1229, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 1034066, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 1229, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1034066, 2024-04-21T09:18:48.039 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:48.040 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 1229, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 1229, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 1229, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1229, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 4, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 468, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 4, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:48.042 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 468, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55e1d9d04500": { 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:48.043 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 242, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 934324, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 242, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 199, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 934324, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:48.044 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:48.045 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:48.045 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:48.045 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:48.045 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-21T09:18:48.814 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-21T09:18:48.814 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-21T09:18:48.814 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-21T09:18:48.814 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-21T09:18:48.814 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-21T09:18:48.814 DEBUG:tasks.check_counter:Getting stats from e 2024-04-21T09:18:48.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-21T09:18:49.097 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:18:49.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:49 smithi083 ceph-mon[31413]: pgmap v329: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:49.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:49 smithi186 ceph-mon[31505]: pgmap v329: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:49.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:49 smithi066 ceph-mon[27953]: pgmap v329: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:50.828 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-21T09:18:50.828 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:50.828 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 248, 2024-04-21T09:18:50.828 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 237, 2024-04-21T09:18:50.828 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1481858, 2024-04-21T09:18:50.828 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 137268, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.047597682, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.014552267, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.034336297, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.005372709, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 236, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.012694248, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000053789 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.829 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1481024, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 136944 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 648, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 810, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 161218, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 650769, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-21T09:18:50.830 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.072211651, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.026665203, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.014625911, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.022450532, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 808, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.050163443, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000062083 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.831 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 446608 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1520, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1522, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 455689, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 483418, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.157190183, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.056995692, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.038681053, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.047470910, 2024-04-21T09:18:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1520, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.064327316, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000042320 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-21T09:18:50.833 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3619, 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.111979187, 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000030942 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-21T09:18:50.834 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-21T09:18:50.835 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 1, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-04-21T09:18:50.836 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-21T09:18:50.837 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T09:18:50.838 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-21T09:18:50.839 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-21T09:18:50.840 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-21T09:18:50.841 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 5, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 2, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-21T09:18:50.842 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4195732, 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 4195591, 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 5 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 3, 2024-04-21T09:18:50.843 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 3, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 1, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 1, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 0, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 0, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-04-21T09:18:50.844 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 50424, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215416 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-21T09:18:50.845 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-21T09:18:50.846 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.846 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.846 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.846 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-21T09:18:50.846 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.847 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-21T09:18:50.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-21T09:18:50.849 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-21T09:18:50.850 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.851 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.851 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.852 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-21T09:18:50.853 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.854 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.856 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.857 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 2, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 1, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 3081, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.859 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:50.860 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:50.861 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:50.862 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:50.862 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:50.862 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:50.862 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 189393, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 51, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:50.863 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 9512, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 182, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-21T09:18:50.864 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 8512, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 13, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 2163, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 2163, 2024-04-21T09:18:50.865 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2163, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.041975544, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000481727 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2163, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2164 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 2163, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 2163, 2024-04-21T09:18:50.866 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 1397, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 764, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-21T09:18:50.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T09:18:50.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-21T09:18:50.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 104, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 13, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 5, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 5, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-21T09:18:50.870 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-21T09:18:50.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.872 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 2412, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1866080, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 2412, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2412, 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1866080, 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:50.874 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2163, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 1433, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2163, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1433, 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:50.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:50.876 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2163, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 2163, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2163, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2163, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.877 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:50.878 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x5623335c26e0": { 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-21T09:18:50.879 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:50.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:50.881 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:50.881 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:50.881 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.881 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:50.881 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-21T09:18:51.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:51 smithi186 ceph-mon[31505]: pgmap v330: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:51.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:51 smithi083 ceph-mon[31413]: pgmap v330: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:51.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:51 smithi066 ceph-mon[27953]: pgmap v330: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:51.690 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-21T09:18:51.690 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-21T09:18:51.690 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-21T09:18:51.691 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-21T09:18:51.691 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T09:18:51.691 DEBUG:tasks.check_counter:Getting stats from h 2024-04-21T09:18:51.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-21T09:18:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:18:53.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:53 smithi083 ceph-mon[31413]: pgmap v331: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:53.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:53 smithi186 ceph-mon[31505]: pgmap v331: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:53.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:53 smithi066 ceph-mon[27953]: pgmap v331: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 40, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 202, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 9019, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 386872, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 15, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.022967017, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.008521999, 2024-04-21T09:18:53.684 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.008234158, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000581138, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 200, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.026175859, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000130879 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 55, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000024024, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000436 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 318880 2024-04-21T09:18:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 304, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 289, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1498950, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 225440, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 12, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.050720002, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.015961243, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.036744665, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.006061207, 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:53.686 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 289, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.028371691, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000098171 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000816, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000136 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1493712, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 141552 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 540, 2024-04-21T09:18:53.687 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 690, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 144805, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 760791, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 43, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.078405854, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.032939155, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.040490921, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.003646072, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 687, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.037352566, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000054370 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.688 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1124, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000272729, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000242 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 129696 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-21T09:18:53.689 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 51, 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001531147, 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000030022 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000007107, 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000003553 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.690 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 3, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 2, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000831173, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000415586 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 1, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 11, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-21T09:18:53.691 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 14, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 14, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 14, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 2, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 2, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 3, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 1, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 1, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-21T09:18:53.692 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 4, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 234, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 6, 2024-04-21T09:18:53.693 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 2, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 3, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T09:18:53.694 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 2, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 1 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 2, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-21T09:18:53.695 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-21T09:18:53.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 5, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 5, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 2, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 2, 2024-04-21T09:18:53.697 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4195732, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.009368675, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002342168 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-21T09:18:53.698 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 17, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 17, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 15, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 16, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 1, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 14, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 14, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 2, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 2, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 52516, 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 223608 2024-04-21T09:18:53.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 4, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 3, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 13, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000715112, 2024-04-21T09:18:53.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000715112 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.701 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000116061, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000116061 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-21T09:18:53.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.703 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-21T09:18:53.704 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.704 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.704 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.704 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-21T09:18:53.704 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.705 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-21T09:18:53.706 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.707 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.709 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.710 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 2, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 1, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 1, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 640, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-21T09:18:53.712 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:53.713 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:53.714 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 278583, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 89, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-21T09:18:53.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 9512, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 182, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 68413, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 291, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-21T09:18:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 42, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 12135, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 33, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 42, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 42, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.225994759, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005380827 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.717 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 42, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 66 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 42, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 0, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 42, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 10, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 0, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 4, 2024-04-21T09:18:53.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 7, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 6, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 20, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-21T09:18:53.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-21T09:18:53.720 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 104, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 13, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 9, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 9, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-21T09:18:53.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 18, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 1 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.722 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 880, 2024-04-21T09:18:53.723 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1556373, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 880, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 880, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1556373, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-21T09:18:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 42, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 12135, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 42, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 12135, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 42, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 42, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 42, 2024-04-21T09:18:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 42, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:53.727 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x559941e86500": { 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:53.728 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 5, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1428, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 5, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 4, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1428, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:53.729 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-21T09:18:54.571 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-21T09:18:54.571 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-21T09:18:54.572 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-21T09:18:54.572 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-21T09:18:54.572 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T09:18:54.572 DEBUG:tasks.check_counter:Getting stats from k 2024-04-21T09:18:54.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-21T09:18:54.807 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T09:18:55.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:55 smithi186 ceph-mon[31505]: pgmap v332: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:55.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:55 smithi083 ceph-mon[31413]: pgmap v332: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:55.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:55 smithi066 ceph-mon[27953]: pgmap v332: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:56.661 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-21T09:18:56.661 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:56.661 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 236, 2024-04-21T09:18:56.661 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 224, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1456848, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 136832, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.047349570, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.014341639, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.035012965, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.005461243, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 224, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.014167948, 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000063249 2024-04-21T09:18:56.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1456848, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 136832 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 169, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 185728, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-21T09:18:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.012023108, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.007770158, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001865791, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 166, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.019390272, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000116808 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:56.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 185728 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-21T09:18:56.665 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.000293330, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000233537, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:56.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.667 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:56.667 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:56.667 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:56.667 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:56.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.669 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T09:18:56.670 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T09:18:56.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.670 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-21T09:18:56.670 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-21T09:18:56.670 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-21T09:18:56.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:56.671 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:56.672 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:56.672 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:56.672 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:56.672 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:56.672 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:56.672 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:56.672 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:56.673 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:56.674 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:56.674 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:56.674 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:56.674 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:56.674 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:56.675 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:56.675 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-21T09:18:56.675 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:56.675 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:56.675 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:56.676 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:56.676 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148714, 2024-04-21T09:18:56.676 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-21T09:18:56.676 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:56.676 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:56.676 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-21T09:18:56.676 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-21T09:18:56.677 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:56.677 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:56.677 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:56.677 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:56.677 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-21T09:18:56.678 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-21T09:18:56.678 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:56.678 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:56.678 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:56.678 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-21T09:18:56.678 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:56.679 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 234, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1421191, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 234, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-21T09:18:56.680 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 234, 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1421191, 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-21T09:18:56.681 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-21T09:18:57.359 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:57 smithi083 ceph-mon[31413]: pgmap v333: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:57.411 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-21T09:18:57.411 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-21T09:18:57.412 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-21T09:18:57.412 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-21T09:18:57.412 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-21T09:18:57.412 DEBUG:tasks.check_counter:Getting stats from c 2024-04-21T09:18:57.412 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-21T09:18:57.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:57 smithi186 ceph-mon[31505]: pgmap v333: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:57.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:57 smithi066 ceph-mon[27953]: pgmap v333: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:18:57.661 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:18:59.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:18:59 smithi186 ceph-mon[31505]: pgmap v334: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:59.373 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-21T09:18:59.373 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:18:59.373 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1645, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1645, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1422665, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 522296, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.164386205, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.053260749, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.042754370, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.052142743, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1644, 2024-04-21T09:18:59.374 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.068200350, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000041484 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 253, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 242, 2024-04-21T09:18:59.375 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1483112, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 139082, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.046939868, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.015377808, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.032535770, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005956105, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 241, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.016439664, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000068214 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.376 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1481456, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 137744 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 769, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 933, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 189253, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 693157, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.085819632, 2024-04-21T09:18:59.377 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.034200002, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.017319884, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.025312927, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 930, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.043137185, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000046384 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.378 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T09:18:59.379 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 449568 2024-04-21T09:18:59.379 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.379 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-21T09:18:59.383 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-21T09:18:59.383 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-21T09:18:59.383 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3983, 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.116660422, 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000029289 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-21T09:18:59.384 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-21T09:18:59.385 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 31, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 6, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 25, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 24, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-21T09:18:59.386 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 2, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-21T09:18:59.387 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 4, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 234, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 6, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T09:18:59.388 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-21T09:18:59.389 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-21T09:18:59.390 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-21T09:18:59.391 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 242, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 10, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4989097, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 5128628, 2024-04-21T09:18:59.392 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 5128492, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 242 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 20, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 24, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 4, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 11, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 11, 2024-04-21T09:18:59.393 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 31, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 31, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 52956, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215416 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-21T09:18:59.394 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.395 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.396 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.397 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.398 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.398 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.398 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-21T09:18:59.398 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.398 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.398 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.399 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.399 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-21T09:18:59.399 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.399 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.399 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.399 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.399 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.400 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.401 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-21T09:18:59.402 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.403 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.404 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.405 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-21T09:18:59.406 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 6, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 4, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 4, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-21T09:18:59.407 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 3084, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:18:59.408 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:18:59.409 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:18:59.410 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 189760, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 62, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 9512, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 182, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:18:59.411 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 92549, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 760, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2411, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 26, 2024-04-21T09:18:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 4, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2411, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2411, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.252977256, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000519691 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2411, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2412 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2411, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2411, 2024-04-21T09:18:59.413 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1530, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 878, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-21T09:18:59.414 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 2, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-21T09:18:59.415 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-21T09:18:59.416 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 104, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 13, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 5, 2024-04-21T09:18:59.417 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 5, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.418 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-21T09:18:59.419 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2663, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 2839657, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2663, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:18:59.420 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2663, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2839657, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:18:59.421 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2411, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 934334, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2411, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 934334, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.422 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2411, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2411, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2411, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2411, 2024-04-21T09:18:59.423 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:59.424 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55a0802046e0": { 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:18:59.425 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:18:59.426 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-21T09:18:59.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:18:59 smithi083 ceph-mon[31413]: pgmap v334: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:18:59.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:18:59 smithi066 ceph-mon[27953]: pgmap v334: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T09:19:00.179 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-21T09:19:00.179 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-21T09:19:00.179 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-21T09:19:00.179 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-21T09:19:00.179 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T09:19:00.179 DEBUG:tasks.check_counter:Getting stats from f 2024-04-21T09:19:00.179 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-21T09:19:00.434 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:19:01.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:01 smithi186 ceph-mon[31505]: pgmap v335: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:19:01.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:01 smithi083 ceph-mon[31413]: pgmap v335: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:19:01.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:01 smithi066 ceph-mon[27953]: pgmap v335: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1017, 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1006, 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1739616, 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 382448, 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2024-04-21T09:19:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.147418254, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.044690498, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.061466723, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.039354984, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1005, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.055299881, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000055024 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1481568, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 138224 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 4, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 170, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 876, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 452058, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.010902512, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006796992, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001971310, 2024-04-21T09:19:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000076308, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 167, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.015355032, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000091946 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 451360 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1429, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1428, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 368426, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 453287, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.104645844, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.035144248, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.026003981, 2024-04-21T09:19:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.035977285, 2024-04-21T09:19:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1427, 2024-04-21T09:19:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.053790210, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000037694 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-21T09:19:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3732, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.110083744, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000029497 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-21T09:19:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-04-21T09:19:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-21T09:19:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-21T09:19:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-21T09:19:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-21T09:19:02.323 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-21T09:19:02.324 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 6, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4196123, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 4195982, 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-21T09:19:02.325 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 6 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 3, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 3, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 1, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 1, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 0, 2024-04-21T09:19:02.326 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 49836, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215416 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-21T09:19:02.327 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.328 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.329 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-21T09:19:02.330 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-21T09:19:02.331 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.332 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.332 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-21T09:19:02.333 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.334 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.336 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.337 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.338 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.339 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 3, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 3, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 2476, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:19:02.340 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:19:02.341 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:19:02.342 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 181353, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 48, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:19:02.343 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 9512, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 182, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 8512, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 13, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-21T09:19:02.344 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2194, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2194, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2194, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.281976696, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000584310 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.345 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2194, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2195 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2194, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2194, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1416, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 776, 2024-04-21T09:19:02.346 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-21T09:19:02.347 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-21T09:19:02.348 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 104, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 13, 2024-04-21T09:19:02.349 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 4, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 4, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.350 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:19:02.351 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2446, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1873069, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2446, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2446, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1873069, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.352 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2194, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 1824, 2024-04-21T09:19:02.353 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2194, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1824, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:19:02.354 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2194, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2194, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2194, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2194, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-21T09:19:02.355 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.356 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x5603de5f46e0": { 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-21T09:19:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:02.358 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:02.358 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:02.358 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:02.358 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.358 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:02.358 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-21T09:19:03.172 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-21T09:19:03.172 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-21T09:19:03.172 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-21T09:19:03.172 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-21T09:19:03.172 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T09:19:03.172 DEBUG:tasks.check_counter:Getting stats from i 2024-04-21T09:19:03.173 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-21T09:19:03.411 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:19:03.533 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:03 smithi186 ceph-mon[31505]: pgmap v336: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:19:03.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:03 smithi083 ceph-mon[31413]: pgmap v336: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:19:03.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:03 smithi066 ceph-mon[27953]: pgmap v336: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T09:19:05.212 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-21T09:19:05.212 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 698, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 860, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1604368, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 464095, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.114973587, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.041688305, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.064195263, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.011043433, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 859, 2024-04-21T09:19:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.050478869, 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000058764 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 975, 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000339000, 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000347 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1494496, 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 141584 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:19:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 71, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 253, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 14776, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 958426, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 54, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.031187416, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.009430024, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.018466566, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000211705, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 249, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.029219029, 2024-04-21T09:19:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000117345 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 84, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000013178, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000156 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 5920, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 460096 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 69, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 69, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 23359, 2024-04-21T09:19:05.216 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 55023, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 9, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.008104054, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.002024829, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.005968604, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000450999, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 69, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.021403784, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000310199 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:05.217 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 103, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000024569, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000238 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-21T09:19:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 36, 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.002144274, 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000059563 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-21T09:19:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000007782, 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000003891 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 1, 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-21T09:19:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 1, 2024-04-21T09:19:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-21T09:19:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-21T09:19:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 10, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 10, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 10, 2024-04-21T09:19:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 1000, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 1, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-21T09:19:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 1, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 4, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 234, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 6, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T09:19:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 1, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-21T09:19:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-21T09:19:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 6, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 6, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 2, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-21T09:19:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4196123, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 5, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.009240600, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001848120 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 13, 2024-04-21T09:19:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 13, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 11, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 12, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 1, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 10, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 10, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 55212, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 223608 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.231 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 1, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 1, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 22, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.232 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-21T09:19:05.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-21T09:19:05.235 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.236 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.236 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.236 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.236 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-21T09:19:05.236 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-21T09:19:05.238 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.239 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.240 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.241 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.242 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.243 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 3, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 619, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-21T09:19:05.244 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:19:05.245 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:19:05.246 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 246327, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 81, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-21T09:19:05.247 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:19:05.248 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 9512, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 182, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 50872, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 198, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-21T09:19:05.250 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 42, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 12211, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 28, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 42, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 42, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.226995947, 2024-04-21T09:19:05.251 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.005404665 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 42, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 64 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 42, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 0, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 42, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-04-21T09:19:05.252 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 5, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 8, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 6, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-21T09:19:05.253 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-21T09:19:05.254 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 104, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 13, 2024-04-21T09:19:05.255 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 9, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 9, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 14, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 1 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.256 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:05.257 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 834, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1549896, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 834, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 834, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1549896, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.258 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 42, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 12211, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 42, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 12211, 2024-04-21T09:19:05.259 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:19:05.260 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 42, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 42, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 42, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 42, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:05.261 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:05.262 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x5595c9d3a500": { 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 6, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1819, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 6, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 5, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1819, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:05.264 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:05.264 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.264 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:05.264 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-21T09:19:05.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:05 smithi186 ceph-mon[31505]: pgmap v337: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:19:05.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:05 smithi083 ceph-mon[31413]: pgmap v337: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:19:05.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:05 smithi066 ceph-mon[27953]: pgmap v337: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T09:19:06.095 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-21T09:19:06.096 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-21T09:19:06.096 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-21T09:19:06.096 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-21T09:19:06.096 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T09:19:06.096 DEBUG:tasks.check_counter:Getting stats from l 2024-04-21T09:19:06.096 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-21T09:19:06.337 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T09:19:07.495 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:07 smithi186 ceph-mon[31505]: pgmap v338: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:19:07.496 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:07 smithi186 ceph-mon[31505]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T09:19:07.609 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:07 smithi083 ceph-mon[31413]: pgmap v338: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:19:07.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:07 smithi083 ceph-mon[31413]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T09:19:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:07 smithi066 ceph-mon[27953]: pgmap v338: 129 pgs: 129 active+clean; 2.2 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T09:19:07.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:07 smithi066 ceph-mon[27953]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T09:19:08.099 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.001091339, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T09:19:08.100 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000292847, 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:08.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 6, 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 175, 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3264, 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 186880, 2024-04-21T09:19:08.102 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-04-21T09:19:08.103 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-21T09:19:08.103 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.011952739, 2024-04-21T09:19:08.103 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.007611358, 2024-04-21T09:19:08.103 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001927632, 2024-04-21T09:19:08.106 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T09:19:08.106 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:08.106 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 173, 2024-04-21T09:19:08.106 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.020508904, 2024-04-21T09:19:08.106 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000118548 2024-04-21T09:19:08.107 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.107 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:08.107 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:08.107 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:08.107 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:08.107 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.107 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-21T09:19:08.108 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 186880 2024-04-21T09:19:08.108 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.108 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 235, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 225, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1454720, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 138304, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.046384117, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.014378709, 2024-04-21T09:19:08.109 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.033490929, 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005905901, 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 224, 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.013395861, 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000059802 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:08.110 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:08.111 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:08.111 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.111 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1454720, 2024-04-21T09:19:08.111 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 138304 2024-04-21T09:19:08.111 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.111 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T09:19:08.112 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T09:19:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T09:19:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T09:19:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T09:19:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-21T09:19:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T09:19:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T09:19:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-21T09:19:08.114 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-21T09:19:08.114 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-21T09:19:08.114 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-21T09:19:08.114 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T09:19:08.114 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T09:19:08.114 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T09:19:08.114 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-21T09:19:08.115 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-21T09:19:08.116 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148714, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 30, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-21T09:19:08.117 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-21T09:19:08.118 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 237, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1421581, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 237, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 237, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1421581, 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-21T09:19:08.119 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-21T09:19:08.120 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-21T09:19:08.120 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-21T09:19:08.120 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:08.120 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-21T09:19:08.120 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-21T09:19:08.977 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-21T09:19:08.978 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-21T09:19:08.978 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-21T09:19:08.978 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-21T09:19:08.978 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-21T09:19:08.978 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-21T09:19:09.459 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T09:19:09.459 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:19:09.459 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T09:19:09.492 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:19:09.493 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T09:19:09.560 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:19:09.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:09 smithi186 ceph-mon[31505]: pgmap v340: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:19:09.610 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:09 smithi083 ceph-mon[31413]: pgmap v340: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:19:09.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:09 smithi066 ceph-mon[27953]: pgmap v340: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T09:19:09.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3631001241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f01011a0 msgr2=0x7f35f0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3631001241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f01011a0 0x7f35f0101620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009930 tx=0x7f35d802bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3631001241 shutdown_connections 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3631001241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f0101b60 0x7f35f010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3631001241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f01011a0 0x7f35f0101620 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3631001241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35f0105e30 0x7f35f0106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3631001241 >> 172.21.15.66:0/3631001241 conn(0x7f35f0077e30 msgr2=0x7f35f0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3631001241 shutdown_connections 2024-04-21T09:19:09.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3631001241 wait complete. 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 Processor -- start 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.724+0000 7f35f5b9d640 1 -- start start 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35f5b9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f01011a0 0x7f35f0197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35f5b9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0101b60 0x7f35f0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35f5b9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35f0105e30 0x7f35f019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35f5b9d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01110c0 con 0x7f35f01011a0 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35f5b9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0110f40 con 0x7f35f0105e30 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35f5b9d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0111240 con 0x7f35f0101b60 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f01011a0 0x7f35f0197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f01011a0 0x7f35f0197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38424/0 (socket says 172.21.15.66:38424) 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 -- 172.21.15.66:0/3235582129 learned_addr learned my addr 172.21.15.66:0/3235582129 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:09.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35e6ffd640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0101b60 0x7f35f0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35effff640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35f0105e30 0x7f35f019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 -- 172.21.15.66:0/3235582129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0101b60 msgr2=0x7f35f0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0101b60 0x7f35f0197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 -- 172.21.15.66:0/3235582129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35f0105e30 msgr2=0x7f35f019bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35f0105e30 0x7f35f019bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 -- 172.21.15.66:0/3235582129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f019c630 con 0x7f35f01011a0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35effff640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35f0105e30 0x7f35f019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.725+0000 7f35ef7fe640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f01011a0 0x7f35f0197640 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009900 tx=0x7f35e0009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.726+0000 7f35ed7fa640 1 -- 172.21.15.66:0/3235582129 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e000d030 con 0x7f35f01011a0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.726+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f019c920 con 0x7f35f01011a0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.726+0000 7f35ed7fa640 1 -- 172.21.15.66:0/3235582129 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e000d1d0 con 0x7f35f01011a0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.726+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f00690e0 con 0x7f35f01011a0 2024-04-21T09:19:09.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.726+0000 7f35ed7fa640 1 -- 172.21.15.66:0/3235582129 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0005280 con 0x7f35f01011a0 2024-04-21T09:19:09.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.727+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f0068de0 con 0x7f35f01011a0 2024-04-21T09:19:09.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.729+0000 7f35ed7fa640 1 -- 172.21.15.66:0/3235582129 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f35e0005420 con 0x7f35f01011a0 2024-04-21T09:19:09.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.730+0000 7f35ed7fa640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f35dc0a3d20 0x7f35dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:09.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.730+0000 7f35e6ffd640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f35dc0a3d20 0x7f35dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:09.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.730+0000 7f35ed7fa640 1 -- 172.21.15.66:0/3235582129 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9584+0+0 (secure 0 0 0) 0x7f35e00d2370 con 0x7f35f01011a0 2024-04-21T09:19:09.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.730+0000 7f35e6ffd640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f35dc0a3d20 0x7f35dc0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005ab0 tx=0x7f35d8005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:09.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.733+0000 7f35ed7fa640 1 -- 172.21.15.66:0/3235582129 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35e009e5f0 con 0x7f35f01011a0 2024-04-21T09:19:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.924+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35f0198990 con 0x7f35f01011a0 2024-04-21T09:19:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.925+0000 7f35ed7fa640 1 -- 172.21.15.66:0/3235582129 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+25272 (secure 0 0 0) 0x7f35e009def0 con 0x7f35f01011a0 2024-04-21T09:19:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:19:09.927 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":106,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:19:09.808716+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:19:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f35dc0a3d20 msgr2=0x7f35dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f35dc0a3d20 0x7f35dc0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005ab0 tx=0x7f35d8005fd0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f01011a0 msgr2=0x7f35f0197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f01011a0 0x7f35f0197640 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009900 tx=0x7f35e0009dc0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 shutdown_connections 2024-04-21T09:19:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f35dc0a3d20 0x7f35dc0a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35f0105e30 0x7f35f019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0101b60 0x7f35f0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 --2- 172.21.15.66:0/3235582129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f01011a0 0x7f35f0197640 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.927+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 >> 172.21.15.66:0/3235582129 conn(0x7f35f0077e30 msgr2=0x7f35f00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.928+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 shutdown_connections 2024-04-21T09:19:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:09.928+0000 7f35f5b9d640 1 -- 172.21.15.66:0/3235582129 wait complete. 2024-04-21T09:19:09.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T09:01:06.302362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T09:05:44.770752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T09:05:45.280115+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T09:05:48.992312+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T09:19:09.940 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T09:19:10.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 -- 172.21.15.66:0/3279321275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c102b40 msgr2=0x7f396c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 --2- 172.21.15.66:0/3279321275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c102b40 0x7f396c106b80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3954009900 tx=0x7f395402bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:10.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 -- 172.21.15.66:0/3279321275 shutdown_connections 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 --2- 172.21.15.66:0/3279321275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f396c107260 0x7f396c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 --2- 172.21.15.66:0/3279321275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c102b40 0x7f396c106b80 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 --2- 172.21.15.66:0/3279321275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c102190 0x7f396c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 -- 172.21.15.66:0/3279321275 >> 172.21.15.66:0/3279321275 conn(0x7f396c0fd990 msgr2=0x7f396c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 -- 172.21.15.66:0/3279321275 shutdown_connections 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 -- 172.21.15.66:0/3279321275 wait complete. 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.088+0000 7f3971f24640 1 Processor -- start 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f3971f24640 1 -- start start 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f3971f24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f396c102190 0x7f396c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f3971f24640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c107260 0x7f396c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f3971f24640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c1985b0 0x7f396c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f3971f24640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f396c1121a0 con 0x7f396c1985b0 2024-04-21T09:19:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f3971f24640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f396c112020 con 0x7f396c102190 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f3971f24640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f396c112320 con 0x7f396c107260 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c1985b0 0x7f396c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c1985b0 0x7f396c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38448/0 (socket says 172.21.15.66:38448) 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 -- 172.21.15.66:0/45846500 learned_addr learned my addr 172.21.15.66:0/45846500 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396affd640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c107260 0x7f396c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396b7fe640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f396c102190 0x7f396c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 -- 172.21.15.66:0/45846500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c107260 msgr2=0x7f396c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c107260 0x7f396c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 -- 172.21.15.66:0/45846500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f396c102190 msgr2=0x7f396c19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f396c102190 0x7f396c19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 -- 172.21.15.66:0/45846500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396c1990d0 con 0x7f396c1985b0 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.089+0000 7f396bfff640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c1985b0 0x7f396c198a60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f3964009fd0 tx=0x7f396400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:10.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.090+0000 7f3968ff9640 1 -- 172.21.15.66:0/45846500 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964017070 con 0x7f396c1985b0 2024-04-21T09:19:10.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.090+0000 7f3968ff9640 1 -- 172.21.15.66:0/45846500 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3964003e20 con 0x7f396c1985b0 2024-04-21T09:19:10.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.090+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396c1993c0 con 0x7f396c1985b0 2024-04-21T09:19:10.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.090+0000 7f3968ff9640 1 -- 172.21.15.66:0/45846500 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964010460 con 0x7f396c1985b0 2024-04-21T09:19:10.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.090+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f396c1a5610 con 0x7f396c1985b0 2024-04-21T09:19:10.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.091+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396c0691c0 con 0x7f396c1985b0 2024-04-21T09:19:10.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.092+0000 7f3968ff9640 1 -- 172.21.15.66:0/45846500 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3964003970 con 0x7f396c1985b0 2024-04-21T09:19:10.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.092+0000 7f3968ff9640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f39580a3da0 0x7f39580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.092+0000 7f396b7fe640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f39580a3da0 0x7f39580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.093+0000 7f3968ff9640 1 -- 172.21.15.66:0/45846500 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f39640d07e0 con 0x7f396c1985b0 2024-04-21T09:19:10.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.093+0000 7f396b7fe640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f39580a3da0 0x7f39580a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f395c005fd0 tx=0x7f395c00b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:10.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.097+0000 7f3968ff9640 1 -- 172.21.15.66:0/45846500 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f396409c460 con 0x7f396c1985b0 2024-04-21T09:19:10.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.281+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f396c10a490 con 0x7f396c1985b0 2024-04-21T09:19:10.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.282+0000 7f3968ff9640 1 -- 172.21.15.66:0/45846500 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f39640a0760 con 0x7f396c1985b0 2024-04-21T09:19:10.283 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-04-21T09:19:10.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.283+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f39580a3da0 msgr2=0x7f39580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.283+0000 7f3971f24640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f39580a3da0 0x7f39580a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f395c005fd0 tx=0x7f395c00b390 comp rx=0 tx=0).stop 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.283+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c1985b0 msgr2=0x7f396c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.283+0000 7f3971f24640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c1985b0 0x7f396c198a60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f3964009fd0 tx=0x7f396400b260 comp rx=0 tx=0).stop 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 shutdown_connections 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f39580a3da0 0x7f39580a6260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f396c1985b0 0x7f396c198a60 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c107260 0x7f396c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 --2- 172.21.15.66:0/45846500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f396c102190 0x7f396c19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 >> 172.21.15.66:0/45846500 conn(0x7f396c0fd990 msgr2=0x7f396c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 shutdown_connections 2024-04-21T09:19:10.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.284+0000 7f3971f24640 1 -- 172.21.15.66:0/45846500 wait complete. 2024-04-21T09:19:10.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 -- 172.21.15.66:0/2075500985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 msgr2=0x7f306c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 --2- 172.21.15.66:0/2075500985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c101130 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f3064009900 tx=0x7f306402bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 -- 172.21.15.66:0/2075500985 shutdown_connections 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 --2- 172.21.15.66:0/2075500985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f306c10e900 0x7f306c110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 --2- 172.21.15.66:0/2075500985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c101670 0x7f306c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 --2- 172.21.15.66:0/2075500985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c101130 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 -- 172.21.15.66:0/2075500985 >> 172.21.15.66:0/2075500985 conn(0x7f306c077f50 msgr2=0x7f306c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:10.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 -- 172.21.15.66:0/2075500985 shutdown_connections 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.403+0000 7f307352b640 1 -- 172.21.15.66:0/2075500985 wait complete. 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 Processor -- start 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 -- start start 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c101670 0x7f306c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f306c10e900 0x7f306c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f306c114430 con 0x7f306c100d50 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1142b0 con 0x7f306c10e900 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f307352b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1145b0 con 0x7f306c101670 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f30712c7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f30712c7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38464/0 (socket says 172.21.15.66:38464) 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f30712c7640 1 -- 172.21.15.66:0/4098599929 learned_addr learned my addr 172.21.15.66:0/4098599929 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.404+0000 7f3070ac6640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c101670 0x7f306c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f3071ac8640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f306c10e900 0x7f306c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f30712c7640 1 -- 172.21.15.66:0/4098599929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c101670 msgr2=0x7f306c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f30712c7640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c101670 0x7f306c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f30712c7640 1 -- 172.21.15.66:0/4098599929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f306c10e900 msgr2=0x7f306c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f30712c7640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f306c10e900 0x7f306c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f30712c7640 1 -- 172.21.15.66:0/4098599929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c19afe0 con 0x7f306c100d50 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f30712c7640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c1a06a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f3064005940 tx=0x7f30640040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f306a7fc640 1 -- 172.21.15.66:0/4098599929 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064043020 con 0x7f306c100d50 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c19b270 con 0x7f306c100d50 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f306a7fc640 1 -- 172.21.15.66:0/4098599929 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30640052b0 con 0x7f306c100d50 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f306a7fc640 1 -- 172.21.15.66:0/4098599929 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064003d00 con 0x7f306c100d50 2024-04-21T09:19:10.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.405+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1a74f0 con 0x7f306c100d50 2024-04-21T09:19:10.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.406+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306c0698e0 con 0x7f306c100d50 2024-04-21T09:19:10.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.407+0000 7f306a7fc640 1 -- 172.21.15.66:0/4098599929 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3064046020 con 0x7f306c100d50 2024-04-21T09:19:10.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.408+0000 7f306a7fc640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30540a3d20 0x7f30540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.408+0000 7f3070ac6640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30540a3d20 0x7f30540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.408+0000 7f306a7fc640 1 -- 172.21.15.66:0/4098599929 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f30640f2f80 con 0x7f306c100d50 2024-04-21T09:19:10.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.408+0000 7f3070ac6640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30540a3d20 0x7f30540a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f306c100090 tx=0x7f3060008040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:10.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.412+0000 7f306a7fc640 1 -- 172.21.15.66:0/4098599929 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f306404d020 con 0x7f306c100d50 2024-04-21T09:19:10.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.600+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f306c109a90 con 0x7f306c100d50 2024-04-21T09:19:10.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.600+0000 7f306a7fc640 1 -- 172.21.15.66:0/4098599929 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f30640dde20 con 0x7f306c100d50 2024-04-21T09:19:10.603 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:19:10.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30540a3d20 msgr2=0x7f30540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30540a3d20 0x7f30540a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f306c100090 tx=0x7f3060008040 comp rx=0 tx=0).stop 2024-04-21T09:19:10.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 msgr2=0x7f306c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c1a06a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f3064005940 tx=0x7f30640040e0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 shutdown_connections 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f30540a3d20 0x7f30540a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f306c10e900 0x7f306c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c101670 0x7f306c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 --2- 172.21.15.66:0/4098599929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306c100d50 0x7f306c1a06a0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.602+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 >> 172.21.15.66:0/4098599929 conn(0x7f306c077f50 msgr2=0x7f306c10ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.603+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 shutdown_connections 2024-04-21T09:19:10.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.603+0000 7f307352b640 1 -- 172.21.15.66:0/4098599929 wait complete. 2024-04-21T09:19:10.614 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T09:19:10.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 -- 172.21.15.66:0/4279943032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc101a50 msgr2=0x7f21fc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 --2- 172.21.15.66:0/4279943032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc101a50 0x7f21fc105a90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009900 tx=0x7f21f002bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 -- 172.21.15.66:0/4279943032 shutdown_connections 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 --2- 172.21.15.66:0/4279943032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21fc106170 0x7f21fc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 --2- 172.21.15.66:0/4279943032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc101a50 0x7f21fc105a90 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 --2- 172.21.15.66:0/4279943032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc1010a0 0x7f21fc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 -- 172.21.15.66:0/4279943032 >> 172.21.15.66:0/4279943032 conn(0x7f21fc077eb0 msgr2=0x7f21fc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 -- 172.21.15.66:0/4279943032 shutdown_connections 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 -- 172.21.15.66:0/4279943032 wait complete. 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.719+0000 7f2201b15640 1 Processor -- start 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f2201b15640 1 -- start start 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f2201b15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc1010a0 0x7f21fc102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f2201b15640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21fc106170 0x7f21fc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f2201b15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc1035b0 0x7f21fc1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f2201b15640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1110b0 con 0x7f21fc1035b0 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f2201b15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc110f30 con 0x7f21fc1010a0 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f2201b15640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc111230 con 0x7f21fc106170 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc1035b0 0x7f21fc1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc1035b0 0x7f21fc1a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38478/0 (socket says 172.21.15.66:38478) 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 -- 172.21.15.66:0/2870471761 learned_addr learned my addr 172.21.15.66:0/2870471761 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21faffd640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21fc106170 0x7f21fc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fb7fe640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc1010a0 0x7f21fc102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 -- 172.21.15.66:0/2870471761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21fc106170 msgr2=0x7f21fc103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21fc106170 0x7f21fc103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 -- 172.21.15.66:0/2870471761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc1010a0 msgr2=0x7f21fc102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc1010a0 0x7f21fc102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 -- 172.21.15.66:0/2870471761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc1a7010 con 0x7f21fc1035b0 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.720+0000 7f21fbfff640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc1035b0 0x7f21fc1a6970 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f21f400eb10 tx=0x7f21f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:10.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.721+0000 7f21f8ff9640 1 -- 172.21.15.66:0/2870471761 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4004b20 con 0x7f21fc1035b0 2024-04-21T09:19:10.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.721+0000 7f21f8ff9640 1 -- 172.21.15.66:0/2870471761 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f4004cc0 con 0x7f21fc1035b0 2024-04-21T09:19:10.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.721+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc1a7300 con 0x7f21fc1035b0 2024-04-21T09:19:10.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.721+0000 7f21f8ff9640 1 -- 172.21.15.66:0/2870471761 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4010670 con 0x7f21fc1035b0 2024-04-21T09:19:10.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.721+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1a7870 con 0x7f21fc1035b0 2024-04-21T09:19:10.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.722+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc0691c0 con 0x7f21fc1035b0 2024-04-21T09:19:10.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.723+0000 7f21f8ff9640 1 -- 172.21.15.66:0/2870471761 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f21f4005020 con 0x7f21fc1035b0 2024-04-21T09:19:10.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.723+0000 7f21f8ff9640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f21e40a3da0 0x7f21e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:10.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.723+0000 7f21fb7fe640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f21e40a3da0 0x7f21e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:10.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.723+0000 7f21f8ff9640 1 -- 172.21.15.66:0/2870471761 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f21f40d10b0 con 0x7f21fc1035b0 2024-04-21T09:19:10.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.724+0000 7f21fb7fe640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f21e40a3da0 0x7f21e40a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005fd0 tx=0x7f21ec005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:10.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.728+0000 7f21f8ff9640 1 -- 172.21.15.66:0/2870471761 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21f409d380 con 0x7f21fc1035b0 2024-04-21T09:19:10.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:10 smithi066 ceph-mon[27953]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T09:19:10.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3235582129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:19:10.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:10 smithi066 ceph-mon[27953]: pgmap v342: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:19:10.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/45846500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:19:10.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:10 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/4098599929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:19:10.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.915+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f21fc101a50 con 0x7f21fc1035b0 2024-04-21T09:19:10.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.915+0000 7f21f8ff9640 1 -- 172.21.15.66:0/2870471761 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f21f409cc80 con 0x7f21fc1035b0 2024-04-21T09:19:10.916 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-04-21T09:19:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f21e40a3da0 msgr2=0x7f21e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f21e40a3da0 0x7f21e40a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005fd0 tx=0x7f21ec005850 comp rx=0 tx=0).stop 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc1035b0 msgr2=0x7f21fc1a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc1035b0 0x7f21fc1a6970 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f21f400eb10 tx=0x7f21f400c2d0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 shutdown_connections 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f21e40a3da0 0x7f21e40a6260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21fc1035b0 0x7f21fc1a6970 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21fc106170 0x7f21fc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 --2- 172.21.15.66:0/2870471761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc1010a0 0x7f21fc102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 >> 172.21.15.66:0/2870471761 conn(0x7f21fc077eb0 msgr2=0x7f21fc06a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 shutdown_connections 2024-04-21T09:19:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:10.917+0000 7f2201b15640 1 -- 172.21.15.66:0/2870471761 wait complete. 2024-04-21T09:19:10.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T09:19:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 -- 172.21.15.66:0/637005513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b010c570 msgr2=0x7f68b010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 --2- 172.21.15.66:0/637005513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b010c570 0x7f68b010e9c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ca0 tx=0x7f68a802c180 comp rx=0 tx=0).stop 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 -- 172.21.15.66:0/637005513 shutdown_connections 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 --2- 172.21.15.66:0/637005513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b010c570 0x7f68b010e9c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 --2- 172.21.15.66:0/637005513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b0103970 0x7f68b010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 --2- 172.21.15.66:0/637005513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b0103050 0x7f68b0103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 -- 172.21.15.66:0/637005513 >> 172.21.15.66:0/637005513 conn(0x7f68b0077fb0 msgr2=0x7f68b00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 -- 172.21.15.66:0/637005513 shutdown_connections 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.033+0000 7f68b7552640 1 -- 172.21.15.66:0/637005513 wait complete. 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 Processor -- start 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 -- start start 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b0103050 0x7f68b0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b0103970 0x7f68b0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b010c570 0x7f68b019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01121a0 con 0x7f68b0103050 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0112020 con 0x7f68b010c570 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b7552640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0112320 con 0x7f68b0103970 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b0103050 0x7f68b0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b0103050 0x7f68b0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38508/0 (socket says 172.21.15.66:38508) 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 -- 172.21.15.66:0/1699749673 learned_addr learned my addr 172.21.15.66:0/1699749673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b5aef640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b010c570 0x7f68b019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 -- 172.21.15.66:0/1699749673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b0103970 msgr2=0x7f68b0198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b0103970 0x7f68b0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 -- 172.21.15.66:0/1699749673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b010c570 msgr2=0x7f68b019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b010c570 0x7f68b019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 -- 172.21.15.66:0/1699749673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b019d730 con 0x7f68b0103050 2024-04-21T09:19:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.034+0000 7f68b52ee640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b0103050 0x7f68b0198740 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d950 tx=0x7f68ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.035+0000 7f68a67fc640 1 -- 172.21.15.66:0/1699749673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac017030 con 0x7f68b0103050 2024-04-21T09:19:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.035+0000 7f68a67fc640 1 -- 172.21.15.66:0/1699749673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68ac004020 con 0x7f68b0103050 2024-04-21T09:19:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.035+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b019da20 con 0x7f68b0103050 2024-04-21T09:19:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.035+0000 7f68a67fc640 1 -- 172.21.15.66:0/1699749673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0047a0 con 0x7f68b0103050 2024-04-21T09:19:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.035+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b01a52b0 con 0x7f68b0103050 2024-04-21T09:19:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.036+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b00684e0 con 0x7f68b0103050 2024-04-21T09:19:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.037+0000 7f68a67fc640 1 -- 172.21.15.66:0/1699749673 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f68ac009a30 con 0x7f68b0103050 2024-04-21T09:19:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.037+0000 7f68a67fc640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f689c0a3da0 0x7f689c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.038+0000 7f68b4aed640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f689c0a3da0 0x7f689c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.038+0000 7f68a67fc640 1 -- 172.21.15.66:0/1699749673 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f68ac0d0630 con 0x7f68b0103050 2024-04-21T09:19:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.038+0000 7f68b4aed640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f689c0a3da0 0x7f689c0a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68b010e290 tx=0x7f68a0009590 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.043+0000 7f68a67fc640 1 -- 172.21.15.66:0/1699749673 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68ac09c900 con 0x7f68b0103050 2024-04-21T09:19:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:10 smithi186 ceph-mon[31505]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T09:19:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3235582129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:19:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:10 smithi186 ceph-mon[31505]: pgmap v342: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:19:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/45846500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:19:11.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:10 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/4098599929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:19:11.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:10 smithi083 ceph-mon[31413]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T09:19:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3235582129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:19:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:10 smithi083 ceph-mon[31413]: pgmap v342: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-21T09:19:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/45846500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T09:19:11.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:10 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/4098599929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T09:19:11.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.232+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68b0107750 con 0x7f68b0103050 2024-04-21T09:19:11.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.232+0000 7f68a67fc640 1 -- 172.21.15.66:0/1699749673 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v106) ==== 103+0+11 (secure 0 0 0) 0x7f68ac09c200 con 0x7f68b0103050 2024-04-21T09:19:11.233 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-04-21T09:19:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f689c0a3da0 msgr2=0x7f689c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f689c0a3da0 0x7f689c0a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68b010e290 tx=0x7f68a0009590 comp rx=0 tx=0).stop 2024-04-21T09:19:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b0103050 msgr2=0x7f68b0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b0103050 0x7f68b0198740 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d950 tx=0x7f68ac00de10 comp rx=0 tx=0).stop 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 shutdown_connections 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f689c0a3da0 0x7f689c0a6260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b010c570 0x7f68b019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b0103970 0x7f68b0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 --2- 172.21.15.66:0/1699749673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68b0103050 0x7f68b0198740 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.234+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 >> 172.21.15.66:0/1699749673 conn(0x7f68b0077fb0 msgr2=0x7f68b00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.235+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 shutdown_connections 2024-04-21T09:19:11.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.235+0000 7f68b7552640 1 -- 172.21.15.66:0/1699749673 wait complete. 2024-04-21T09:19:11.246 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T09:19:11.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 -- 172.21.15.66:0/2171519289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 msgr2=0x7f59501035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 --2- 172.21.15.66:0/2171519289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f59501035b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5948009ca0 tx=0x7f594802c230 comp rx=0 tx=0).stop 2024-04-21T09:19:11.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 -- 172.21.15.66:0/2171519289 shutdown_connections 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 --2- 172.21.15.66:0/2171519289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595010c510 0x7f595010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 --2- 172.21.15.66:0/2171519289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5950103af0 0x7f595010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 --2- 172.21.15.66:0/2171519289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f59501035b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 -- 172.21.15.66:0/2171519289 >> 172.21.15.66:0/2171519289 conn(0x7f5950077f50 msgr2=0x7f5950078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 -- 172.21.15.66:0/2171519289 shutdown_connections 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 -- 172.21.15.66:0/2171519289 wait complete. 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 Processor -- start 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 -- start start 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f5950198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5950103af0 0x7f5950198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595010c510 0x7f595019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.349+0000 7f5957414640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59501121a0 con 0x7f59501031d0 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f5957414640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5950112020 con 0x7f595010c510 2024-04-21T09:19:11.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f5957414640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5950112320 con 0x7f5950103af0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f5950198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f5950198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38540/0 (socket says 172.21.15.66:38540) 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 -- 172.21.15.66:0/1843394054 learned_addr learned my addr 172.21.15.66:0/1843394054 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59559b1640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595010c510 0x7f595019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59549af640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5950103af0 0x7f5950198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 -- 172.21.15.66:0/1843394054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5950103af0 msgr2=0x7f5950198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5950103af0 0x7f5950198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 -- 172.21.15.66:0/1843394054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595010c510 msgr2=0x7f595019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595010c510 0x7f595019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 -- 172.21.15.66:0/1843394054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595019d730 con 0x7f59501031d0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f59551b0640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f5950198740 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5948005e30 tx=0x7f5948035ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f594e7fc640 1 -- 172.21.15.66:0/1843394054 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948039040 con 0x7f59501031d0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f594e7fc640 1 -- 172.21.15.66:0/1843394054 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5948044070 con 0x7f59501031d0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595019d9c0 con 0x7f59501031d0 2024-04-21T09:19:11.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f594e7fc640 1 -- 172.21.15.66:0/1843394054 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948032880 con 0x7f59501031d0 2024-04-21T09:19:11.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.350+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59501a52b0 con 0x7f59501031d0 2024-04-21T09:19:11.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.351+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5950068d70 con 0x7f59501031d0 2024-04-21T09:19:11.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.353+0000 7f594e7fc640 1 -- 172.21.15.66:0/1843394054 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5948032ae0 con 0x7f59501031d0 2024-04-21T09:19:11.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.353+0000 7f594e7fc640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59380a3da0 0x7f59380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:11.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.354+0000 7f59549af640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59380a3da0 0x7f59380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:11.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.354+0000 7f594e7fc640 1 -- 172.21.15.66:0/1843394054 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f59480f3aa0 con 0x7f59501031d0 2024-04-21T09:19:11.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.354+0000 7f59549af640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59380a3da0 0x7f59380a6260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5944005fd0 tx=0x7f5944005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:11.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.357+0000 7f594e7fc640 1 -- 172.21.15.66:0/1843394054 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59480bfe80 con 0x7f59501031d0 2024-04-21T09:19:11.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.557+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5950107690 con 0x7f59501031d0 2024-04-21T09:19:11.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.557+0000 7f594e7fc640 1 -- 172.21.15.66:0/1843394054 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v106) ==== 81+0+940 (secure 0 0 0) 0x7f59480bf780 con 0x7f59501031d0 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3032531159 2024-04-22T09:08:12.028364+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3032531159 2024-04-22T09:08:12.028364+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/838597709 2024-04-22T09:08:12.028364+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1898303764 2024-04-22T09:08:12.028364+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/612283217 2024-04-22T08:58:38.211749+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3217056690 2024-04-22T09:08:12.028364+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/763565357 2024-04-22T08:58:38.211749+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3624224927 2024-04-22T08:58:12.038096+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2187684654 2024-04-22T08:58:12.038096+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3374606173 2024-04-22T08:58:38.211749+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/763565357 2024-04-22T08:58:38.211749+0000 2024-04-21T09:19:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/502604018 2024-04-22T08:58:12.038096+0000 2024-04-21T09:19:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3624224927 2024-04-22T08:58:12.038096+0000 2024-04-21T09:19:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2365699589 2024-04-22T08:58:38.211749+0000 2024-04-21T09:19:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/389506800 2024-04-22T09:08:12.028364+0000 2024-04-21T09:19:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3868723687 2024-04-22T08:58:12.038096+0000 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:listed 16 entries 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59380a3da0 msgr2=0x7f59380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59380a3da0 0x7f59380a6260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5944005fd0 tx=0x7f5944005850 comp rx=0 tx=0).stop 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 msgr2=0x7f5950198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f5950198740 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5948005e30 tx=0x7f5948035ea0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 shutdown_connections 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f59380a3da0 0x7f59380a6260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595010c510 0x7f595019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5950103af0 0x7f5950198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 --2- 172.21.15.66:0/1843394054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59501031d0 0x7f5950198740 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 >> 172.21.15.66:0/1843394054 conn(0x7f5950077f50 msgr2=0x7f59500fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 shutdown_connections 2024-04-21T09:19:11.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:11.559+0000 7f5957414640 1 -- 172.21.15.66:0/1843394054 wait complete. 2024-04-21T09:19:11.570 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-21T09:19:11.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:19:11.571 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-21T09:19:11.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:19:11.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:19:11.623 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-21T09:19:11.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:19:11.646 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-21T09:19:11.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:19:11.704 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T09:19:11.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:19:11.770 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-04-21T09:19:11.831 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2870471761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:19:11.831 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1699749673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:19:11.832 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:11 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1843394054' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:19:11.878 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T09:19:11.878 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-21T09:19:11.878 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T09:19:11.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056090,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19917 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/6e6e8460c85900d354f99492666cf45a112a4c5e5d80ed4505d0d695213e4ebf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/6e6e8460c85900d354f99492666cf45a112a4c5e5d80ed4505d0d695213e4ebf/diff,workdir=/var/lib/containers/storage/overlay/6e6e8460c85900d354f99492666cf45a112a4c5e5d80ed4505d0d695213e4ebf/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/28c2811ee65339a5f6133fcaf26e0478427064b9e5134460eb14662089134e0a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/28c2811ee65339a5f6133fcaf26e0478427064b9e5134460eb14662089134e0a/diff,workdir=/var/lib/containers/storage/overlay/28c2811ee65339a5f6133fcaf26e0478427064b9e5134460eb14662089134e0a/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/ebacaecddd897b0717be8c5e994f713a59d4b59b4b7cbec947de8056306d4257/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/ebacaecddd897b0717be8c5e994f713a59d4b59b4b7cbec947de8056306d4257/diff,workdir=/var/lib/containers/storage/overlay/ebacaecddd897b0717be8c5e994f713a59d4b59b4b7cbec947de8056306d4257/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2a64c64163cddf5406e2227fa5928cb0de234046eb8cf1bdd2bd03ab1dbd4c0d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/2a64c64163cddf5406e2227fa5928cb0de234046eb8cf1bdd2bd03ab1dbd4c0d/diff,workdir=/var/lib/containers/storage/overlay/2a64c64163cddf5406e2227fa5928cb0de234046eb8cf1bdd2bd03ab1dbd4c0d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/87782a91e1792af1c9f3f137312fc27f25b7f0a2dcd2e14a0064fb2a445c47d2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/87782a91e1792af1c9f3f137312fc27f25b7f0a2dcd2e14a0064fb2a445c47d2/diff,workdir=/var/lib/containers/storage/overlay/87782a91e1792af1c9f3f137312fc27f25b7f0a2dcd2e14a0064fb2a445c47d2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/50a23de37d368ae1a88b0b10fe7d274914c26f92e162b6ee414be1b1d59b2e1e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/50a23de37d368ae1a88b0b10fe7d274914c26f92e162b6ee414be1b1d59b2e1e/diff,workdir=/var/lib/containers/storage/overlay/50a23de37d368ae1a88b0b10fe7d274914c26f92e162b6ee414be1b1d59b2e1e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e20f8fb5db133b7fd76f44f49c63a760ed8e0091b627ba44d420e609eb3c7e15/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/e20f8fb5db133b7fd76f44f49c63a760ed8e0091b627ba44d420e609eb3c7e15/diff,workdir=/var/lib/containers/storage/overlay/e20f8fb5db133b7fd76f44f49c63a760ed8e0091b627ba44d420e609eb3c7e15/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8c6fbdd2a573cc394556328d9a55b214b5ee0521d1a07bb71891179e6c2c42cc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/8c6fbdd2a573cc394556328d9a55b214b5ee0521d1a07bb71891179e6c2c42cc/diff,workdir=/var/lib/containers/storage/overlay/8c6fbdd2a573cc394556328d9a55b214b5ee0521d1a07bb71891179e6c2c42cc/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b677b9811d912a62a75447145c4c165f37ad64285d7c9a01b156f4eb128b4fcf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/b677b9811d912a62a75447145c4c165f37ad64285d7c9a01b156f4eb128b4fcf/diff,workdir=/var/lib/containers/storage/overlay/b677b9811d912a62a75447145c4c165f37ad64285d7c9a01b156f4eb128b4fcf/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bacf92df245ca9e132a8fde2aca28cef4652195e21fac7b75aff553fce0adadd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J55FZZ3JS4XP3CPNKSYO57QCAQ:/var/lib/containers/storage/overlay/l/YD5KCC72UY7773Z673TEQMLLCY,upperdir=/var/lib/containers/storage/overlay/bacf92df245ca9e132a8fde2aca28cef4652195e21fac7b75aff553fce0adadd/diff,workdir=/var/lib/containers/storage/overlay/bacf92df245ca9e132a8fde2aca28cef4652195e21fac7b75aff553fce0adadd/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-21T09:19:11.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:19:11.935 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-21T09:19:11.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T09:19:11.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-04-21T09:19:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-04-21T09:19:12.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:19:12.059 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-21T09:19:12.071 INFO:tasks.cephadm:Teardown begin 2024-04-21T09:19:12.072 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-21T09:19:12.072 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:19:12.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2870471761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:19:12.087 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1699749673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:19:12.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:11 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1843394054' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:19:12.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2870471761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T09:19:12.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1699749673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T09:19:12.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:11 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1843394054' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T09:19:12.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.203+0000 7f0e41a4e640 1 -- 172.21.15.66:0/1984250127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c1011a0 msgr2=0x7f0e3c101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.203+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/1984250127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c1011a0 0x7f0e3c101620 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009930 tx=0x7f0e3002bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:12.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 -- 172.21.15.66:0/1984250127 shutdown_connections 2024-04-21T09:19:12.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/1984250127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c101b60 0x7f0e3c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/1984250127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c1011a0 0x7f0e3c101620 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/1984250127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3c105e30 0x7f0e3c106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 -- 172.21.15.66:0/1984250127 >> 172.21.15.66:0/1984250127 conn(0x7f0e3c077e30 msgr2=0x7f0e3c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 -- 172.21.15.66:0/1984250127 shutdown_connections 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 -- 172.21.15.66:0/1984250127 wait complete. 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 Processor -- start 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.204+0000 7f0e41a4e640 1 -- start start 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e41a4e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c1011a0 0x7f0e3c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e41a4e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c101b60 0x7f0e3c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e41a4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3c105e30 0x7f0e3c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e41a4e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c112240 con 0x7f0e3c101b60 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e41a4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1120c0 con 0x7f0e3c105e30 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e41a4e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c1123c0 con 0x7f0e3c1011a0 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c101b60 0x7f0e3c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c101b60 0x7f0e3c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38552/0 (socket says 172.21.15.66:38552) 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 -- 172.21.15.66:0/2134037543 learned_addr learned my addr 172.21.15.66:0/2134037543 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3b7fe640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3c105e30 0x7f0e3c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3affd640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c1011a0 0x7f0e3c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 -- 172.21.15.66:0/2134037543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c1011a0 msgr2=0x7f0e3c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c1011a0 0x7f0e3c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 -- 172.21.15.66:0/2134037543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3c105e30 msgr2=0x7f0e3c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3c105e30 0x7f0e3c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 -- 172.21.15.66:0/2134037543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c19b020 con 0x7f0e3c101b60 2024-04-21T09:19:12.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e3a7fc640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c101b60 0x7f0e3c1a0be0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009230 tx=0x7f0e3002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:12.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.205+0000 7f0e40a4c640 1 -- 172.21.15.66:0/2134037543 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30043020 con 0x7f0e3c101b60 2024-04-21T09:19:12.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.206+0000 7f0e40a4c640 1 -- 172.21.15.66:0/2134037543 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e300042d0 con 0x7f0e3c101b60 2024-04-21T09:19:12.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.206+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c19b2b0 con 0x7f0e3c101b60 2024-04-21T09:19:12.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.206+0000 7f0e40a4c640 1 -- 172.21.15.66:0/2134037543 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30003a80 con 0x7f0e3c101b60 2024-04-21T09:19:12.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.206+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c1a7620 con 0x7f0e3c101b60 2024-04-21T09:19:12.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.206+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c19b620 con 0x7f0e3c101b60 2024-04-21T09:19:12.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.208+0000 7f0e40a4c640 1 -- 172.21.15.66:0/2134037543 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0e30046020 con 0x7f0e3c101b60 2024-04-21T09:19:12.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.208+0000 7f0e40a4c640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0e240a3d20 0x7f0e240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.208+0000 7f0e3affd640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0e240a3d20 0x7f0e240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.209+0000 7f0e40a4c640 1 -- 172.21.15.66:0/2134037543 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f0e300f2ed0 con 0x7f0e3c101b60 2024-04-21T09:19:12.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.209+0000 7f0e3affd640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0e240a3d20 0x7f0e240a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009bd0 tx=0x7f0e34005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:12.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.212+0000 7f0e40a4c640 1 -- 172.21.15.66:0/2134037543 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e3004d020 con 0x7f0e3c101b60 2024-04-21T09:19:12.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.441+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e3c102570 con 0x7f0e3c101b60 2024-04-21T09:19:12.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.441+0000 7f0e40a4c640 1 -- 172.21.15.66:0/2134037543 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f0e300bf2b0 con 0x7f0e3c101b60 2024-04-21T09:19:12.442 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:19:12.442 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:19:12.442 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.443+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0e240a3d20 msgr2=0x7f0e240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.443+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0e240a3d20 0x7f0e240a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009bd0 tx=0x7f0e34005f90 comp rx=0 tx=0).stop 2024-04-21T09:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c101b60 msgr2=0x7f0e3c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c101b60 0x7f0e3c1a0be0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009230 tx=0x7f0e3002c5d0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 shutdown_connections 2024-04-21T09:19:12.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0e240a3d20 0x7f0e240a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3c105e30 0x7f0e3c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e3c101b60 0x7f0e3c1a0be0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 --2- 172.21.15.66:0/2134037543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c1011a0 0x7f0e3c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 >> 172.21.15.66:0/2134037543 conn(0x7f0e3c077e30 msgr2=0x7f0e3c10c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:12.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 shutdown_connections 2024-04-21T09:19:12.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.444+0000 7f0e41a4e640 1 -- 172.21.15.66:0/2134037543 wait complete. 2024-04-21T09:19:12.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.557+0000 7f267f431640 1 -- 172.21.15.66:0/3219816419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2678101a60 msgr2=0x7f2678105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.557+0000 7f267f431640 1 --2- 172.21.15.66:0/3219816419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2678101a60 0x7f2678105aa0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f2668009900 tx=0x7f266802bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.557+0000 7f267f431640 1 -- 172.21.15.66:0/3219816419 shutdown_connections 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.557+0000 7f267f431640 1 --2- 172.21.15.66:0/3219816419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678106180 0x7f267810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.557+0000 7f267f431640 1 --2- 172.21.15.66:0/3219816419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2678101a60 0x7f2678105aa0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.557+0000 7f267f431640 1 --2- 172.21.15.66:0/3219816419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26781010b0 0x7f2678101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.557+0000 7f267f431640 1 -- 172.21.15.66:0/3219816419 >> 172.21.15.66:0/3219816419 conn(0x7f26780771d0 msgr2=0x7f26780775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 -- 172.21.15.66:0/3219816419 shutdown_connections 2024-04-21T09:19:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 -- 172.21.15.66:0/3219816419 wait complete. 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 Processor -- start 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 -- start start 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26781010b0 0x7f267810e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678106180 0x7f267810ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f267810f160 0x7f26781a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26781121c0 con 0x7f267810f160 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2678112040 con 0x7f26781010b0 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267f431640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2678112340 con 0x7f2678106180 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.558+0000 7f267d9ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f267810f160 0x7f26781a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f267810f160 0x7f26781a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38570/0 (socket says 172.21.15.66:38570) 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 -- 172.21.15.66:0/758648577 learned_addr learned my addr 172.21.15.66:0/758648577 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d1cd640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26781010b0 0x7f267810e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267c9cc640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678106180 0x7f267810ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 -- 172.21.15.66:0/758648577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678106180 msgr2=0x7f267810ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678106180 0x7f267810ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 -- 172.21.15.66:0/758648577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26781010b0 msgr2=0x7f267810e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26781010b0 0x7f267810e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 -- 172.21.15.66:0/758648577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26781a6fd0 con 0x7f267810f160 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267d9ce640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f267810f160 0x7f26781a68d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f267000cad0 tx=0x7f267000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f266e7fc640 1 -- 172.21.15.66:0/758648577 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267000ae20 con 0x7f267810f160 2024-04-21T09:19:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26781a72c0 con 0x7f267810f160 2024-04-21T09:19:12.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f266e7fc640 1 -- 172.21.15.66:0/758648577 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2670005020 con 0x7f267810f160 2024-04-21T09:19:12.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f266e7fc640 1 -- 172.21.15.66:0/758648577 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267000f6b0 con 0x7f267810f160 2024-04-21T09:19:12.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.559+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26781a7790 con 0x7f267810f160 2024-04-21T09:19:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.560+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2678069230 con 0x7f267810f160 2024-04-21T09:19:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.562+0000 7f266e7fc640 1 -- 172.21.15.66:0/758648577 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f26700051d0 con 0x7f267810f160 2024-04-21T09:19:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.562+0000 7f266e7fc640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26640a3d50 0x7f26640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.562+0000 7f267d1cd640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26640a3d50 0x7f26640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.562+0000 7f266e7fc640 1 -- 172.21.15.66:0/758648577 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f26700d0f30 con 0x7f267810f160 2024-04-21T09:19:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.563+0000 7f267d1cd640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26640a3d50 0x7f26640a6210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26740058c0 tx=0x7f267400a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:12.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.566+0000 7f266e7fc640 1 -- 172.21.15.66:0/758648577 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2670014030 con 0x7f267810f160 2024-04-21T09:19:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.790+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2678101a60 con 0x7f267810f160 2024-04-21T09:19:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.791+0000 7f266e7fc640 1 -- 172.21.15.66:0/758648577 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11867 (secure 0 0 0) 0x7f267009d200 con 0x7f267810f160 2024-04-21T09:19:12.792 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:19:12.792 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3512926718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3512926718},{"type":"v1","addr":"172.21.15.66:6835","nonce":3512926718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2533275448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2533275448},{"type":"v1","addr":"172.21.15.66:6837","nonce":2533275448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2337695414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2337695414},{"type":"v1","addr":"172.21.15.66:6839","nonce":2337695414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874916254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874916254},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874916254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/446664633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":446664633},{"type":"v1","addr":"172.21.15.186:6833","nonce":446664633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/325280073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":325280073},{"type":"v1","addr":"172.21.15.83:6833","nonce":325280073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T09:05:46.330734+0000","modified":"2024-04-21T09:08:10.160804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24490,"mds_2":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/2851815427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2851815427},{"type":"v1","addr":"172.21.15.83:6835","nonce":2851815427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1405629184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1405629184},{"type":"v1","addr":"172.21.15.186:6835","nonce":1405629184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6837/1780100152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1780100152},{"type":"v1","addr":"172.21.15.186:6837","nonce":1780100152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/1568837633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1568837633},{"type":"v1","addr":"172.21.15.83:6837","nonce":1568837633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/599772329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":599772329},{"type":"v1","addr":"172.21.15.186:6839","nonce":599772329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/4140026829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4140026829},{"type":"v1","addr":"172.21.15.83:6839","nonce":4140026829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24490,24476]},"id":1}]} 2024-04-21T09:19:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-04-21T09:19:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26640a3d50 msgr2=0x7f26640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26640a3d50 0x7f26640a6210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26740058c0 tx=0x7f267400a2d0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f267810f160 msgr2=0x7f26781a68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f267810f160 0x7f26781a68d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f267000cad0 tx=0x7f267000cf90 comp rx=0 tx=0).stop 2024-04-21T09:19:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 shutdown_connections 2024-04-21T09:19:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f26640a3d50 0x7f26640a6210 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f267810f160 0x7f26781a68d0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678106180 0x7f267810ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 --2- 172.21.15.66:0/758648577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26781010b0 0x7f267810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 >> 172.21.15.66:0/758648577 conn(0x7f26780771d0 msgr2=0x7f26780fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 shutdown_connections 2024-04-21T09:19:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.793+0000 7f267f431640 1 -- 172.21.15.66:0/758648577 wait complete. 2024-04-21T09:19:12.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T09:19:12.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 -- 172.21.15.66:0/2972036419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934101a60 msgr2=0x7f8934105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 --2- 172.21.15.66:0/2972036419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934101a60 0x7f8934105aa0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f8928009900 tx=0x7f892802bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 -- 172.21.15.66:0/2972036419 shutdown_connections 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 --2- 172.21.15.66:0/2972036419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8934106180 0x7f893410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 --2- 172.21.15.66:0/2972036419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934101a60 0x7f8934105aa0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 --2- 172.21.15.66:0/2972036419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89341010b0 0x7f8934101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 -- 172.21.15.66:0/2972036419 >> 172.21.15.66:0/2972036419 conn(0x7f89340771d0 msgr2=0x7f89340775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 -- 172.21.15.66:0/2972036419 shutdown_connections 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.908+0000 7f8939a81640 1 -- 172.21.15.66:0/2972036419 wait complete. 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 Processor -- start 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 -- start start 2024-04-21T09:19:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89341010b0 0x7f893419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934106180 0x7f893419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8934198590 0x7f8934198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89341110a0 con 0x7f8934106180 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8934110f20 con 0x7f8934198590 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8939a81640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8934111220 con 0x7f89341010b0 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8932ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934106180 0x7f893419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8932ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934106180 0x7f893419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38586/0 (socket says 172.21.15.66:38586) 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8932ffd640 1 -- 172.21.15.66:0/3478468118 learned_addr learned my addr 172.21.15.66:0/3478468118 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.909+0000 7f8933fff640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8934198590 0x7f8934198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f89337fe640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89341010b0 0x7f893419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8932ffd640 1 -- 172.21.15.66:0/3478468118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89341010b0 msgr2=0x7f893419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8932ffd640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89341010b0 0x7f893419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8932ffd640 1 -- 172.21.15.66:0/3478468118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8934198590 msgr2=0x7f8934198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8932ffd640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8934198590 0x7f8934198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8932ffd640 1 -- 172.21.15.66:0/3478468118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89341990b0 con 0x7f8934106180 2024-04-21T09:19:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8933fff640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8934198590 0x7f8934198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T09:19:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8932ffd640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934106180 0x7f893419e920 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8928036040 tx=0x7f892802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8930ff9640 1 -- 172.21.15.66:0/3478468118 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928043020 con 0x7f8934106180 2024-04-21T09:19:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8930ff9640 1 -- 172.21.15.66:0/3478468118 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892802cbe0 con 0x7f8934106180 2024-04-21T09:19:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8934199340 con 0x7f8934106180 2024-04-21T09:19:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8930ff9640 1 -- 172.21.15.66:0/3478468118 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928003bf0 con 0x7f8934106180 2024-04-21T09:19:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.910+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89341a5590 con 0x7f8934106180 2024-04-21T09:19:12.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.911+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8934069230 con 0x7f8934106180 2024-04-21T09:19:12.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.912+0000 7f8930ff9640 1 -- 172.21.15.66:0/3478468118 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8928046020 con 0x7f8934106180 2024-04-21T09:19:12.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.913+0000 7f8930ff9640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f891c0a3d20 0x7f891c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:12.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.913+0000 7f89337fe640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f891c0a3d20 0x7f891c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:12.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.913+0000 7f8930ff9640 1 -- 172.21.15.66:0/3478468118 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f89280f2dc0 con 0x7f8934106180 2024-04-21T09:19:12.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.913+0000 7f89337fe640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f891c0a3d20 0x7f891c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89240058c0 tx=0x7f8924005c60 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:12.917+0000 7f8930ff9640 1 -- 172.21.15.66:0/3478468118 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f892804d020 con 0x7f8934106180 2024-04-21T09:19:13.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:12 smithi186 ceph-mon[31505]: pgmap v343: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:19:13.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2134037543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:19:13.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:12 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/758648577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:19:13.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.104+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8934101ac0 con 0x7f8934106180 2024-04-21T09:19:13.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.105+0000 7f8930ff9640 1 -- 172.21.15.66:0/3478468118 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+25272 (secure 0 0 0) 0x7f89280bf1a0 con 0x7f8934106180 2024-04-21T09:19:13.106 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-21T09:19:13.106 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":106,"fsid":"0cca84f6-ffbd-11ee-bc93-c7b262605968","created":"2024-04-21T08:57:34.912480+0000","modified":"2024-04-21T09:19:09.808716+0000","last_up_change":"2024-04-21T09:04:56.402560+0000","last_in_change":"2024-04-21T09:04:36.594979+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T09:01:06.302362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T09:05:44.770752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T09:05:45.280115+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T09:05:48.992312+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5f4d872-e0ff-41aa-bd69-ac73da467920","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6803","nonce":1993480786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6805","nonce":1993480786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6809","nonce":1993480786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1993480786},{"type":"v1","addr":"172.21.15.66:6807","nonce":1993480786}]},"public_addr":"172.21.15.66:6803/1993480786","cluster_addr":"172.21.15.66:6805/1993480786","heartbeat_back_addr":"172.21.15.66:6809/1993480786","heartbeat_front_addr":"172.21.15.66:6807/1993480786","state":["exists","up"]},{"osd":1,"uuid":"49f582ca-69a5-425c-a239-3f46fb57c283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6801","nonce":2284873950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6803","nonce":2284873950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6807","nonce":2284873950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2284873950},{"type":"v1","addr":"172.21.15.83:6805","nonce":2284873950}]},"public_addr":"172.21.15.83:6801/2284873950","cluster_addr":"172.21.15.83:6803/2284873950","heartbeat_back_addr":"172.21.15.83:6807/2284873950","heartbeat_front_addr":"172.21.15.83:6805/2284873950","state":["exists","up"]},{"osd":2,"uuid":"c0462467-e4ef-4fc1-8f53-5ce7742ff9af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6801","nonce":993640486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6803","nonce":993640486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6807","nonce":993640486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":993640486},{"type":"v1","addr":"172.21.15.186:6805","nonce":993640486}]},"public_addr":"172.21.15.186:6801/993640486","cluster_addr":"172.21.15.186:6803/993640486","heartbeat_back_addr":"172.21.15.186:6807/993640486","heartbeat_front_addr":"172.21.15.186:6805/993640486","state":["exists","up"]},{"osd":3,"uuid":"e6c91e9b-d431-49b2-8b56-562e85e16010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6811","nonce":4132171023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6813","nonce":4132171023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6817","nonce":4132171023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4132171023},{"type":"v1","addr":"172.21.15.66:6815","nonce":4132171023}]},"public_addr":"172.21.15.66:6811/4132171023","cluster_addr":"172.21.15.66:6813/4132171023","heartbeat_back_addr":"172.21.15.66:6817/4132171023","heartbeat_front_addr":"172.21.15.66:6815/4132171023","state":["exists","up"]},{"osd":4,"uuid":"0cdc8c7f-b8e8-44ce-9bd2-c5de5efb7c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6809","nonce":2820280622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6811","nonce":2820280622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6815","nonce":2820280622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2820280622},{"type":"v1","addr":"172.21.15.83:6813","nonce":2820280622}]},"public_addr":"172.21.15.83:6809/2820280622","cluster_addr":"172.21.15.83:6811/2820280622","heartbeat_back_addr":"172.21.15.83:6815/2820280622","heartbeat_front_addr":"172.21.15.83:6813/2820280622","state":["exists","up"]},{"osd":5,"uuid":"c32caead-4fab-4b5e-9bc5-7d1bb3877747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6809","nonce":3723805891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6811","nonce":3723805891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6815","nonce":3723805891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3723805891},{"type":"v1","addr":"172.21.15.186:6813","nonce":3723805891}]},"public_addr":"172.21.15.186:6809/3723805891","cluster_addr":"172.21.15.186:6811/3723805891","heartbeat_back_addr":"172.21.15.186:6815/3723805891","heartbeat_front_addr":"172.21.15.186:6813/3723805891","state":["exists","up"]},{"osd":6,"uuid":"1ffb4890-5238-4ba0-9197-82b350fbdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6819","nonce":455614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6821","nonce":455614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6825","nonce":455614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":455614173},{"type":"v1","addr":"172.21.15.66:6823","nonce":455614173}]},"public_addr":"172.21.15.66:6819/455614173","cluster_addr":"172.21.15.66:6821/455614173","heartbeat_back_addr":"172.21.15.66:6825/455614173","heartbeat_front_addr":"172.21.15.66:6823/455614173","state":["exists","up"]},{"osd":7,"uuid":"8a6d8f44-b895-4c32-a527-c129c044aa7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6817","nonce":1973931443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6819","nonce":1973931443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6823","nonce":1973931443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1973931443},{"type":"v1","addr":"172.21.15.83:6821","nonce":1973931443}]},"public_addr":"172.21.15.83:6817/1973931443","cluster_addr":"172.21.15.83:6819/1973931443","heartbeat_back_addr":"172.21.15.83:6823/1973931443","heartbeat_front_addr":"172.21.15.83:6821/1973931443","state":["exists","up"]},{"osd":8,"uuid":"b2c7530c-00b0-49d6-9157-9c3c86f9b4a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6817","nonce":2023460936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6819","nonce":2023460936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6823","nonce":2023460936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2023460936},{"type":"v1","addr":"172.21.15.186:6821","nonce":2023460936}]},"public_addr":"172.21.15.186:6817/2023460936","cluster_addr":"172.21.15.186:6819/2023460936","heartbeat_back_addr":"172.21.15.186:6823/2023460936","heartbeat_front_addr":"172.21.15.186:6821/2023460936","state":["exists","up"]},{"osd":9,"uuid":"7b58b67c-416e-4de6-b41c-334ef88ea3ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6827","nonce":1483038287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6829","nonce":1483038287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6833","nonce":1483038287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1483038287},{"type":"v1","addr":"172.21.15.66:6831","nonce":1483038287}]},"public_addr":"172.21.15.66:6827/1483038287","cluster_addr":"172.21.15.66:6829/1483038287","heartbeat_back_addr":"172.21.15.66:6833/1483038287","heartbeat_front_addr":"172.21.15.66:6831/1483038287","state":["exists","up"]},{"osd":10,"uuid":"0a1a9369-e557-41ed-99a8-942cc6a1bd91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6825","nonce":2551875994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6827","nonce":2551875994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6831","nonce":2551875994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2551875994},{"type":"v1","addr":"172.21.15.83:6829","nonce":2551875994}]},"public_addr":"172.21.15.83:6825/2551875994","cluster_addr":"172.21.15.83:6827/2551875994","heartbeat_back_addr":"172.21.15.83:6831/2551875994","heartbeat_front_addr":"172.21.15.83:6829/2551875994","state":["exists","up"]},{"osd":11,"uuid":"6d75b4f1-8fc1-4a24-9aaf-1f17cf25a4bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6825","nonce":3472608146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6827","nonce":3472608146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6831","nonce":3472608146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3472608146},{"type":"v1","addr":"172.21.15.186:6829","nonce":3472608146}]},"public_addr":"172.21.15.186:6825/3472608146","cluster_addr":"172.21.15.186:6827/3472608146","heartbeat_back_addr":"172.21.15.186:6831/3472608146","heartbeat_front_addr":"172.21.15.186:6829/3472608146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:14.344184+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:00:38.514963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:02.691210+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:28.364413+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:01:53.973025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:19.473535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:02:45.301701+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:10.547426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:03:36.074757+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:02.568174+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:28.377180+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T09:04:55.090619+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6801/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6800/3032531159":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/838597709":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/1898303764":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/612283217":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/3217056690":"2024-04-22T09:08:12.028364+0000","172.21.15.66:6801/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2187684654":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/3374606173":"2024-04-22T08:58:38.211749+0000","172.21.15.66:6800/763565357":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/502604018":"2024-04-22T08:58:12.038096+0000","172.21.15.66:6801/3624224927":"2024-04-22T08:58:12.038096+0000","172.21.15.66:0/2365699589":"2024-04-22T08:58:38.211749+0000","172.21.15.66:0/389506800":"2024-04-22T09:08:12.028364+0000","172.21.15.66:0/3868723687":"2024-04-22T08:58:12.038096+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T09:19:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.107+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f891c0a3d20 msgr2=0x7f891c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f891c0a3d20 0x7f891c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89240058c0 tx=0x7f8924005c60 comp rx=0 tx=0).stop 2024-04-21T09:19:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934106180 msgr2=0x7f893419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934106180 0x7f893419e920 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8928036040 tx=0x7f892802c7e0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 shutdown_connections 2024-04-21T09:19:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f891c0a3d20 0x7f891c0a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8934198590 0x7f8934198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8934106180 0x7f893419e920 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 --2- 172.21.15.66:0/3478468118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89341010b0 0x7f893419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 >> 172.21.15.66:0/3478468118 conn(0x7f89340771d0 msgr2=0x7f8934104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 shutdown_connections 2024-04-21T09:19:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.108+0000 7f8939a81640 1 -- 172.21.15.66:0/3478468118 wait complete. 2024-04-21T09:19:13.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:12 smithi083 ceph-mon[31413]: pgmap v343: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:19:13.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2134037543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:19:13.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:12 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/758648577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:19:13.118 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:12 smithi066 ceph-mon[27953]: pgmap v343: 129 pgs: 129 active+clean; 2.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T09:19:13.118 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2134037543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:19:13.118 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:12 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/758648577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T09:19:13.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 -- 172.21.15.66:0/3276780530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718104380 msgr2=0x7f8718104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 --2- 172.21.15.66:0/3276780530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718104380 0x7f8718104760 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8708009900 tx=0x7f870802bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 -- 172.21.15.66:0/3276780530 shutdown_connections 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 --2- 172.21.15.66:0/3276780530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718109450 0x7f8718111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 --2- 172.21.15.66:0/3276780530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718104d30 0x7f8718108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 --2- 172.21.15.66:0/3276780530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8718104380 0x7f8718104760 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 -- 172.21.15.66:0/3276780530 >> 172.21.15.66:0/3276780530 conn(0x7f87180ffbc0 msgr2=0x7f8718101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 -- 172.21.15.66:0/3276780530 shutdown_connections 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.225+0000 7f871e481640 1 -- 172.21.15.66:0/3276780530 wait complete. 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 Processor -- start 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 -- start start 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718104380 0x7f87181a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718109450 0x7f87181a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871819a840 0x7f871819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87181143c0 con 0x7f871819a840 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8718114240 con 0x7f8718109450 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871e481640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8718114540 con 0x7f8718104380 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.226+0000 7f871ca1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871819a840 0x7f871819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871819a840 0x7f871819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38594/0 (socket says 172.21.15.66:38594) 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 -- 172.21.15.66:0/2336932874 learned_addr learned my addr 172.21.15.66:0/2336932874 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f87177fe640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718109450 0x7f87181a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f8717fff640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718104380 0x7f87181a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 -- 172.21.15.66:0/2336932874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718104380 msgr2=0x7f87181a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718104380 0x7f87181a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 -- 172.21.15.66:0/2336932874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718109450 msgr2=0x7f87181a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718109450 0x7f87181a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 -- 172.21.15.66:0/2336932874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871819b3d0 con 0x7f871819a840 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871ca1e640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871819a840 0x7f871819acf0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f871000eaa0 tx=0x7f871000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f87157fa640 1 -- 172.21.15.66:0/2336932874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871000cdb0 con 0x7f871819a840 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f87157fa640 1 -- 172.21.15.66:0/2336932874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8710005020 con 0x7f871819a840 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871819b6c0 con 0x7f871819a840 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f87157fa640 1 -- 172.21.15.66:0/2336932874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100116d0 con 0x7f871819a840 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.227+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181a7850 con 0x7f871819a840 2024-04-21T09:19:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.228+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8718105850 con 0x7f871819a840 2024-04-21T09:19:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.229+0000 7f87157fa640 1 -- 172.21.15.66:0/2336932874 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f87100051d0 con 0x7f871819a840 2024-04-21T09:19:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.230+0000 7f87157fa640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f87000a3d20 0x7f87000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.230+0000 7f87157fa640 1 -- 172.21.15.66:0/2336932874 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f87100d0d10 con 0x7f871819a840 2024-04-21T09:19:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.230+0000 7f8717fff640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f87000a3d20 0x7f87000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.231+0000 7f8717fff640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f87000a3d20 0x7f87000a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f870802c6b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.235+0000 7f87157fa640 1 -- 172.21.15.66:0/2336932874 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f871819b6c0 con 0x7f871819a840 2024-04-21T09:19:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.461+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8718104d30 con 0x7f871819a840 2024-04-21T09:19:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.464+0000 7f87157fa640 1 -- 172.21.15.66:0/2336932874 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f8710085e20 con 0x7f871819a840 2024-04-21T09:19:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-21T09:19:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f87000a3d20 msgr2=0x7f87000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f87000a3d20 0x7f87000a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f870802c6b0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871819a840 msgr2=0x7f871819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871819a840 0x7f871819acf0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f871000eaa0 tx=0x7f871000ef60 comp rx=0 tx=0).stop 2024-04-21T09:19:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 shutdown_connections 2024-04-21T09:19:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f87000a3d20 0x7f87000a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871819a840 0x7f871819acf0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718109450 0x7f87181a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 --2- 172.21.15.66:0/2336932874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8718104380 0x7f87181a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 >> 172.21.15.66:0/2336932874 conn(0x7f87180ffbc0 msgr2=0x7f8718100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 shutdown_connections 2024-04-21T09:19:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.467+0000 7f871e481640 1 -- 172.21.15.66:0/2336932874 wait complete. 2024-04-21T09:19:13.480 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.589+0000 7f0fa2fca640 1 -- 172.21.15.66:0/1013401788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c101a30 msgr2=0x7f0f9c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.589+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/1013401788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c101a30 0x7f0f9c105a70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009900 tx=0x7f0f8802bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 -- 172.21.15.66:0/1013401788 shutdown_connections 2024-04-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/1013401788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c106150 0x7f0f9c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/1013401788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c101a30 0x7f0f9c105a70 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/1013401788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9c101080 0x7f0f9c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 -- 172.21.15.66:0/1013401788 >> 172.21.15.66:0/1013401788 conn(0x7f0f9c078000 msgr2=0x7f0f9c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 -- 172.21.15.66:0/1013401788 shutdown_connections 2024-04-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 -- 172.21.15.66:0/1013401788 wait complete. 2024-04-21T09:19:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 Processor -- start 2024-04-21T09:19:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.590+0000 7f0fa2fca640 1 -- start start 2024-04-21T09:19:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa2fca640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c101080 0x7f0f9c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa2fca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9c106150 0x7f0f9c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa2fca640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c19bf60 0x7f0f9c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa2fca640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c111070 con 0x7f0f9c19bf60 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa2fca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c110ef0 con 0x7f0f9c106150 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa2fca640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1111f0 con 0x7f0f9c101080 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c101080 0x7f0f9c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa1567640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c19bf60 0x7f0f9c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c101080 0x7f0f9c1976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.66:37588/0 (socket says 172.21.15.66:37588) 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 -- 172.21.15.66:0/2198061430 learned_addr learned my addr 172.21.15.66:0/2198061430 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0f9bfff640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9c106150 0x7f0f9c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 -- 172.21.15.66:0/2198061430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9c106150 msgr2=0x7f0f9c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9c106150 0x7f0f9c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 -- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c19bf60 msgr2=0x7f0f9c19c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c19bf60 0x7f0f9c19c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa0d66640 1 -- 172.21.15.66:0/2198061430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c19ca50 con 0x7f0f9c101080 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.591+0000 7f0fa1567640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c19bf60 0x7f0f9c19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.592+0000 7f0fa0d66640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c101080 0x7f0f9c1976a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000ea60 tx=0x7f0f9000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.592+0000 7f0f99ffb640 1 -- 172.21.15.66:0/2198061430 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000cde0 con 0x7f0f9c101080 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.592+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c19cd40 con 0x7f0f9c101080 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.592+0000 7f0f99ffb640 1 -- 172.21.15.66:0/2198061430 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f900035f0 con 0x7f0f9c101080 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.592+0000 7f0f99ffb640 1 -- 172.21.15.66:0/2198061430 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90010690 con 0x7f0f9c101080 2024-04-21T09:19:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.592+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c1a4500 con 0x7f0f9c101080 2024-04-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.593+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c068d70 con 0x7f0f9c101080 2024-04-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.594+0000 7f0f99ffb640 1 -- 172.21.15.66:0/2198061430 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0f900037a0 con 0x7f0f9c101080 2024-04-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.595+0000 7f0f99ffb640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f8c0a3d20 0x7f0f8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.595+0000 7f0f9bfff640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f8c0a3d20 0x7f0f8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.595+0000 7f0f99ffb640 1 -- 172.21.15.66:0/2198061430 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9666+0+0 (secure 0 0 0) 0x7f0f900d0f80 con 0x7f0f9c101080 2024-04-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.596+0000 7f0f9bfff640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f8c0a3d20 0x7f0f8c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009230 tx=0x7f0f88005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.600+0000 7f0f99ffb640 1 -- 172.21.15.66:0/2198061430 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f9009d140 con 0x7f0f9c101080 2024-04-21T09:19:13.826 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a[27949]: 2024-04-21T09:19:13.462+0000 7f0318de1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T09:19:13.826 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i[63118]: ignoring --setuser ceph since I am not root 2024-04-21T09:19:13.827 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i[63118]: ignoring --setgroup ceph since I am not root 2024-04-21T09:19:13.827 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mds[63122]: -- 172.21.15.186:0/988550989 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ef2733a40 con 0x560ef2707000 2024-04-21T09:19:13.827 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i[63118]: starting mds.i at 2024-04-21T09:19:13.827 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h[62149]: ignoring --setuser ceph since I am not root 2024-04-21T09:19:13.827 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h[62149]: ignoring --setgroup ceph since I am not root 2024-04-21T09:19:13.827 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mds[62153]: -- 172.21.15.83:0/3121223540 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b83e2d7a40 con 0x55b83e2aac00 2024-04-21T09:19:13.827 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h[62149]: starting mds.h at 2024-04-21T09:19:13.828 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c[62802]: ignoring --setuser ceph since I am not root 2024-04-21T09:19:13.828 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c[62802]: ignoring --setgroup ceph since I am not root 2024-04-21T09:19:13.828 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mds[62806]: -- 172.21.15.186:0/3292079098 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d50bf05a40 con 0x55d50bed9000 2024-04-21T09:19:13.828 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c[62802]: starting mds.c at 2024-04-21T09:19:13.828 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: ignoring --setuser ceph since I am not root 2024-04-21T09:19:13.828 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: ignoring --setgroup ceph since I am not root 2024-04-21T09:19:13.828 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mds[62459]: -- 172.21.15.83:0/758678872 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f04a4da40 con 0x559f04a20800 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: starting mds.b at 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f[62492]: ignoring --setuser ceph since I am not root 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f[62492]: ignoring --setgroup ceph since I am not root 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mds[62496]: -- 172.21.15.186:0/971280490 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583a5127a40 con 0x5583a50fa800 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f[62492]: starting mds.f at 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e[61836]: ignoring --setuser ceph since I am not root 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e[61836]: ignoring --setgroup ceph since I am not root 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mds[61840]: -- 172.21.15.83:0/1930598253 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ccd365a40 con 0x564ccd338800 2024-04-21T09:19:13.829 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e[61836]: starting mds.e at 2024-04-21T09:19:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:13.850+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0f9c109380 con 0x7f0f9c101080 2024-04-21T09:19:14.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/3478468118' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:19:14.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2336932874' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T09:19:14.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mon[31505]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T09:19:14.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mon[31505]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T09:19:14.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2336932874' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T09:19:14.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mon[31505]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T09:19:14.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:13 smithi186 ceph-mon[31505]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-21T09:19:14.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/3478468118' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:19:14.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2336932874' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T09:19:14.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mon[31413]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T09:19:14.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mon[31413]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T09:19:14.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2336932874' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T09:19:14.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mon[31413]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T09:19:14.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:13 smithi083 ceph-mon[31413]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-21T09:19:14.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/3478468118' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T09:19:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2336932874' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T09:19:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-mon[27953]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T09:19:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-mon[27953]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T09:19:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2336932874' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T09:19:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-mon[27953]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T09:19:14.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:13 smithi066 ceph-mon[27953]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-21T09:19:14.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.469+0000 7f0f99ffb640 1 -- 172.21.15.66:0/2198061430 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f0f9009ca40 con 0x7f0f9c101080 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f8c0a3d20 msgr2=0x7f0f8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f8c0a3d20 0x7f0f8c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009230 tx=0x7f0f88005fd0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c101080 msgr2=0x7f0f9c1976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c101080 0x7f0f9c1976a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000ea60 tx=0x7f0f9000ef20 comp rx=0 tx=0).stop 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 shutdown_connections 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f0f8c0a3d20 0x7f0f8c0a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f9c19bf60 0x7f0f9c19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9c106150 0x7f0f9c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 --2- 172.21.15.66:0/2198061430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f9c101080 0x7f0f9c1976a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 >> 172.21.15.66:0/2198061430 conn(0x7f0f9c078000 msgr2=0x7f0f9c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 shutdown_connections 2024-04-21T09:19:14.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.472+0000 7f0fa2fca640 1 -- 172.21.15.66:0/2198061430 wait complete. 2024-04-21T09:19:14.484 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-21T09:19:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 -- 172.21.15.66:0/4271717609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 msgr2=0x7f8a1810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/4271717609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a1810ee40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009ca0 tx=0x7f8a1002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 -- 172.21.15.66:0/4271717609 shutdown_connections 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/4271717609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a1810ee40 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/4271717609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a18102b50 0x7f8a18106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/4271717609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a181021a0 0x7f8a18102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 -- 172.21.15.66:0/4271717609 >> 172.21.15.66:0/4271717609 conn(0x7f8a180fda40 msgr2=0x7f8a180ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 -- 172.21.15.66:0/4271717609 shutdown_connections 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 -- 172.21.15.66:0/4271717609 wait complete. 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.591+0000 7f8a1d99d640 1 Processor -- start 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a1d99d640 1 -- start start 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a1d99d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a181021a0 0x7f8a18198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a1d99d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a18198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a1d99d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1819cfb0 0x7f8a1819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a1d99d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181121c0 con 0x7f8a18107270 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a1d99d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18112040 con 0x7f8a181021a0 2024-04-21T09:19:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a1d99d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18112340 con 0x7f8a1819cfb0 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a18198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a18198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38630/0 (socket says 172.21.15.66:38630) 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 -- 172.21.15.66:0/2019841812 learned_addr learned my addr 172.21.15.66:0/2019841812 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a16ffd640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a181021a0 0x7f8a18198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a177fe640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1819cfb0 0x7f8a1819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 -- 172.21.15.66:0/2019841812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1819cfb0 msgr2=0x7f8a1819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1819cfb0 0x7f8a1819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 -- 172.21.15.66:0/2019841812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a181021a0 msgr2=0x7f8a18198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a181021a0 0x7f8a18198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 -- 172.21.15.66:0/2019841812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1819daa0 con 0x7f8a18107270 2024-04-21T09:19:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.592+0000 7f8a167fc640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a18198c80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d770 tx=0x7f8a0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:14.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.593+0000 7f8a1c99b640 1 -- 172.21.15.66:0/2019841812 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08017070 con 0x7f8a18107270 2024-04-21T09:19:14.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.593+0000 7f8a1c99b640 1 -- 172.21.15.66:0/2019841812 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a08009c30 con 0x7f8a18107270 2024-04-21T09:19:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.593+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1819dd90 con 0x7f8a18107270 2024-04-21T09:19:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.593+0000 7f8a1c99b640 1 -- 172.21.15.66:0/2019841812 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800f620 con 0x7f8a18107270 2024-04-21T09:19:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.593+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181a5580 con 0x7f8a18107270 2024-04-21T09:19:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.594+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a180691c0 con 0x7f8a18107270 2024-04-21T09:19:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.596+0000 7f8a1c99b640 1 -- 172.21.15.66:0/2019841812 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8a0801a070 con 0x7f8a18107270 2024-04-21T09:19:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.596+0000 7f8a1c99b640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a040a3d20 0x7f8a040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.596+0000 7f8a16ffd640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a040a3d20 0x7f8a040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.596+0000 7f8a1c99b640 1 -- 172.21.15.66:0/2019841812 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9666+0+0 (secure 0 0 0) 0x7f8a080d0640 con 0x7f8a18107270 2024-04-21T09:19:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.597+0000 7f8a16ffd640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a040a3d20 0x7f8a040a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8a000058c0 tx=0x7f8a0000a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.600+0000 7f8a1c99b640 1 -- 172.21.15.66:0/2019841812 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a08009de0 con 0x7f8a18107270 2024-04-21T09:19:14.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:14.788+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a18102b50 con 0x7f8a18107270 2024-04-21T09:19:15.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2198061430' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: pgmap v345: 129 pgs: 129 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.0 KiB/s wr, 9 op/s 2024-04-21T09:19:15.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T09:19:15.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T09:19:15.088 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: Cluster is now healthy 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6834/2229846319,v1:172.21.15.186:6835/2229846319] up:boot 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6834/3779834702,v1:172.21.15.83:6835/3779834702] up:boot 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6836/877802766,v1:172.21.15.83:6838/877802766] up:boot 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6836/2677025443,v1:172.21.15.186:6838/2677025443] up:boot 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.83:6837/3789275818,v1:172.21.15.83:6839/3789275818] up:boot 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: mds.? [v2:172.21.15.186:6837/690166422,v1:172.21.15.186:6839/690166422] up:boot 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: fsmap 12 up:standby 2024-04-21T09:19:15.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:19:15.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:19:15.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:19:15.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:19:15.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:19:15.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:19:15.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:14 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2019841812' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2198061430' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: pgmap v345: 129 pgs: 129 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.0 KiB/s wr, 9 op/s 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: Cluster is now healthy 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6834/2229846319,v1:172.21.15.186:6835/2229846319] up:boot 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6834/3779834702,v1:172.21.15.83:6835/3779834702] up:boot 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6836/877802766,v1:172.21.15.83:6838/877802766] up:boot 2024-04-21T09:19:15.110 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6836/2677025443,v1:172.21.15.186:6838/2677025443] up:boot 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.83:6837/3789275818,v1:172.21.15.83:6839/3789275818] up:boot 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: mds.? [v2:172.21.15.186:6837/690166422,v1:172.21.15.186:6839/690166422] up:boot 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: fsmap 12 up:standby 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:19:15.111 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:14 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2019841812' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.119 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2198061430' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: pgmap v345: 129 pgs: 129 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.0 KiB/s wr, 9 op/s 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: Cluster is now healthy 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6834/2229846319,v1:172.21.15.186:6835/2229846319] up:boot 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6834/3779834702,v1:172.21.15.83:6835/3779834702] up:boot 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6836/877802766,v1:172.21.15.83:6838/877802766] up:boot 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6836/2677025443,v1:172.21.15.186:6838/2677025443] up:boot 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.83:6837/3789275818,v1:172.21.15.83:6839/3789275818] up:boot 2024-04-21T09:19:15.120 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: mds.? [v2:172.21.15.186:6837/690166422,v1:172.21.15.186:6839/690166422] up:boot 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: fsmap 12 up:standby 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='mgr.15456 172.21.15.66:0/2425280947' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T09:19:15.121 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:14 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2019841812' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:15.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.467+0000 7f8a1c99b640 1 -- 172.21.15.66:0/2019841812 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f8a0809c210 con 0x7f8a18107270 2024-04-21T09:19:15.468 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a040a3d20 msgr2=0x7f8a040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a040a3d20 0x7f8a040a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8a000058c0 tx=0x7f8a0000a2d0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 msgr2=0x7f8a18198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a18198c80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d770 tx=0x7f8a0800dc30 comp rx=0 tx=0).stop 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 shutdown_connections 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f8a040a3d20 0x7f8a040a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a1819cfb0 0x7f8a1819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a18107270 0x7f8a18198c80 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 --2- 172.21.15.66:0/2019841812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a181021a0 0x7f8a18198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 >> 172.21.15.66:0/2019841812 conn(0x7f8a180fda40 msgr2=0x7f8a180fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 shutdown_connections 2024-04-21T09:19:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.471+0000 7f8a1d99d640 1 -- 172.21.15.66:0/2019841812 wait complete. 2024-04-21T09:19:15.486 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-21T09:19:15.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 -- 172.21.15.66:0/3611192507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 msgr2=0x7fa95c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:15.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 --2- 172.21.15.66:0/3611192507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c101470 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009900 tx=0x7fa94c02bc30 comp rx=0 tx=0).stop 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 -- 172.21.15.66:0/3611192507 shutdown_connections 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 --2- 172.21.15.66:0/3611192507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c106160 0x7fa95c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 --2- 172.21.15.66:0/3611192507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa95c101a40 0x7fa95c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 --2- 172.21.15.66:0/3611192507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c101470 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 -- 172.21.15.66:0/3611192507 >> 172.21.15.66:0/3611192507 conn(0x7fa95c077f50 msgr2=0x7fa95c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 -- 172.21.15.66:0/3611192507 shutdown_connections 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 -- 172.21.15.66:0/3611192507 wait complete. 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 Processor -- start 2024-04-21T09:19:15.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.599+0000 7fa9629b6640 1 -- start start 2024-04-21T09:19:15.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa9629b6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:15.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa9629b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c106160 0x7fa95c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:15.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa9629b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa95c19cfc0 0x7fa95c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa9629b6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1121e0 con 0x7fa95c101090 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa9629b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c112060 con 0x7fa95c19cfc0 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa9629b6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c112360 con 0x7fa95c106160 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52898/0 (socket says 172.21.15.66:52898) 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 -- 172.21.15.66:0/2681807942 learned_addr learned my addr 172.21.15.66:0/2681807942 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa960f53640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa95c19cfc0 0x7fa95c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95b7fe640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c106160 0x7fa95c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 -- 172.21.15.66:0/2681807942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c106160 msgr2=0x7fa95c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c106160 0x7fa95c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 -- 172.21.15.66:0/2681807942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa95c19cfc0 msgr2=0x7fa95c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa95c19cfc0 0x7fa95c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:15.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 -- 172.21.15.66:0/2681807942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c19dab0 con 0x7fa95c101090 2024-04-21T09:19:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa95bfff640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c198700 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005940 tx=0x7fa94c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.600+0000 7fa9597fa640 1 -- 172.21.15.66:0/2681807942 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c043020 con 0x7fa95c101090 2024-04-21T09:19:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.601+0000 7fa9597fa640 1 -- 172.21.15.66:0/2681807942 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c02caf0 con 0x7fa95c101090 2024-04-21T09:19:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.601+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c19dd40 con 0x7fa95c101090 2024-04-21T09:19:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.601+0000 7fa9597fa640 1 -- 172.21.15.66:0/2681807942 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c003af0 con 0x7fa95c101090 2024-04-21T09:19:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.601+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c1a5560 con 0x7fa95c101090 2024-04-21T09:19:15.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.602+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95c068d70 con 0x7fa95c101090 2024-04-21T09:19:15.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.603+0000 7fa9597fa640 1 -- 172.21.15.66:0/2681807942 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa94c046020 con 0x7fa95c101090 2024-04-21T09:19:15.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.603+0000 7fa9597fa640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa9440a3d20 0x7fa9440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:15.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.604+0000 7fa95b7fe640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa9440a3d20 0x7fa9440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:15.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.604+0000 7fa9597fa640 1 -- 172.21.15.66:0/2681807942 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9112+0+0 (secure 0 0 0) 0x7fa94c0f2be0 con 0x7fa95c101090 2024-04-21T09:19:15.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.604+0000 7fa95b7fe640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa9440a3d20 0x7fa9440a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa95c1009d0 tx=0x7fa950005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:15.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.608+0000 7fa9597fa640 1 -- 172.21.15.66:0/2681807942 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa94c04d020 con 0x7fa95c101090 2024-04-21T09:19:15.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:15.796+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa95c101a40 con 0x7fa95c101090 2024-04-21T09:19:16.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.475+0000 7fa9597fa640 1 -- 172.21.15.66:0/2681807942 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7fa94c03c350 con 0x7fa95c101090 2024-04-21T09:19:16.476 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-04-21T09:19:16.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa9440a3d20 msgr2=0x7fa9440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa9440a3d20 0x7fa9440a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa95c1009d0 tx=0x7fa950005f90 comp rx=0 tx=0).stop 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 msgr2=0x7fa95c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c198700 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005940 tx=0x7fa94c0045e0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 shutdown_connections 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7fa9440a3d20 0x7fa9440a61e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa95c19cfc0 0x7fa95c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c106160 0x7fa95c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 --2- 172.21.15.66:0/2681807942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95c101090 0x7fa95c198700 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 >> 172.21.15.66:0/2681807942 conn(0x7fa95c077f50 msgr2=0x7fa95c0feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 shutdown_connections 2024-04-21T09:19:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.480+0000 7fa9629b6640 1 -- 172.21.15.66:0/2681807942 wait complete. 2024-04-21T09:19:16.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-21T09:19:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 -- 172.21.15.66:0/2733166165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8106150 msgr2=0x7f73c810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 --2- 172.21.15.66:0/2733166165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8106150 0x7f73c810dd20 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009ca0 tx=0x7f73c002c0f0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 -- 172.21.15.66:0/2733166165 shutdown_connections 2024-04-21T09:19:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 --2- 172.21.15.66:0/2733166165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8106150 0x7f73c810dd20 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 --2- 172.21.15.66:0/2733166165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c8101a30 0x7f73c8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 --2- 172.21.15.66:0/2733166165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73c8101080 0x7f73c8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 -- 172.21.15.66:0/2733166165 >> 172.21.15.66:0/2733166165 conn(0x7f73c8078000 msgr2=0x7f73c8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 -- 172.21.15.66:0/2733166165 shutdown_connections 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 -- 172.21.15.66:0/2733166165 wait complete. 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 Processor -- start 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.609+0000 7f73cd69a640 1 -- start start 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8101080 0x7f73c8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c8106150 0x7f73c81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73c8199f50 0x7f73c81a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73c81110a0 con 0x7f73c8101080 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73c8110f20 con 0x7f73c8199f50 2024-04-21T09:19:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f73c8111220 con 0x7f73c8106150 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8101080 0x7f73c8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8101080 0x7f73c8199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52922/0 (socket says 172.21.15.66:52922) 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 -- 172.21.15.66:0/1110595570 learned_addr learned my addr 172.21.15.66:0/1110595570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c77fe640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73c8199f50 0x7f73c81a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73bffff640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c8106150 0x7f73c81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 -- 172.21.15.66:0/1110595570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c8106150 msgr2=0x7f73c81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c8106150 0x7f73c81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 -- 172.21.15.66:0/1110595570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73c8199f50 msgr2=0x7f73c81a4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73c8199f50 0x7f73c81a4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 -- 172.21.15.66:0/1110595570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c81a4d00 con 0x7f73c8101080 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c6ffd640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8101080 0x7f73c8199480 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f73b800e9f0 tx=0x7f73b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c4ff9640 1 -- 172.21.15.66:0/1110595570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b800ccc0 con 0x7f73c8101080 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c4ff9640 1 -- 172.21.15.66:0/1110595570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73b80108e0 con 0x7f73c8101080 2024-04-21T09:19:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c81a4ff0 con 0x7f73c8101080 2024-04-21T09:19:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73c4ff9640 1 -- 172.21.15.66:0/1110595570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8018880 con 0x7f73c8101080 2024-04-21T09:19:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.610+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c81a54b0 con 0x7f73c8101080 2024-04-21T09:19:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.611+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c8068d70 con 0x7f73c8101080 2024-04-21T09:19:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.613+0000 7f73c4ff9640 1 -- 172.21.15.66:0/1110595570 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f73b8010430 con 0x7f73c8101080 2024-04-21T09:19:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.614+0000 7f73c4ff9640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f73b40a3d20 0x7f73b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T09:19:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.614+0000 7f73bffff640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f73b40a3d20 0x7f73b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T09:19:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.614+0000 7f73c4ff9640 1 -- 172.21.15.66:0/1110595570 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8582+0+0 (secure 0 0 0) 0x7f73b80d1080 con 0x7f73c8101080 2024-04-21T09:19:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.615+0000 7f73bffff640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f73b40a3d20 0x7f73b40a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73c8069c50 tx=0x7f73b000a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T09:19:16.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.618+0000 7f73c4ff9640 1 -- 172.21.15.66:0/1110595570 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73b809d6e0 con 0x7f73c8101080 2024-04-21T09:19:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:16.805+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f73c80696f0 con 0x7f73c8101080 2024-04-21T09:19:16.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2019841812' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T09:19:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:16 smithi186 ceph-mon[31505]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T09:19:16.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:16 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2681807942' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:16.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2019841812' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T09:19:16.859 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:16 smithi083 ceph-mon[31413]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T09:19:16.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:16 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2681807942' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:16.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2019841812' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T09:19:16.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:16 smithi066 ceph-mon[27953]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T09:19:16.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:16 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2681807942' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.477+0000 7f73c4ff9640 1 -- 172.21.15.66:0/1110595570 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v110) ==== 178+0+0 (secure 0 0 0) 0x7f73b809cfe0 con 0x7f73c8101080 2024-04-21T09:19:17.478 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' removed 2024-04-21T09:19:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.482+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f73b40a3d20 msgr2=0x7f73b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.482+0000 7f73cd69a640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f73b40a3d20 0x7f73b40a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73c8069c50 tx=0x7f73b000a2d0 comp rx=0 tx=0).stop 2024-04-21T09:19:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.482+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8101080 msgr2=0x7f73c8199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T09:19:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.482+0000 7f73cd69a640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8101080 0x7f73c8199480 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f73b800e9f0 tx=0x7f73b800edf0 comp rx=0 tx=0).stop 2024-04-21T09:19:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 shutdown_connections 2024-04-21T09:19:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:6800/3952680321,v1:172.21.15.66:6801/3952680321] conn(0x7f73b40a3d20 0x7f73b40a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73c8199f50 0x7f73c81a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c8106150 0x7f73c81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 --2- 172.21.15.66:0/1110595570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c8101080 0x7f73c8199480 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T09:19:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 >> 172.21.15.66:0/1110595570 conn(0x7f73c8078000 msgr2=0x7f73c80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T09:19:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 shutdown_connections 2024-04-21T09:19:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-21T09:19:17.483+0000 7f73cd69a640 1 -- 172.21.15.66:0/1110595570 wait complete. 2024-04-21T09:19:17.505 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:19:17.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:19:17.563 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:19:17.594 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-21T09:19:17.594 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T09:19:17.610 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T09:19:17.626 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T09:19:17.649 INFO:tasks.cephadm:Stopping all daemons... 2024-04-21T09:19:17.649 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-21T09:19:17.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a 2024-04-21T09:19:17.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:17 smithi186 ceph-mon[31505]: pgmap v347: 97 pgs: 97 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:19:17.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/2681807942' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T09:19:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:17 smithi186 ceph-mon[31505]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T09:19:17.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:17 smithi186 ceph-mon[31505]: from='client.? 172.21.15.66:0/1110595570' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:17.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:17 smithi083 ceph-mon[31413]: pgmap v347: 97 pgs: 97 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:19:17.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/2681807942' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T09:19:17.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:17 smithi083 ceph-mon[31413]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T09:19:17.860 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:17 smithi083 ceph-mon[31413]: from='client.? 172.21.15.66:0/1110595570' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:17.869 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:17 smithi066 ceph-mon[27953]: pgmap v347: 97 pgs: 97 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T09:19:17.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/2681807942' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T09:19:17.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:17 smithi066 ceph-mon[27953]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T09:19:17.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:17 smithi066 ceph-mon[27953]: from='client.? 172.21.15.66:0/1110595570' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T09:19:17.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:17 smithi066 systemd[1]: Stopping Ceph mon.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:18.367 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:17 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a[27949]: 2024-04-21T09:19:17.965+0000 7f031e5ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:18.367 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:17 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a[27949]: 2024-04-21T09:19:17.965+0000 7f031e5ec700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-21T09:19:18.367 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:18 smithi066 podman[95911]: 2024-04-21 09:19:18.079282036 +0000 UTC m=+0.232625063 container died 8ad485c04e099fdaa3858d19741662bb72a551f19a99292213fda28bcd976c03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T09:19:18.619 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:18 smithi066 podman[95911]: 2024-04-21 09:19:18.458829873 +0000 UTC m=+0.612172898 container cleanup 8ad485c04e099fdaa3858d19741662bb72a551f19a99292213fda28bcd976c03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-21T09:19:18.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Apr 21 09:19:18 smithi066 bash[95911]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-a 2024-04-21T09:19:18.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.a.service' 2024-04-21T09:19:18.900 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:18.900 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-21T09:19:18.900 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-21T09:19:18.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.b 2024-04-21T09:19:19.212 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:18 smithi083 systemd[1]: Stopping Ceph mon.b for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:19.538 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-b[31409]: 2024-04-21T09:19:19.211+0000 7fe51dfaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:19.538 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-b[31409]: 2024-04-21T09:19:19.211+0000 7fe51dfaf700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-21T09:19:19.538 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:19 smithi083 podman[67789]: 2024-04-21 09:19:19.25287244 +0000 UTC m=+0.159207359 container died a844b996d03665f806104e7207f8fed6f679a70b5cf1b6aec04e813432a5ddf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T09:19:19.855 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:19 smithi083 podman[67789]: 2024-04-21 09:19:19.629652943 +0000 UTC m=+0.535987862 container cleanup a844b996d03665f806104e7207f8fed6f679a70b5cf1b6aec04e813432a5ddf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-21T09:19:19.855 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:19 smithi083 bash[67789]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-b 2024-04-21T09:19:20.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.b.service' 2024-04-21T09:19:20.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:19 smithi083 podman[67804]: 2024-04-21 09:19:19.856243196 +0000 UTC m=+0.603565778 container remove a844b996d03665f806104e7207f8fed6f679a70b5cf1b6aec04e813432a5ddf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:19:20.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:20 smithi083 systemd[1]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.b.service: Deactivated successfully. 2024-04-21T09:19:20.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:20 smithi083 systemd[1]: Stopped Ceph mon.b for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:19:20.109 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 21 09:19:20 smithi083 systemd[1]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.b.service: Consumed 12.288s CPU time. 2024-04-21T09:19:20.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:20.153 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-21T09:19:20.153 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-21T09:19:20.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.c 2024-04-21T09:19:20.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:20 smithi186 systemd[1]: Stopping Ceph mon.c for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:20.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-c[31501]: 2024-04-21T09:19:20.477+0000 7f5865b46700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:20.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-c[31501]: 2024-04-21T09:19:20.477+0000 7f5865b46700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-21T09:19:20.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:20 smithi186 podman[68562]: 2024-04-21 09:19:20.610160908 +0000 UTC m=+0.258501316 container died 9bbbd0ab3cbc484f6ed52d4e674da6c13dc37761460d781e52e633e6be2c7c08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-21T09:19:21.279 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:21 smithi186 podman[68562]: 2024-04-21 09:19:21.020162468 +0000 UTC m=+0.668502886 container cleanup 9bbbd0ab3cbc484f6ed52d4e674da6c13dc37761460d781e52e633e6be2c7c08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-21T09:19:21.280 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 21 09:19:21 smithi186 bash[68562]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mon-c 2024-04-21T09:19:21.498 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mon.c.service' 2024-04-21T09:19:21.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:21.541 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-21T09:19:21.541 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-21T09:19:21.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.x 2024-04-21T09:19:21.856 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:19:21 smithi066 systemd[1]: Stopping Ceph mgr.x for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:22.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:19:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:19:21.854+0000 7f1cf940f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:22.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:19:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x[28187]: 2024-04-21T09:19:21.854+0000 7f1cf940f700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-21T09:19:22.120 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:19:21 smithi066 podman[96034]: 2024-04-21 09:19:21.886772257 +0000 UTC m=+0.151229887 container died d5f0a88d9d49850e725280cb02f1c415f8cf66ed958e37ceb0a1e3bc0b3dfaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-04-21T09:19:22.524 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:19:22 smithi066 podman[96034]: 2024-04-21 09:19:22.265357737 +0000 UTC m=+0.529815364 container cleanup d5f0a88d9d49850e725280cb02f1c415f8cf66ed958e37ceb0a1e3bc0b3dfaaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-21T09:19:22.524 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Apr 21 09:19:22 smithi066 bash[96034]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-x 2024-04-21T09:19:22.646 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.x.service' 2024-04-21T09:19:22.680 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:22.681 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-21T09:19:22.681 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-21T09:19:22.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.y 2024-04-21T09:19:22.994 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:19:22 smithi083 systemd[1]: Stopping Ceph mgr.y for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:23.352 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:19:23 smithi083 podman[67915]: 2024-04-21 09:19:23.047782585 +0000 UTC m=+0.157029368 container died 60da962a56036fddf1aae179af68ce272b6871f2b9f4f1ffbc5bb49879dfde87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T09:19:23.610 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:19:23 smithi083 podman[67915]: 2024-04-21 09:19:23.435046688 +0000 UTC m=+0.544293462 container cleanup 60da962a56036fddf1aae179af68ce272b6871f2b9f4f1ffbc5bb49879dfde87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-21T09:19:23.610 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:19:23 smithi083 bash[67915]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y 2024-04-21T09:19:23.909 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:19:23 smithi083 podman[67929]: 2024-04-21 09:19:23.653720257 +0000 UTC m=+0.598560022 container remove 60da962a56036fddf1aae179af68ce272b6871f2b9f4f1ffbc5bb49879dfde87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-21T09:19:23.910 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 21 09:19:23 smithi083 systemd[1]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-21T09:19:23.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.y.service' 2024-04-21T09:19:23.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:23.950 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-21T09:19:23.950 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-21T09:19:23.950 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.z 2024-04-21T09:19:24.304 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:19:23 smithi186 systemd[1]: Stopping Ceph mgr.z for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:24.588 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:19:24 smithi186 podman[68696]: 2024-04-21 09:19:24.304571684 +0000 UTC m=+0.164522917 container died 26a6274ad30b3c4972905a34b2307c7c54124b1bdd653da785647c90b76b89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:19:25.001 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:19:24 smithi186 podman[68696]: 2024-04-21 09:19:24.709361395 +0000 UTC m=+0.569312628 container cleanup 26a6274ad30b3c4972905a34b2307c7c54124b1bdd653da785647c90b76b89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:19:25.002 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 21 09:19:24 smithi186 bash[68696]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mgr-z 2024-04-21T09:19:25.136 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mgr.z.service' 2024-04-21T09:19:25.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:25.169 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-21T09:19:25.170 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-21T09:19:25.170 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.0 2024-04-21T09:19:25.494 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:25 smithi066 systemd[1]: Stopping Ceph osd.0 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:25.869 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:25 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0[40464]: 2024-04-21T09:19:25.494+0000 7fe21ba15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:25.869 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:25 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0[40464]: 2024-04-21T09:19:25.494+0000 7fe21ba15700 -1 osd.0 110 *** Got signal Terminated *** 2024-04-21T09:19:25.869 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:25 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0[40464]: 2024-04-21T09:19:25.494+0000 7fe21ba15700 -1 osd.0 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:19:31.221 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:30 smithi066 podman[96159]: 2024-04-21 09:19:30.957235977 +0000 UTC m=+5.588374756 container died 99d4fe73114ad6d6d74ddb8809fb514316d321fa7b3bd849a251988af53204f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-21T09:19:31.612 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:31 smithi066 podman[96159]: 2024-04-21 09:19:31.312049113 +0000 UTC m=+5.943187881 container cleanup 99d4fe73114ad6d6d74ddb8809fb514316d321fa7b3bd849a251988af53204f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T09:19:31.612 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:31 smithi066 bash[96159]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0 2024-04-21T09:19:31.870 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:31 smithi066 podman[96177]: 2024-04-21 09:19:31.613652006 +0000 UTC m=+0.652807353 container remove 99d4fe73114ad6d6d74ddb8809fb514316d321fa7b3bd849a251988af53204f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T09:19:32.291 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:31 smithi066 podman[96273]: 2024-04-21 09:19:31.849525794 +0000 UTC m=+0.018583953 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:19:32.291 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:31 smithi066 podman[96273]: 2024-04-21 09:19:31.964740936 +0000 UTC m=+0.133799097 container create b1c4ddf62d24f182fe019c438106d3b98ffe9b274c1c3cf520e96e5ff103997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:19:32.291 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:32 smithi066 podman[96273]: 2024-04-21 09:19:32.19228005 +0000 UTC m=+0.361338211 container init b1c4ddf62d24f182fe019c438106d3b98ffe9b274c1c3cf520e96e5ff103997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-21T09:19:32.291 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:32 smithi066 podman[96273]: 2024-04-21 09:19:32.195634476 +0000 UTC m=+0.364692641 container start b1c4ddf62d24f182fe019c438106d3b98ffe9b274c1c3cf520e96e5ff103997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-21T09:19:32.292 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:32 smithi066 podman[96273]: 2024-04-21 09:19:32.292366569 +0000 UTC m=+0.461424744 container attach b1c4ddf62d24f182fe019c438106d3b98ffe9b274c1c3cf520e96e5ff103997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-21T09:19:32.620 INFO:journalctl@ceph.osd.0.smithi066.stdout:Apr 21 09:19:32 smithi066 podman[96273]: 2024-04-21 09:19:32.509927908 +0000 UTC m=+0.678986066 container died b1c4ddf62d24f182fe019c438106d3b98ffe9b274c1c3cf520e96e5ff103997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-21T09:19:33.309 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.0.service' 2024-04-21T09:19:33.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:33.357 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-21T09:19:33.357 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-21T09:19:33.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.1 2024-04-21T09:19:33.859 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:33 smithi083 systemd[1]: Stopping Ceph osd.1 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:34.359 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1[36037]: 2024-04-21T09:19:33.906+0000 7fae60969700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:34.360 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1[36037]: 2024-04-21T09:19:33.906+0000 7fae60969700 -1 osd.1 110 *** Got signal Terminated *** 2024-04-21T09:19:34.360 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1[36037]: 2024-04-21T09:19:33.906+0000 7fae60969700 -1 osd.1 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:19:39.732 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:39 smithi083 podman[68048]: 2024-04-21 09:19:39.42955128 +0000 UTC m=+5.874210589 container died ce37e4c099c86f419fbe2a7d9ed04227b489622bf377f3f778d3a931a1c41fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:19:40.082 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:39 smithi083 podman[68048]: 2024-04-21 09:19:39.831550725 +0000 UTC m=+6.276210022 container cleanup ce37e4c099c86f419fbe2a7d9ed04227b489622bf377f3f778d3a931a1c41fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:19:40.082 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:39 smithi083 bash[68048]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1 2024-04-21T09:19:40.359 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:40 smithi083 podman[68066]: 2024-04-21 09:19:40.083061188 +0000 UTC m=+0.651967760 container remove ce37e4c099c86f419fbe2a7d9ed04227b489622bf377f3f778d3a931a1c41fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T09:19:40.694 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:40 smithi083 podman[68158]: 2024-04-21 09:19:40.2903771 +0000 UTC m=+0.017783918 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:19:40.694 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:40 smithi083 podman[68158]: 2024-04-21 09:19:40.425601045 +0000 UTC m=+0.153007909 container create f6b42323402408be1cb04afdfd349e919acc0b964be6277176fcdd2f2f6258b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T09:19:40.694 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:40 smithi083 podman[68158]: 2024-04-21 09:19:40.69524138 +0000 UTC m=+0.422648198 container init f6b42323402408be1cb04afdfd349e919acc0b964be6277176fcdd2f2f6258b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-21T09:19:40.991 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:40 smithi083 podman[68158]: 2024-04-21 09:19:40.698703256 +0000 UTC m=+0.426110074 container start f6b42323402408be1cb04afdfd349e919acc0b964be6277176fcdd2f2f6258b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-21T09:19:40.992 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:40 smithi083 podman[68158]: 2024-04-21 09:19:40.820631542 +0000 UTC m=+0.548038373 container attach f6b42323402408be1cb04afdfd349e919acc0b964be6277176fcdd2f2f6258b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T09:19:41.262 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 21 09:19:40 smithi083 podman[68158]: 2024-04-21 09:19:40.992910671 +0000 UTC m=+0.720317492 container died f6b42323402408be1cb04afdfd349e919acc0b964be6277176fcdd2f2f6258b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-21T09:19:41.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.1.service' 2024-04-21T09:19:41.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:41.689 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-21T09:19:41.689 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-21T09:19:41.689 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.2 2024-04-21T09:19:42.014 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:41 smithi186 systemd[1]: Stopping Ceph osd.2 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:42.337 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2[36820]: 2024-04-21T09:19:42.012+0000 7febff024700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:42.338 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2[36820]: 2024-04-21T09:19:42.012+0000 7febff024700 -1 osd.2 110 *** Got signal Terminated *** 2024-04-21T09:19:42.338 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2[36820]: 2024-04-21T09:19:42.012+0000 7febff024700 -1 osd.2 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:19:47.798 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:47 smithi186 podman[68825]: 2024-04-21 09:19:47.483983705 +0000 UTC m=+5.597098666 container died c3317b071730083267eb4cb5b73ac0522a61b5fce524ab1d0b641f9890263491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-21T09:19:48.088 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:47 smithi186 podman[68825]: 2024-04-21 09:19:47.897338981 +0000 UTC m=+6.010453940 container cleanup c3317b071730083267eb4cb5b73ac0522a61b5fce524ab1d0b641f9890263491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-21T09:19:48.088 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:47 smithi186 bash[68825]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2 2024-04-21T09:19:48.566 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:48 smithi186 podman[68844]: 2024-04-21 09:19:48.12395158 +0000 UTC m=+0.633048517 container remove c3317b071730083267eb4cb5b73ac0522a61b5fce524ab1d0b641f9890263491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T09:19:48.837 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:48 smithi186 podman[68939]: 2024-04-21 09:19:48.47077505 +0000 UTC m=+0.018124239 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:19:48.838 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:48 smithi186 podman[68939]: 2024-04-21 09:19:48.608467546 +0000 UTC m=+0.155816720 container create 680cb9cb6069e9ed957d7a7ee54b1fe3e142619a52c0279a8722b603a5bcd20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-21T09:19:49.178 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:48 smithi186 podman[68939]: 2024-04-21 09:19:48.878345228 +0000 UTC m=+0.425694402 container init 680cb9cb6069e9ed957d7a7ee54b1fe3e142619a52c0279a8722b603a5bcd20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T09:19:49.178 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:48 smithi186 podman[68939]: 2024-04-21 09:19:48.88158247 +0000 UTC m=+0.428931644 container start 680cb9cb6069e9ed957d7a7ee54b1fe3e142619a52c0279a8722b603a5bcd20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-21T09:19:49.178 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:48 smithi186 podman[68939]: 2024-04-21 09:19:48.978459274 +0000 UTC m=+0.525808475 container attach 680cb9cb6069e9ed957d7a7ee54b1fe3e142619a52c0279a8722b603a5bcd20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:19:49.455 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 21 09:19:49 smithi186 podman[68939]: 2024-04-21 09:19:49.179390761 +0000 UTC m=+0.726739949 container died 680cb9cb6069e9ed957d7a7ee54b1fe3e142619a52c0279a8722b603a5bcd20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, ceph=True) 2024-04-21T09:19:49.816 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.2.service' 2024-04-21T09:19:49.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:49.855 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-21T09:19:49.856 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-21T09:19:49.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.3 2024-04-21T09:19:50.175 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:49 smithi066 systemd[1]: Stopping Ceph osd.3 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:50.620 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:50 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:19:50.174+0000 7f0e32e02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:50.620 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:50 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:19:50.174+0000 7f0e32e02700 -1 osd.3 110 *** Got signal Terminated *** 2024-04-21T09:19:50.620 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:50 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:19:50.174+0000 7f0e32e02700 -1 osd.3 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:19:52.110 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:51.914+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:52.119 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:51 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:19:51.792+0000 7f0e2f3f6700 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:25.981701+0000 front 2024-04-21T09:19:25.981722+0000 (oldest deadline 2024-04-21T09:19:51.281944+0000) 2024-04-21T09:19:52.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:52.278+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:19:53.049 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:52 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:19:52.765+0000 7f0e2f3f6700 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:25.981701+0000 front 2024-04-21T09:19:25.981722+0000 (oldest deadline 2024-04-21T09:19:51.281944+0000) 2024-04-21T09:19:53.359 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:52 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:52.953+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:53.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:53 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:53.050+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:19:53.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:53.257+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:19:54.071 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:53 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:19:53.725+0000 7f0e2f3f6700 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:25.981701+0000 front 2024-04-21T09:19:25.981722+0000 (oldest deadline 2024-04-21T09:19:51.281944+0000) 2024-04-21T09:19:54.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:53 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:53.947+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:54.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:54 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:54.072+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:19:54.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:54.244+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:19:55.050 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:54 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3[47339]: 2024-04-21T09:19:54.678+0000 7f0e2f3f6700 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:25.981701+0000 front 2024-04-21T09:19:25.981722+0000 (oldest deadline 2024-04-21T09:19:51.281944+0000) 2024-04-21T09:19:55.359 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:54 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:54.988+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:55.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:55 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:55.051+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:19:55.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:55.244+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:19:55.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:19:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:19:55.139+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:19:55.877 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:55 smithi066 podman[96392]: 2024-04-21 09:19:55.60809379 +0000 UTC m=+5.549110957 container died 0aba80cbf42b9b56111de1e687c015a7f91de0f67ace3d8d914e3b34737e6466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:19:56.242 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:56 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:56.021+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:19:56.243 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:55 smithi066 podman[96392]: 2024-04-21 09:19:55.991394546 +0000 UTC m=+5.932411721 container cleanup 0aba80cbf42b9b56111de1e687c015a7f91de0f67ace3d8d914e3b34737e6466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T09:19:56.243 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:55 smithi066 bash[96392]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3 2024-04-21T09:19:56.359 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:55 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:55.949+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:56.544 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:56 smithi066 podman[96409]: 2024-04-21 09:19:56.242906913 +0000 UTC m=+0.629398212 container remove 0aba80cbf42b9b56111de1e687c015a7f91de0f67ace3d8d914e3b34737e6466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-21T09:19:56.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:56.238+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:19:56.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:19:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:19:56.091+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:19:56.870 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:56 smithi066 podman[96500]: 2024-04-21 09:19:56.446317441 +0000 UTC m=+0.018576574 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:19:56.870 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:56 smithi066 podman[96500]: 2024-04-21 09:19:56.568728657 +0000 UTC m=+0.140987824 container create 9d0fc79b49e0420867b96235f0132949583035c09eca354020602468dbed14a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-21T09:19:56.870 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:56 smithi066 podman[96500]: 2024-04-21 09:19:56.779648557 +0000 UTC m=+0.351907676 container init 9d0fc79b49e0420867b96235f0132949583035c09eca354020602468dbed14a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:19:56.870 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:56 smithi066 podman[96500]: 2024-04-21 09:19:56.782872435 +0000 UTC m=+0.355131554 container start 9d0fc79b49e0420867b96235f0132949583035c09eca354020602468dbed14a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:19:57.359 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:56 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:56.955+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:57.364 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:56 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:56.994+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:19:57.364 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:56 smithi066 podman[96500]: 2024-04-21 09:19:56.871132922 +0000 UTC m=+0.443392070 container attach 9d0fc79b49e0420867b96235f0132949583035c09eca354020602468dbed14a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-21T09:19:57.364 INFO:journalctl@ceph.osd.3.smithi066.stdout:Apr 21 09:19:57 smithi066 podman[96500]: 2024-04-21 09:19:57.08613296 +0000 UTC m=+0.658392087 container died 9d0fc79b49e0420867b96235f0132949583035c09eca354020602468dbed14a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux ) 2024-04-21T09:19:57.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:57.214+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:19:57.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:19:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:19:57.113+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:19:57.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.3.service' 2024-04-21T09:19:57.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:19:57.808 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-21T09:19:57.808 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-21T09:19:57.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.4 2024-04-21T09:19:58.110 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:19:57 smithi083 systemd[1]: Stopping Ceph osd.4 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:19:58.110 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:58 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:58.002+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:58.334 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:58 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:58.025+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:19:58.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:19:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:19:58.094+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:19:58.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:58.253+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:19:58.610 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:19:58 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4[42993]: 2024-04-21T09:19:58.152+0000 7f47f2fab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:19:58.610 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:19:58 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4[42993]: 2024-04-21T09:19:58.152+0000 7f47f2fab700 -1 osd.4 110 *** Got signal Terminated *** 2024-04-21T09:19:58.611 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:19:58 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4[42993]: 2024-04-21T09:19:58.152+0000 7f47f2fab700 -1 osd.4 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:19:59.359 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:19:59 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:19:59.028+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:19:59.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:59 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:59.002+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:19:59.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:19:59 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:19:59.098+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:19:59.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:19:59 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:19:59.220+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:00.235 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:00 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:00.101+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:00.359 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:00 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:00.013+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:00.360 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:00 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:00.077+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:00.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:19:59 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:19:59.982+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:00.485 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:00 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:00.235+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:01.272 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:01 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:01.012+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:01.360 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:01 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:01.099+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:01.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:01 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:01.063+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:01.590 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:01 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:01.191+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:01.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:01 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:01.097+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:02.318 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:02 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:02.041+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:02.319 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:02 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:02.145+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:02.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:02 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:02.004+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:02.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:02 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:02.096+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:02.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:02 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:02.096+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:02.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:02 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:02.169+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:02.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:02 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:02.169+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:02.609 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:02 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4[42993]: 2024-04-21T09:20:02.318+0000 7f47ef59f700 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.192560+0000 front 2024-04-21T09:19:36.192526+0000 (oldest deadline 2024-04-21T09:20:02.092661+0000) 2024-04-21T09:20:02.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:02 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:02.396+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:03.337 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:03 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:03.080+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:03.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:03 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:03.080+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:03.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:03 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:03.132+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:03.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:03 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:03.132+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:03.352 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:03 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:03.043+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:03.352 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:03 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:03.043+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:03.359 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:03 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:03.104+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:03.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:03 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:03.063+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:03.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:03 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:03.063+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:03.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:03 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:03.352+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:03.828 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:03 smithi083 podman[68288]: 2024-04-21 09:20:03.526519899 +0000 UTC m=+5.521110126 container died b5417f2d446a2f3842d86f3669abc003dde8c2f7dbeb922b57bdcdf110622a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T09:20:04.109 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:03 smithi083 podman[68288]: 2024-04-21 09:20:03.936627804 +0000 UTC m=+5.931218015 container cleanup b5417f2d446a2f3842d86f3669abc003dde8c2f7dbeb922b57bdcdf110622a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T09:20:04.110 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:03 smithi083 bash[68288]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4 2024-04-21T09:20:04.110 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:04 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:04.064+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:04.110 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:04 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:04.064+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:04.337 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:04.031+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:04.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:04.122+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:04.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:04.122+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:04.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:04.132+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:04.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:04 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:04.132+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:04.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:04.082+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:04.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:04.082+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:04.499 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:04 smithi083 podman[68306]: 2024-04-21 09:20:04.187975017 +0000 UTC m=+0.657824699 container remove b5417f2d446a2f3842d86f3669abc003dde8c2f7dbeb922b57bdcdf110622a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T09:20:04.499 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:04 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:04.129+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:04.815 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:04 smithi083 podman[68400]: 2024-04-21 09:20:04.4032558 +0000 UTC m=+0.018195885 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:20:04.815 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:04 smithi083 podman[68400]: 2024-04-21 09:20:04.555552367 +0000 UTC m=+0.170492442 container create f9bdb417dbee8a70a170baf64d9a1cfb224ffdad53cbfbc8ba3a71e73af7b75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD) 2024-04-21T09:20:04.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:04 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:04.381+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:05.093 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:04 smithi083 podman[68400]: 2024-04-21 09:20:04.816546428 +0000 UTC m=+0.431486501 container init f9bdb417dbee8a70a170baf64d9a1cfb224ffdad53cbfbc8ba3a71e73af7b75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-21T09:20:05.093 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:04 smithi083 podman[68400]: 2024-04-21 09:20:04.819713118 +0000 UTC m=+0.434653197 container start f9bdb417dbee8a70a170baf64d9a1cfb224ffdad53cbfbc8ba3a71e73af7b75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:20:05.093 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:04 smithi083 podman[68400]: 2024-04-21 09:20:04.916715706 +0000 UTC m=+0.531655797 container attach f9bdb417dbee8a70a170baf64d9a1cfb224ffdad53cbfbc8ba3a71e73af7b75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-21T09:20:05.337 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:05.011+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:05.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:05.166+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:05.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:05.166+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:05.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:05.174+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:05.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:05.174+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:05.359 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 21 09:20:05 smithi083 podman[68400]: 2024-04-21 09:20:05.122084196 +0000 UTC m=+0.737024275 container died f9bdb417dbee8a70a170baf64d9a1cfb224ffdad53cbfbc8ba3a71e73af7b75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:20:05.360 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:05 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:05.171+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:05.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:05 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:05.092+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:05.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:05 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:05.092+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:05.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:05.092+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:05.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:05.092+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:05.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.4.service' 2024-04-21T09:20:05.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:20:05.838 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-21T09:20:05.839 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-21T09:20:05.839 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.5 2024-04-21T09:20:05.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:05 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:05.397+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:06.121 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:05 smithi186 systemd[1]: Stopping Ceph osd.5 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:20:06.121 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:05 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:05.983+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:06.359 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:06 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:06.195+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:06.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:06 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:06.043+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:06.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:06 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:06.043+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:06.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:06 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:06.079+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:06.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:06 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:06.079+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:06.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:06.121+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:06.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:06.121+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:06.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:06.185+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:06.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:06.185+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:06.588 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:06.161+0000 7f0177b98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:20:06.588 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:06.161+0000 7f0177b98700 -1 osd.5 110 *** Got signal Terminated *** 2024-04-21T09:20:06.588 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:06.161+0000 7f0177b98700 -1 osd.5 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:20:06.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:06 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:06.394+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:07.337 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:06 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:06.968+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:07.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:07.156+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:07.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:07.156+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:07.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:07.155+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:07.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:07.155+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:07.359 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:07 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:07.219+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:07.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:07 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:07.009+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:07.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:07 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:07.009+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:07.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:07.074+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:07.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:07.074+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:07.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:07.440+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:08.338 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:07 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:07.933+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:08.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:08.182+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:08.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:08.182+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:08.339 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:08.171+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:08.339 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:08.171+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:08.360 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:08 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:08.248+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:08.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:08 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:08.037+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:08.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:08 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:08.037+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:08.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:08.027+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:08.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:08.027+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:08.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:08.027+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:08.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:08 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:08.415+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:09.176 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:08.920+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:09.176 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:08 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:08.920+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:43.973585+0000 front 2024-04-21T09:19:43.973457+0000 (oldest deadline 2024-04-21T09:20:08.673627+0000) 2024-04-21T09:20:09.282 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:09.027+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:09.283 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:09.027+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:09.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:09.068+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:09.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:09.068+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:09.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:09.068+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:09.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:09.228+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:09.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:09.228+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:09.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:09.175+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:09.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:09.175+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:09.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:09.282+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:09.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:09.282+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:09.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:09 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:09.392+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:10.195 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:09.943+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:10.196 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:09 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:09.943+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:43.973585+0000 front 2024-04-21T09:19:43.973457+0000 (oldest deadline 2024-04-21T09:20:08.673627+0000) 2024-04-21T09:20:10.245 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:09.987+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:10.245 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:09 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:09.987+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:10.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:10.074+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:10.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:10.074+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:10.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:10.074+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:10.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:10.213+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:10.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:10.213+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:10.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:10.196+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:10.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:10.196+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:10.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:10.196+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:10.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:10.245+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:10.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:10.245+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:10.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:10 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:10.418+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:11.282 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:10.995+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:11.282 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:10 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:10.995+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:11.327 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:11.061+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:11.328 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:11.061+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:11.328 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:11.061+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:11.337 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:11 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:11.198+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:11.338 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:11 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:11.198+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:11.338 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:10.960+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.073147+0000 front 2024-04-21T09:19:38.073027+0000 (oldest deadline 2024-04-21T09:20:03.972970+0000) 2024-04-21T09:20:11.338 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:10 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5[43614]: 2024-04-21T09:20:10.960+0000 7f0170184700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:43.973585+0000 front 2024-04-21T09:19:43.973457+0000 (oldest deadline 2024-04-21T09:20:08.673627+0000) 2024-04-21T09:20:11.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:11 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:11.203+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:11.338 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:11 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:11.203+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:11.339 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:11 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:11.203+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:11.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:11 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:11.282+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:11.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:11 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:11.282+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:11.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:11.394+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:11.620 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:11.394+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:11.904 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:11 smithi186 podman[69060]: 2024-04-21 09:20:11.603921929 +0000 UTC m=+5.567488148 container died 86ffb474f4afe63d1a369fd8d1beb5d788ed8f44a393665f9428e34bcee2fed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-21T09:20:12.210 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:12 smithi186 podman[69060]: 2024-04-21 09:20:12.002857348 +0000 UTC m=+5.966423562 container cleanup 86ffb474f4afe63d1a369fd8d1beb5d788ed8f44a393665f9428e34bcee2fed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T09:20:12.210 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:12 smithi186 bash[69060]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5 2024-04-21T09:20:12.274 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:12 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:12.022+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:12.274 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:12 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:12.022+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:12.274 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:12 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:12.022+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:12.362 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:12.103+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:12.362 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:12.103+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:12.362 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:12.103+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:12.588 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:12 smithi186 podman[69079]: 2024-04-21 09:20:12.229143697 +0000 UTC m=+0.619856860 container remove 86ffb474f4afe63d1a369fd8d1beb5d788ed8f44a393665f9428e34bcee2fed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T09:20:12.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:12 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:12.246+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:12.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:12 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:12.246+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:12.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:12 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:12.246+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:12.589 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:12 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:12.210+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:12.589 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:12 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:12.210+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:12.589 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:12 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:12.210+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:12.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:12 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:12.274+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:12.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:12 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:12.274+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:12.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:12.363+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:12.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:12 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:12.363+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:12.932 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:12 smithi186 podman[69167]: 2024-04-21 09:20:12.547945572 +0000 UTC m=+0.017856419 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:20:12.932 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:12 smithi186 podman[69167]: 2024-04-21 09:20:12.663812779 +0000 UTC m=+0.133723671 container create 23444003347ddb9490c71395b2368e1505792d4ece56195eed6c2df543c4bf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T09:20:12.933 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:12 smithi186 podman[69167]: 2024-04-21 09:20:12.933353129 +0000 UTC m=+0.403263958 container init 23444003347ddb9490c71395b2368e1505792d4ece56195eed6c2df543c4bf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T09:20:13.229 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:12 smithi186 podman[69167]: 2024-04-21 09:20:12.937141676 +0000 UTC m=+0.407052512 container start 23444003347ddb9490c71395b2368e1505792d4ece56195eed6c2df543c4bf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T09:20:13.229 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:13 smithi186 podman[69167]: 2024-04-21 09:20:13.033427908 +0000 UTC m=+0.503338749 container attach 23444003347ddb9490c71395b2368e1505792d4ece56195eed6c2df543c4bf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:20:13.359 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:13.235+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:13.359 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:13.235+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:13.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:13.067+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:13.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:13.067+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:13.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:13 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:13.067+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:13.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:13.092+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:13.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:13.092+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:13.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:13.092+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:13.585 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 21 09:20:13 smithi186 podman[69167]: 2024-04-21 09:20:13.230161206 +0000 UTC m=+0.700072041 container died 23444003347ddb9490c71395b2368e1505792d4ece56195eed6c2df543c4bf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-21T09:20:13.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:13.242+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:13.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:13.242+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:13.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:13.242+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:13.586 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:13.229+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:13.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:13.230+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:13.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:13 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:13.230+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:13.870 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:13.379+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:13.870 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:13 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:13.379+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:13.926 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.5.service' 2024-04-21T09:20:13.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:20:13.969 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-21T09:20:13.969 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-21T09:20:13.969 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.6 2024-04-21T09:20:14.286 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:14 smithi066 systemd[1]: Stopping Ceph osd.6 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:20:14.287 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:14.056+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:14.287 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:14.056+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:14.287 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:14.056+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:14.360 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:14 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:14.261+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:14.360 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:14 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:14.261+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:14.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:14 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:14.109+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:14.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:14 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:14.109+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:14.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:14 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:14.109+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:14.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:14 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:14.230+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:14.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:14 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:14.230+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:14.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:14 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:14.230+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:14.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:14 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:14.263+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:14.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:14 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:14.263+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:14.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:14 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:14.263+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:14.620 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:14.287+0000 7f3b7e83a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:20:14.621 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:14.287+0000 7f3b7e83a700 -1 osd.6 110 *** Got signal Terminated *** 2024-04-21T09:20:14.621 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:14.287+0000 7f3b7e83a700 -1 osd.6 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:20:14.621 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:14.388+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:14.621 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:14 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:14.388+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:15.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:15 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:15.025+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:15.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:15 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:15.025+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:15.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:15 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:15.025+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:15.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:15 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:15.271+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:15.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:15 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:15.271+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:15.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:15 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:15.271+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:15.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:15 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:15.275+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:15.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:15 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:15.275+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:15.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:15 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:15.275+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:15.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:15 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:15.275+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:15.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:15 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:15.275+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:15.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:15 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:15.143+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:15.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:15 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:15.143+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:15.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:15 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:15.143+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:15.869 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:15 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:15.373+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:15.870 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:15 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:15.373+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:16.360 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:16.046+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:16.360 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:16.046+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:16.360 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:16.046+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:16.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:16 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:16.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:16.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:16 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:16.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:16.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:16 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:16.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:16.590 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:16 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:16.258+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:16.590 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:16 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:16.258+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:16.590 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:16 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:16.258+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:16.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:16 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:16.259+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:16.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:16 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:16.259+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:16.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:16 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:16.136+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:16.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:16 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:16.136+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:16.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:16 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:16.136+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:16.620 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:16.361+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:16.620 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:16.361+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:17.325 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:17 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:17.062+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:17.326 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:17 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:17.062+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:17.326 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:17 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:17.062+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:17.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:17 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:17.260+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:17.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:17 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:17.261+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:17.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:17 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:17.261+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:17.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:17 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:17.220+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:17.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:17 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:17.220+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:17.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:17 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:17.220+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:17.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:17 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:17.220+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:17.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:17 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:17.269+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:17.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:17 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:17.269+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:17.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:17 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:17.171+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:17.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:17 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:17.171+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:17.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:17 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:17.171+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:17.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:17 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:17.171+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:17.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:17 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:17.326+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:17.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:17 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:17.326+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:18.314 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:18 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:18.031+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:18.314 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:18 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:18.031+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:18.314 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:18 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:18.031+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:18.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:18 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:18.267+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:18.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:18 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:18.267+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:18.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:18 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:18.267+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:18.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:18 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:18.217+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:18.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:18 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:18.217+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:18.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:18 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:18.217+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:18.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:18 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:18.217+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:18.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:18 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:18.309+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:18.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:18 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:18.309+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:18.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:18 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:18.217+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:18.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:18 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:18.217+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:18.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:18 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:18.217+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:18.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:18 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:18.217+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:18.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:18 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:18.315+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:18.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:18 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:18.315+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:19.276 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:19 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:19.013+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:19.276 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:19 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:19.013+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:19.276 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:19 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:19.013+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:19.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:19 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:19.285+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:19.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:19 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:19.285+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:19.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:19 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:19.285+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:19.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:19 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:19.253+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:19.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:19 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:19.253+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:19.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:19 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:19.253+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:19.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:19 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:19.253+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:19.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:19.320+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:19.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:19.320+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:19.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:19.320+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:19.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:19.237+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:19.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:19.237+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:19.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:19.237+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:19.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:19 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:19.237+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:19.619 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:19 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:19.276+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:35.833793+0000 front 2024-04-21T09:19:35.833746+0000 (oldest deadline 2024-04-21T09:20:01.733754+0000) 2024-04-21T09:20:19.620 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:19 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:19.276+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:45.835219+0000 front 2024-04-21T09:19:45.835032+0000 (oldest deadline 2024-04-21T09:20:11.135050+0000) 2024-04-21T09:20:19.620 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:19 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6[53912]: 2024-04-21T09:20:19.276+0000 7f3b7ae2e700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:54.536844+0000 front 2024-04-21T09:19:54.536817+0000 (oldest deadline 2024-04-21T09:20:19.237149+0000) 2024-04-21T09:20:20.087 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:19 smithi066 podman[96618]: 2024-04-21 09:20:19.781496906 +0000 UTC m=+5.607006900 container died 3c8242b5e539c52c7f58d2db22fd397ea0800e9927b48b2e218040579a790be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T09:20:20.087 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:20 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:20.001+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:20.087 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:20 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:20.001+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:20.087 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:20 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:20.001+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:20.087 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:20 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:20.001+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:20.369 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:20 smithi066 podman[96618]: 2024-04-21 09:20:20.186724408 +0000 UTC m=+6.012234422 container cleanup 3c8242b5e539c52c7f58d2db22fd397ea0800e9927b48b2e218040579a790be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T09:20:20.370 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:20 smithi066 bash[96618]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6 2024-04-21T09:20:20.489 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:20.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:20.489 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:20.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:20.489 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:20.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:20.489 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:20.234+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:20.489 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:20.234+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:20.489 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:20.234+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:20.490 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:20 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:20.234+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:20.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:20.351+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:20.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:20.351+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:20.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:20.351+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:20.611 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:20.282+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:20.611 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:20.282+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:20.611 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:20.282+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:20.611 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:20.282+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:20.761 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:20 smithi066 podman[96635]: 2024-04-21 09:20:20.438280334 +0000 UTC m=+0.650459128 container remove 3c8242b5e539c52c7f58d2db22fd397ea0800e9927b48b2e218040579a790be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.010+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.010+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.010+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.010+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:20 smithi066 podman[96726]: 2024-04-21 09:20:20.665494089 +0000 UTC m=+0.018095186 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:20 smithi066 podman[96726]: 2024-04-21 09:20:20.772399801 +0000 UTC m=+0.125000880 container create a809dd7f2ed81469ba5b8cde729e2489c40faaf8d725d30470205d8c0b579219 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:21 smithi066 podman[96726]: 2024-04-21 09:20:21.008585989 +0000 UTC m=+0.361187083 container init a809dd7f2ed81469ba5b8cde729e2489c40faaf8d725d30470205d8c0b579219 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-21T09:20:21.082 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:21 smithi066 podman[96726]: 2024-04-21 09:20:21.011975664 +0000 UTC m=+0.364576751 container start a809dd7f2ed81469ba5b8cde729e2489c40faaf8d725d30470205d8c0b579219 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-21T09:20:21.083 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:21 smithi066 podman[96726]: 2024-04-21 09:20:21.082947472 +0000 UTC m=+0.435548559 container attach a809dd7f2ed81469ba5b8cde729e2489c40faaf8d725d30470205d8c0b579219 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:20:21.369 INFO:journalctl@ceph.osd.6.smithi066.stdout:Apr 21 09:20:21 smithi066 podman[96726]: 2024-04-21 09:20:21.319135347 +0000 UTC m=+0.671736426 container died a809dd7f2ed81469ba5b8cde729e2489c40faaf8d725d30470205d8c0b579219 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T09:20:21.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:21 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:21.301+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:21.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:21 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:21.301+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:21.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:21 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:21.301+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:21.589 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:21 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:21.268+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:21.589 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:21 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:21.268+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:21.589 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:21 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:21.268+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:21.589 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:21 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:21.268+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:21.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:21.330+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:21.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:21.330+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:21.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:21.330+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:21.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:21.330+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:21.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:21.330+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:21.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:21.330+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:21.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:21 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:21.330+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:21.937 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.6.service' 2024-04-21T09:20:21.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:20:21.989 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-21T09:20:21.989 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-21T09:20:21.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.7 2024-04-21T09:20:22.314 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:22 smithi083 systemd[1]: Stopping Ceph osd.7 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:20:22.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.974+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:22.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.974+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:22.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.974+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:22.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:21 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:21.974+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:22.571 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:22.351+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:22.351+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:22.351+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:22.351+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:22.314+0000 7f8a3490b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:22.314+0000 7f8a3490b700 -1 osd.7 110 *** Got signal Terminated *** 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:22.314+0000 7f8a3490b700 -1 osd.7 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:22.343+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:22.572 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:22.343+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:22.573 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:22.343+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:22.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:22 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:22.304+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:22.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:22 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:22.304+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:22.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:22 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:22.304+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:22.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:22 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:22.304+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:22.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:22 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:22.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:22.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:22 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:22.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:22.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:22 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:22.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:23.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:22 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:22.949+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:23.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:22 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:22.949+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:23.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:22 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:22.949+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:23.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:22 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:22.949+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:23.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:23.319+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:23.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:23.319+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:23.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:23.319+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:23.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:23.259+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:23.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:23.259+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:23.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:23.259+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:23.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:23 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:23.259+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:23.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:23 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:23.321+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:23.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:23 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:23.321+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:23.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:23 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:23.321+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:23.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:23 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:23.324+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:23.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:23 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:23.324+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:23.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:23 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:23.324+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:23.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:23 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:23.324+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:24.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:23 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:23.987+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:24.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:23 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:23.987+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:24.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:23 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:23.987+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:24.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:23 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:23.987+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:24.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:24 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:24.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:24.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:24 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:24.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:24.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:24 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:24.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:24.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:24 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:24.242+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:24.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:24 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:24.242+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:24.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:24 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:24.242+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:24.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:24 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:24.242+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:24.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:24 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:24.337+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:24.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:24 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:24.337+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:24.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:24 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:24.337+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:24.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:24 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:24.327+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:24.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:24 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:24.327+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:24.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:24 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:24.327+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:24.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:24 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:24.327+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:25.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:24 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:24.964+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:25.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:24 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:24.964+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:25.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:24 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:24.964+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:25.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:24 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:24.964+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:25.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:25 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:25.316+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:25.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:25 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:25.316+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:25.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:25 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:25.316+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:25.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:25 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:25.267+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:25.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:25 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:25.267+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:25.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:25 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:25.267+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:25.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:25 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:25.267+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:25.339+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:25.339+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:25.339+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:25.279+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:25.279+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:25.279+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:25.279+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:25.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:25 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:25.279+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:26.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:25 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:25.929+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:26.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:25 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:25.929+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:26.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:25 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:25.929+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:26.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:25 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:25.929+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:26.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:26.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:26.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:26.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:26.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:26.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:26.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:26.238+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:26.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:26.238+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:26.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:26.238+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:26.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:26 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:26.238+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:26.609 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:26.383+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.835690+0000 front 2024-04-21T09:19:38.835645+0000 (oldest deadline 2024-04-21T09:19:59.935787+0000) 2024-04-21T09:20:26.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:26.383+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:44.537276+0000 front 2024-04-21T09:19:44.537365+0000 (oldest deadline 2024-04-21T09:20:08.637201+0000) 2024-04-21T09:20:26.610 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7[49851]: 2024-04-21T09:20:26.383+0000 7f8a2cef7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.638756+0000 front 2024-04-21T09:19:52.638799+0000 (oldest deadline 2024-04-21T09:20:18.538656+0000) 2024-04-21T09:20:26.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:26.294+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:26.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:26.294+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:26.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:26.294+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:26.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:26.294+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:26.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:26.294+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:27.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:26 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:26.913+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:27.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:26 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:26.913+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:27.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:26 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:26.913+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:27.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:26 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:26.913+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:27.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:27.286+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:27.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:27.286+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:27.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:27.286+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:27.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:27.286+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:27.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:27.378+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:27.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:27.378+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:27.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:27.379+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:27.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:27 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:27.379+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:27.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:27 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:27.296+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:27.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:27 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:27.296+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:27.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:27 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:27.296+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:27.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:27 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:27.296+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:27.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:27 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:27.296+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:28.109 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:27 smithi083 podman[68505]: 2024-04-21 09:20:27.851730347 +0000 UTC m=+5.667546006 container died 4fc4388edaf6fee2d1082eb811067ebb9e26b95b24bafb77ae72c3d089f009d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-21T09:20:28.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:27 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:27.939+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:28.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:27 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:27.939+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:28.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:27 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:27.939+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:28.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:27 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:27.939+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:28.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:27 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:27.939+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:28.523 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:28 smithi083 podman[68505]: 2024-04-21 09:20:28.2568629 +0000 UTC m=+6.072678545 container cleanup 4fc4388edaf6fee2d1082eb811067ebb9e26b95b24bafb77ae72c3d089f009d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-21T09:20:28.523 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:28 smithi083 bash[68505]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7 2024-04-21T09:20:28.524 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:28 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:28.281+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:28.524 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:28 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:28.281+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:28.524 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:28 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:28.281+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:28.524 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:28 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:28.281+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:28.524 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:28 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:28.281+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:28.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:28.236+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:28.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:28.236+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:28.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:28.236+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:28.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:28.236+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:28.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:28.332+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:28.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:28.332+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:28.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:28.332+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:28.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:28 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:28.332+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:28.859 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:28 smithi083 podman[68533]: 2024-04-21 09:20:28.524570573 +0000 UTC m=+0.668673969 container remove 4fc4388edaf6fee2d1082eb811067ebb9e26b95b24bafb77ae72c3d089f009d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:20:29.127 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:28 smithi083 podman[68627]: 2024-04-21 09:20:28.774786786 +0000 UTC m=+0.017707012 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:20:29.127 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:28 smithi083 podman[68627]: 2024-04-21 09:20:28.883990771 +0000 UTC m=+0.126911034 container create 218610a8994978edf0d670c3889b9350b8dbec9bf896861f9fc6d68d867a907a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-21T09:20:29.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:28 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:28.917+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:29.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:28 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:28.917+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:29.371 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:28 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:28.917+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:29.371 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:28 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:28.917+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:29.371 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:28 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:28.917+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:29.426 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:29 smithi083 podman[68627]: 2024-04-21 09:20:29.128086676 +0000 UTC m=+0.371006901 container init 218610a8994978edf0d670c3889b9350b8dbec9bf896861f9fc6d68d867a907a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-21T09:20:29.427 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:29 smithi083 podman[68627]: 2024-04-21 09:20:29.131729161 +0000 UTC m=+0.374649388 container start 218610a8994978edf0d670c3889b9350b8dbec9bf896861f9fc6d68d867a907a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:20:29.427 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:29 smithi083 podman[68627]: 2024-04-21 09:20:29.236241283 +0000 UTC m=+0.479161516 container attach 218610a8994978edf0d670c3889b9350b8dbec9bf896861f9fc6d68d867a907a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD) 2024-04-21T09:20:29.427 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:29.322+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:29.427 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:29.322+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:29.427 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:29.322+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:29.427 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:29.322+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:29.427 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:29 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:29.322+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:29.258+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:29.258+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:29.258+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:29.258+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:29.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:29.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:29.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:29.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:29 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:29.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:29.722 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 21 09:20:29 smithi083 podman[68627]: 2024-04-21 09:20:29.42745253 +0000 UTC m=+0.670372739 container died 218610a8994978edf0d670c3889b9350b8dbec9bf896861f9fc6d68d867a907a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-21T09:20:30.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.7.service' 2024-04-21T09:20:30.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:20:30.115 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-21T09:20:30.116 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-21T09:20:30.116 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.8 2024-04-21T09:20:30.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:29 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:29.882+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:30.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:29 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:29.882+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:30.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:29 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:29.882+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:30.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:29 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:29.882+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:30.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:29 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:29.882+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:30.404 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 systemd[1]: Stopping Ceph osd.8 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:20:30.404 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:30.299+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:30.404 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:30.299+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:30.405 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:30.299+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:30.405 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:30.299+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:30.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:30 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:30.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:30.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:30 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:30.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:30.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:30 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:30.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:30.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:30 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:30.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:30.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:30 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:30.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:30.837 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:30.639+0000 7fc7e93e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:20:30.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:30.639+0000 7fc7e93e9700 -1 osd.8 110 *** Got signal Terminated *** 2024-04-21T09:20:30.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:30.639+0000 7fc7e93e9700 -1 osd.8 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:20:30.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:30.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:30.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:30.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:30.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:30.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:30.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:30 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:30.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:31.271 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:30 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:30.931+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:31.271 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:30 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:30.931+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:31.271 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:30 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:30.931+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:31.271 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:30 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:30.931+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:31.271 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:30 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:30.931+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:31.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:31.284+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:31.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:31.284+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:31.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:31.284+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:31.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:31.284+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:31.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:31.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:31.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:31.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:31.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:31.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:31.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:31 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:31.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:31.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:31 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:31.338+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:31.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:31 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:31.338+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:31.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:31 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:31.338+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:31.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:31 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:31.338+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:31.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:31 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:31.338+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:32.287 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:31.963+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:32.287 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:31.963+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:32.287 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:31.963+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:32.288 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:31.963+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:32.288 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:31 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:31.963+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:32.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:32.327+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:32.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:32.327+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:32.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:32.327+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:32.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:32.327+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:32.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:32.385+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:32.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:32.385+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:32.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:32.385+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:32.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:32 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:32.385+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:32.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:32 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:32.339+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:32.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:32 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:32.339+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:32.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:32 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:32.339+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:32.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:32 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:32.339+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:32.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:32 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:32.339+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:33.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:32 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:32.975+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:33.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:32 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:32.975+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:33.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:32 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:32.975+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:33.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:32 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:32.975+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:33.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:32 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:32.975+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:33.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:33.366+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:33.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:33.366+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:33.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:33.366+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:33.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:33.366+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:33.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:33.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:33.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:33.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:33.839 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:33.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:33.839 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:33.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:33.839 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:33.403+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:33.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:33.384+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:33.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:33.384+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:33.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:33.384+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:33.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:33.384+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:33.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:33 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:33.384+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:34.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:33.960+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:34.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:33.960+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:34.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:33.960+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:34.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:33.960+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:34.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:33 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:33.960+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:34.837 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:34.360+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:34.360+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:34.360+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:34.360+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:34.432+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:34.432+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:34.432+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:34.432+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:34.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:34 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:34.432+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:34.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:34 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:34.370+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:34.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:34 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:34.370+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:34.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:34 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:34.370+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:34.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:34 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:34.370+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:34.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:34 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:34.370+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:35.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:34.928+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:35.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:34.928+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:35.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:34.928+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:35.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:34.928+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:35.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:34 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:34.928+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:35.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:35.394+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:35.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:35.394+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:35.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:35.394+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:35.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:35.394+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:35.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:35.394+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:35.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:35.372+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:30.422600+0000 front 2024-04-21T09:19:30.422793+0000 (oldest deadline 2024-04-21T09:19:55.122606+0000) 2024-04-21T09:20:35.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:35.372+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.223647+0000 front 2024-04-21T09:19:36.223734+0000 (oldest deadline 2024-04-21T09:20:01.424311+0000) 2024-04-21T09:20:35.839 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:35.372+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.625946+0000 front 2024-04-21T09:19:46.626058+0000 (oldest deadline 2024-04-21T09:20:09.526107+0000) 2024-04-21T09:20:35.839 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:35.372+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:52.327830+0000 front 2024-04-21T09:19:52.327809+0000 (oldest deadline 2024-04-21T09:20:16.427774+0000) 2024-04-21T09:20:35.839 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:35 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8[50297]: 2024-04-21T09:20:35.372+0000 7fc7e19d5700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:10.430006+0000 front 2024-04-21T09:20:10.430036+0000 (oldest deadline 2024-04-21T09:20:34.530246+0000) 2024-04-21T09:20:35.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:35 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:35.412+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:35.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:35 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:35.412+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:35.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:35 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:35.412+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:35.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:35 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:35.412+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:35.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:35 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:35.412+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:36.273 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:35 smithi186 podman[69285]: 2024-04-21 09:20:35.99600543 +0000 UTC m=+5.685107773 container died f0c5a6450c2de69bcdfb169b12ff18e617ef15d38c8eb7051d1cfbd7fd38e46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:20:36.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:35.894+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:36.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:35.894+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:36.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:35.894+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:36.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:35.894+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:36.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:35.894+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:36.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:35 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:35.894+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:36.587 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:36 smithi186 podman[69285]: 2024-04-21 09:20:36.407231531 +0000 UTC m=+6.096333874 container cleanup f0c5a6450c2de69bcdfb169b12ff18e617ef15d38c8eb7051d1cfbd7fd38e46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T09:20:36.588 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:36 smithi186 bash[69285]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8 2024-04-21T09:20:36.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:36 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:36.355+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:36.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:36 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:36.355+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:36.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:36 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:36.355+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:36.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:36 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:36.355+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:36.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:36 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:36.355+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:36.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:36 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:36.437+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:36.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:36 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:36.437+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:36.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:36 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:36.437+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:36.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:36 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:36.437+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:36.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:36 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:36.437+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:36.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:36 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:36.437+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:36.978 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:36 smithi186 podman[69301]: 2024-04-21 09:20:36.650207677 +0000 UTC m=+0.649710978 container remove f0c5a6450c2de69bcdfb169b12ff18e617ef15d38c8eb7051d1cfbd7fd38e46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:20:37.244 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:36 smithi186 podman[69393]: 2024-04-21 09:20:36.882543151 +0000 UTC m=+0.017945834 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:20:37.244 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:36 smithi186 podman[69393]: 2024-04-21 09:20:36.984285964 +0000 UTC m=+0.119688667 container create b88f7882456ae1b284ee8bafed465b2bcc19fa5e8c08c91ca0a26271ada25b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-21T09:20:37.244 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:37 smithi186 podman[69393]: 2024-04-21 09:20:37.245355624 +0000 UTC m=+0.380758290 container init b88f7882456ae1b284ee8bafed465b2bcc19fa5e8c08c91ca0a26271ada25b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-21T09:20:37.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:36.916+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:37.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:36.916+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:37.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:36.916+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:37.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:36.916+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:37.371 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:36.916+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:37.371 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:36 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:36.916+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:37.548 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:37 smithi186 podman[69393]: 2024-04-21 09:20:37.248681168 +0000 UTC m=+0.384083829 container start b88f7882456ae1b284ee8bafed465b2bcc19fa5e8c08c91ca0a26271ada25b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-21T09:20:37.548 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:37 smithi186 podman[69393]: 2024-04-21 09:20:37.337240908 +0000 UTC m=+0.472643584 container attach b88f7882456ae1b284ee8bafed465b2bcc19fa5e8c08c91ca0a26271ada25b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T09:20:37.548 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:37 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:37.338+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:37.548 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:37 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:37.338+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:37.548 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:37 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:37.338+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:37.548 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:37 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:37.338+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:37.548 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:37 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:37.338+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:37.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 21 09:20:37 smithi186 podman[69393]: 2024-04-21 09:20:37.549175954 +0000 UTC m=+0.684578631 container died b88f7882456ae1b284ee8bafed465b2bcc19fa5e8c08c91ca0a26271ada25b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T09:20:37.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:37 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:37.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:37.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:37 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:37.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:37.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:37 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:37.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:37.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:37 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:37.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:37.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:37 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:37.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:37.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:37 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:37.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:38.166 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.8.service' 2024-04-21T09:20:38.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:20:38.203 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-21T09:20:38.203 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-21T09:20:38.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.9 2024-04-21T09:20:38.231 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:37.924+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:38.232 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:37.924+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:38.232 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:37.924+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:38.232 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:37.924+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:38.232 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:37.924+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:38.232 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:37 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:37.924+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:38.505 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 systemd[1]: Stopping Ceph osd.9 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:20:38.505 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.505+0000 7f031be2f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:20:38.506 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.505+0000 7f031be2f700 -1 osd.9 110 *** Got signal Terminated *** 2024-04-21T09:20:38.506 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.505+0000 7f031be2f700 -1 osd.9 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:20:38.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:38 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:38.358+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:38.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:38 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:38.358+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:38.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:38 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:38.358+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:38.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:38 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:38.358+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:38.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:38 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:38.358+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:38.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:38 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:38.366+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:38.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:38 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:38.366+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:38.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:38 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:38.366+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:38.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:38 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:38.366+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:38.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:38 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:38.366+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:38.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:38 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:38.366+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:39.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.896+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:39.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.896+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:39.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.896+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:39.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.896+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:39.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.896+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:39.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:38 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:38.896+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:39.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:39 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:39.327+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:39.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:39 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:39.327+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:39.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:39 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:39.327+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:39.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:39 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:39.327+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:39.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:39 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:39.327+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:39.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:39.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:39.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:39.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:39.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:39.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:39.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:39.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:39.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:39.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:39.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:39 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:39.414+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:40.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:39 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:39.881+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:40.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:39 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:39.881+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:40.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:39 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:39.881+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:40.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:39 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:39.881+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:40.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:39 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:39.881+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:40.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:39 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:39.881+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:40.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:40 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:40.305+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:40.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:40 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:40.305+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:40.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:40 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:40.305+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:40.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:40 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:40.305+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:40.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:40 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:40.305+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:40.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:40 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:40.457+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:40.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:40 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:40.457+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:40.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:40 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:40.457+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:40.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:40 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:40.457+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:40.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:40 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:40.457+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:40.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:40 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:40.457+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:41.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:40 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:40.909+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:41.369 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:40 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:40.909+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:41.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:40 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:40.909+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:41.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:40 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:40.909+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:41.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:40 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:40.909+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:41.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:40 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:40.909+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:41.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:41 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:41.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:41.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:41 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:41.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:41.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:41 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:41.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:41.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:41 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:41.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:41.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:41 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:41.317+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:41.862 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:41 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:41.507+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:41.862 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:41 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:41.507+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:41.863 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:41 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:41.507+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:41.863 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:41 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:41.507+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:41.863 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:41 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:41.507+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:41.863 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:41 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:41.507+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:42.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:41 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:41.955+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:42.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:41 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:41.955+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:42.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:41 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:41.955+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:42.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:41 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:41.955+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:42.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:41 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:41.955+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:42.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:41 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:41.955+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:42.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:42.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:42.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:42.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:42.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:42.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:42.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:42.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:42.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:42.280+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:42.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:42 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:42.490+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:42.874 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:42 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:42.490+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:42.874 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:42 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:42.490+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:42.875 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:42 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:42.490+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:42.875 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:42 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:42.490+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:42.875 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:42 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:42.490+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:43.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:42 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:42.979+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.375706+0000 front 2024-04-21T09:19:27.375976+0000 (oldest deadline 2024-04-21T09:19:52.075791+0000) 2024-04-21T09:20:43.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:42 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:42.979+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.777113+0000 front 2024-04-21T09:19:36.776859+0000 (oldest deadline 2024-04-21T09:20:02.676920+0000) 2024-04-21T09:20:43.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:42 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:42.979+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:42.677552+0000 front 2024-04-21T09:19:42.677389+0000 (oldest deadline 2024-04-21T09:20:07.977478+0000) 2024-04-21T09:20:43.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:42 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:42.979+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:50.278364+0000 front 2024-04-21T09:19:50.278304+0000 (oldest deadline 2024-04-21T09:20:19.678947+0000) 2024-04-21T09:20:43.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:42 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:42.979+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.781260+0000 front 2024-04-21T09:20:01.781186+0000 (oldest deadline 2024-04-21T09:20:27.081018+0000) 2024-04-21T09:20:43.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:42 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9[60811]: 2024-04-21T09:20:42.979+0000 7f0318423700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:09.982126+0000 front 2024-04-21T09:20:09.981905+0000 (oldest deadline 2024-04-21T09:20:35.281758+0000) 2024-04-21T09:20:43.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:43 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:43.325+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:43.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:43 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:43.325+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:43.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:43 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:43.325+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:43.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:43 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:43.325+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:43.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:43 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:43.325+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:43.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:43 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:43.462+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:43.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:43 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:43.462+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:43.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:43 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:43.462+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:43.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:43 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:43.462+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:43.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:43 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:43.462+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:43.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:43 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:43.462+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:44.237 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:43 smithi066 podman[96844]: 2024-04-21 09:20:43.926491807 +0000 UTC m=+5.519904462 container died aecfc98c68925d246745990ce34c1729198a74c6fcd3a9f3b6679ddb76cff242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T09:20:44.238 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:44 smithi066 podman[96844]: 2024-04-21 09:20:44.238734592 +0000 UTC m=+5.832147253 container cleanup aecfc98c68925d246745990ce34c1729198a74c6fcd3a9f3b6679ddb76cff242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-21T09:20:44.619 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:44 smithi066 bash[96844]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9 2024-04-21T09:20:44.620 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:44 smithi066 podman[96861]: 2024-04-21 09:20:44.406523836 +0000 UTC m=+0.482712880 container remove aecfc98c68925d246745990ce34c1729198a74c6fcd3a9f3b6679ddb76cff242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T09:20:44.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:44 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:44.354+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:44.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:44 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:44.354+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:44.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:44 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:44.354+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:44.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:44 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:44.354+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:44.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:44 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:44.354+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:44.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:44 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:44.354+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:44.859 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:44 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:44.476+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:44.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:44 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:44.476+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:44.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:44 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:44.476+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:44.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:44 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:44.476+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:44.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:44 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:44.476+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:44.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:44 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:44.476+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:44.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:44 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:44.476+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:45.083 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:44 smithi066 podman[96951]: 2024-04-21 09:20:44.720263909 +0000 UTC m=+0.018563813 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:20:45.084 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:44 smithi066 podman[96951]: 2024-04-21 09:20:44.882458637 +0000 UTC m=+0.180758567 container create 8378136a766c8621102ed6b9bc4158557be0f6ab1576d7e28b85f8407d30e630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:20:45.084 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:45 smithi066 podman[96951]: 2024-04-21 09:20:45.084772116 +0000 UTC m=+0.383072013 container init 8378136a766c8621102ed6b9bc4158557be0f6ab1576d7e28b85f8407d30e630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T09:20:45.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:45 smithi066 podman[96951]: 2024-04-21 09:20:45.088234803 +0000 UTC m=+0.386534695 container start 8378136a766c8621102ed6b9bc4158557be0f6ab1576d7e28b85f8407d30e630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T09:20:45.370 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:45 smithi066 podman[96951]: 2024-04-21 09:20:45.159570131 +0000 UTC m=+0.457870037 container attach 8378136a766c8621102ed6b9bc4158557be0f6ab1576d7e28b85f8407d30e630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-21T09:20:45.686 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:45 smithi066 podman[96951]: 2024-04-21 09:20:45.398680133 +0000 UTC m=+0.696980022 container died 8378136a766c8621102ed6b9bc4158557be0f6ab1576d7e28b85f8407d30e630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-21T09:20:45.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:45 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:45.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:45.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:45 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:45.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:45.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:45 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:45.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:45.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:45 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:45.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:45.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:45 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:45.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:45.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:45 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:45.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:45.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:45 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:45.444+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:45.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:45 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:45.444+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:45.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:45 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:45.444+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:45.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:45 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:45.444+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:45.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:45 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:45.444+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:45.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:45 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:45.444+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:45.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:45 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:45.444+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:45.940 INFO:journalctl@ceph.osd.9.smithi066.stdout:Apr 21 09:20:45 smithi066 podman[96976]: 2024-04-21 09:20:45.929713465 +0000 UTC m=+0.521504325 container remove 8378136a766c8621102ed6b9bc4158557be0f6ab1576d7e28b85f8407d30e630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:20:45.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.9.service' 2024-04-21T09:20:46.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:20:46.028 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-21T09:20:46.028 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-21T09:20:46.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.10 2024-04-21T09:20:46.325 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 systemd[1]: Stopping Ceph osd.10 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:20:46.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.325+0000 7ff5e76c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.325+0000 7ff5e76c6700 -1 osd.10 110 *** Got signal Terminated *** 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.325+0000 7ff5e76c6700 -1 osd.10 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.402+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.402+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.402+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.402+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.402+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.402+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:46.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:46 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:46.402+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:46.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:46.377+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:46.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:46.377+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:46.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:46.377+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:46.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:46.377+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:46.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:46.377+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:46.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:46.377+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:47.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:47 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:47.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:47.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:47 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:47.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:47.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:47 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:47.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:47.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:47 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:47.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:47.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:47 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:47.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:47.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:47 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:47.374+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:47.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:47 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:47.368+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:47.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:47 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:47.368+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:47.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:47 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:47.368+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:47.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:47 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:47.368+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:47.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:47 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:47.368+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:47.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:47 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:47.368+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:47.860 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:47 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:47.368+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:48.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:48 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:48.348+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:48.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:48 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:48.348+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:48.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:48 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:48.348+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:48.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:48 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:48.348+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:48.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:48 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:48.348+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:48.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:48 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:48.348+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:48.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:48 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:48.348+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:48.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:48.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:48.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:48.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:48.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:48.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:48.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:48.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:48.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:48.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:48.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:48 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:48.356+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:49.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:49.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:49.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:49.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:49.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:49.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:49.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:49.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:49.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:49.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:49.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:49.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:49.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:49 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:49.337+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:49.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:49 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:49.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:49.609 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:49 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:49.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:49.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:49 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:49.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:49.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:49 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:49.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:49.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:49 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:49.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:49.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:49 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:49.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:49.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:49 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:49.302+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:50.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:50 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:50.314+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:50.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:50 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:50.314+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:50.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:50 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:50.314+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:50.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:50 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:50.314+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:50.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:50 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:50.314+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:50.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:50 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:50.314+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:50.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:50 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:50.314+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:50.758 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:50.339+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:50.758 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:50.339+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:50.758 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:50.339+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:50.758 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:50.339+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:50.759 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:50.339+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:50.759 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:50.339+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:50.759 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:50 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:50.339+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:51.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:51.325+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:27.182305+0000 front 2024-04-21T09:19:27.182397+0000 (oldest deadline 2024-04-21T09:19:51.282526+0000) 2024-04-21T09:20:51.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:51.325+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:38.884051+0000 front 2024-04-21T09:19:38.884091+0000 (oldest deadline 2024-04-21T09:20:02.384198+0000) 2024-04-21T09:20:51.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:51.325+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.385618+0000 front 2024-04-21T09:19:46.385852+0000 (oldest deadline 2024-04-21T09:20:11.685699+0000) 2024-04-21T09:20:51.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:51.325+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2024-04-21T09:19:51.686225+0000 front 2024-04-21T09:19:51.686294+0000 (oldest deadline 2024-04-21T09:20:16.986056+0000) 2024-04-21T09:20:51.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:51.325+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:01.587456+0000 front 2024-04-21T09:20:01.587401+0000 (oldest deadline 2024-04-21T09:20:25.087464+0000) 2024-04-21T09:20:51.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:51.325+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:05.588501+0000 front 2024-04-21T09:20:05.588431+0000 (oldest deadline 2024-04-21T09:20:35.588786+0000) 2024-04-21T09:20:51.610 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10[56924]: 2024-04-21T09:20:51.325+0000 7ff5dfcb2700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.990024+0000 front 2024-04-21T09:20:18.990109+0000 (oldest deadline 2024-04-21T09:20:43.689948+0000) 2024-04-21T09:20:51.676 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:51.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:51.676 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:51.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:51.676 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:51.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:51.676 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:51.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:51.676 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:51.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:51.676 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:51.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:51.676 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:51 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:51.341+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:52.057 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:51 smithi083 podman[68731]: 2024-04-21 09:20:51.743221497 +0000 UTC m=+5.520744047 container died 7b37fef0a6cceab6f9ac11fe92cc540ffdd1e94a802fd5148b09126d6b4c36b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T09:20:52.057 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 podman[68731]: 2024-04-21 09:20:52.058463441 +0000 UTC m=+5.835985992 container cleanup 7b37fef0a6cceab6f9ac11fe92cc540ffdd1e94a802fd5148b09126d6b4c36b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-21T09:20:52.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 bash[68731]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10 2024-04-21T09:20:52.360 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 podman[68748]: 2024-04-21 09:20:52.29316779 +0000 UTC m=+0.548183941 container remove 7b37fef0a6cceab6f9ac11fe92cc540ffdd1e94a802fd5148b09126d6b4c36b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0) 2024-04-21T09:20:52.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:52.313+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:52.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:52.313+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:52.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:52.313+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:52.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:52.313+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:52.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:52.313+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:52.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:52.314+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:52.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:52 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:52.314+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:52.995 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 podman[68840]: 2024-04-21 09:20:52.60647739 +0000 UTC m=+0.017822007 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:20:52.995 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 podman[68840]: 2024-04-21 09:20:52.743915045 +0000 UTC m=+0.155259666 container create 22b0d7aa041f2c8b277342d33a8cf900cf3b5fdfec195dbc9eb083e41d04bd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T09:20:52.995 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 podman[68840]: 2024-04-21 09:20:52.92971762 +0000 UTC m=+0.341062239 container init 22b0d7aa041f2c8b277342d33a8cf900cf3b5fdfec195dbc9eb083e41d04bd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-21T09:20:52.995 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 podman[68840]: 2024-04-21 09:20:52.932890105 +0000 UTC m=+0.344234719 container start 22b0d7aa041f2c8b277342d33a8cf900cf3b5fdfec195dbc9eb083e41d04bd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T09:20:52.995 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:52 smithi083 podman[68840]: 2024-04-21 09:20:52.996034504 +0000 UTC m=+0.407379135 container attach 22b0d7aa041f2c8b277342d33a8cf900cf3b5fdfec195dbc9eb083e41d04bd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:20:53.246 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 21 09:20:53 smithi083 podman[68840]: 2024-04-21 09:20:53.222097309 +0000 UTC m=+0.633441927 container died 22b0d7aa041f2c8b277342d33a8cf900cf3b5fdfec195dbc9eb083e41d04bd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:20:53.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:53.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:53.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:53.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:53.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:53.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:53.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:53.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:53.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:53.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:53.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:53.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:53.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:53.321+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:53.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.10.service' 2024-04-21T09:20:53.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:20:53.876 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-21T09:20:53.876 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-21T09:20:53.876 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.11 2024-04-21T09:20:54.184 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:53 smithi186 systemd[1]: Stopping Ceph osd.11 for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.185+0000 7f94a069e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.185+0000 7f94a069e700 -1 osd.11 110 *** Got signal Terminated *** 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.185+0000 7f94a069e700 -1 osd.11 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.308+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.308+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.308+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.308+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.308+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:54.588 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.308+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:54.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:54 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:54.308+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:55.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:55.347+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:55.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:55.347+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:55.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:55.347+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:55.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:55.347+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:55.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:55.347+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:55.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:55.347+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:55.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:55 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:55.347+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:56.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:56.369+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:56.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:56.369+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:56.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:56.369+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:56.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:56.369+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:56.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:56.369+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:56.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:56.369+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:56.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:56 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:56.369+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:57.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:57.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:57.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:57.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:57.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:57.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:57.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:57.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:57.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:57.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:57.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:57.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:57.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:57 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:57.397+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:58.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:58.388+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-04-21T09:19:28.606777+0000 front 2024-04-21T09:19:28.606478+0000 (oldest deadline 2024-04-21T09:19:51.506473+0000) 2024-04-21T09:20:58.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:58.388+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-21T09:19:36.008975+0000 front 2024-04-21T09:19:36.008888+0000 (oldest deadline 2024-04-21T09:20:01.908976+0000) 2024-04-21T09:20:58.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:58.388+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-21T09:19:46.609733+0000 front 2024-04-21T09:19:46.609714+0000 (oldest deadline 2024-04-21T09:20:11.909903+0000) 2024-04-21T09:20:58.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:58.388+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-21T09:20:02.412122+0000 front 2024-04-21T09:20:02.412102+0000 (oldest deadline 2024-04-21T09:20:27.012582+0000) 2024-04-21T09:20:58.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:58.388+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-04-21T09:20:07.013045+0000 front 2024-04-21T09:20:07.012931+0000 (oldest deadline 2024-04-21T09:20:32.913002+0000) 2024-04-21T09:20:58.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:58.388+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2024-04-21T09:20:18.114560+0000 front 2024-04-21T09:20:18.114520+0000 (oldest deadline 2024-04-21T09:20:43.414480+0000) 2024-04-21T09:20:58.838 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:58 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11[57333]: 2024-04-21T09:20:58.388+0000 7f949cc92700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-21T09:20:25.115541+0000 front 2024-04-21T09:20:25.115702+0000 (oldest deadline 2024-04-21T09:20:49.215474+0000) 2024-04-21T09:20:59.918 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:20:59 smithi186 podman[69509]: 2024-04-21 09:20:59.627257466 +0000 UTC m=+5.552667515 container died 3cb7e3ddc6a5abbdf4f177973a5847d90a3cdd29db3b685c233d0cdc0cb25e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-21T09:21:00.268 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 podman[69509]: 2024-04-21 09:21:00.00181225 +0000 UTC m=+5.927222289 container cleanup 3cb7e3ddc6a5abbdf4f177973a5847d90a3cdd29db3b685c233d0cdc0cb25e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:21:00.268 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 bash[69509]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11 2024-04-21T09:21:00.587 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 podman[69527]: 2024-04-21 09:21:00.269660484 +0000 UTC m=+0.637019386 container remove 3cb7e3ddc6a5abbdf4f177973a5847d90a3cdd29db3b685c233d0cdc0cb25e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-21T09:21:00.880 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 podman[69617]: 2024-04-21 09:21:00.520130896 +0000 UTC m=+0.017750261 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T09:21:00.881 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 podman[69617]: 2024-04-21 09:21:00.628779865 +0000 UTC m=+0.126399229 container create ea1cbedc33db10a817158a1be124a03b1e7477ce4608c533b8f6210fc0177d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-21T09:21:00.881 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 podman[69617]: 2024-04-21 09:21:00.881579768 +0000 UTC m=+0.379199138 container init ea1cbedc33db10a817158a1be124a03b1e7477ce4608c533b8f6210fc0177d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-21T09:21:01.183 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 podman[69617]: 2024-04-21 09:21:00.885011724 +0000 UTC m=+0.382631089 container start ea1cbedc33db10a817158a1be124a03b1e7477ce4608c533b8f6210fc0177d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-21T09:21:01.183 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:00 smithi186 podman[69617]: 2024-04-21 09:21:00.972806337 +0000 UTC m=+0.470425703 container attach ea1cbedc33db10a817158a1be124a03b1e7477ce4608c533b8f6210fc0177d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:21:01.527 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 21 09:21:01 smithi186 podman[69617]: 2024-04-21 09:21:01.187784876 +0000 UTC m=+0.685404242 container died ea1cbedc33db10a817158a1be124a03b1e7477ce4608c533b8f6210fc0177d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:21:01.827 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@osd.11.service' 2024-04-21T09:21:01.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:01.864 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-21T09:21:01.864 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-21T09:21:01.864 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.a 2024-04-21T09:21:02.225 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:21:01 smithi066 systemd[1]: Stopping Ceph mds.a for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:02.572 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:21:02 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a[67210]: 2024-04-21T09:21:02.225+0000 7f1d9f626700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:02.572 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:21:02 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a[67210]: 2024-04-21T09:21:02.225+0000 7f1d9f626700 -1 mds.a *** got signal Terminated *** 2024-04-21T09:21:06.370 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:21:06 smithi066 podman[97069]: 2024-04-21 09:21:06.065417835 +0000 UTC m=+4.004793490 container died c81f341de8057637ae653f80b2505fb9990934826b76e27b2a5799a6f4862893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-04-21T09:21:06.673 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:21:06 smithi066 podman[97069]: 2024-04-21 09:21:06.45621987 +0000 UTC m=+4.395595513 container cleanup c81f341de8057637ae653f80b2505fb9990934826b76e27b2a5799a6f4862893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-21T09:21:06.673 INFO:journalctl@ceph.mds.a.smithi066.stdout:Apr 21 09:21:06 smithi066 bash[97069]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-a 2024-04-21T09:21:06.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.a.service' 2024-04-21T09:21:06.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:06.821 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-21T09:21:06.821 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-21T09:21:06.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.d 2024-04-21T09:21:07.119 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:21:06 smithi066 systemd[1]: Stopping Ceph mds.d for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:07.619 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:21:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d[66251]: 2024-04-21T09:21:07.151+0000 7f910693f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:07.619 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:21:07 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d[66251]: 2024-04-21T09:21:07.151+0000 7f910693f700 -1 mds.d *** got signal Terminated *** 2024-04-21T09:21:10.619 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:21:10 smithi066 podman[97202]: 2024-04-21 09:21:10.347536722 +0000 UTC m=+3.295961511 container died 5e59e9398166770e91e8f8177eb2828e522c633cf6408077cdc49f1d8bcace32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-21T09:21:11.084 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:21:10 smithi066 podman[97202]: 2024-04-21 09:21:10.660230975 +0000 UTC m=+3.608655753 container cleanup 5e59e9398166770e91e8f8177eb2828e522c633cf6408077cdc49f1d8bcace32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-21T09:21:11.084 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:21:10 smithi066 bash[97202]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d 2024-04-21T09:21:11.084 INFO:journalctl@ceph.mds.d.smithi066.stdout:Apr 21 09:21:10 smithi066 podman[97217]: 2024-04-21 09:21:10.861340238 +0000 UTC m=+0.513006099 container remove 5e59e9398166770e91e8f8177eb2828e522c633cf6408077cdc49f1d8bcace32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T09:21:11.101 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.d.service' 2024-04-21T09:21:11.135 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:11.136 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-21T09:21:11.136 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-21T09:21:11.136 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.g 2024-04-21T09:21:11.369 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:21:11 smithi066 systemd[1]: Stopping Ceph mds.g for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:11.869 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:21:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g[66862]: 2024-04-21T09:21:11.454+0000 7f49fc29b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:11.870 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:21:11 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g[66862]: 2024-04-21T09:21:11.454+0000 7f49fc29b700 -1 mds.g *** got signal Terminated *** 2024-04-21T09:21:15.619 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:21:15 smithi066 podman[97322]: 2024-04-21 09:21:15.347094213 +0000 UTC m=+3.988071156 container died 7422db8583215f803276163acacf116013fdeac5f219c67f885549974a54783a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-21T09:21:16.063 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:21:15 smithi066 podman[97322]: 2024-04-21 09:21:15.821762222 +0000 UTC m=+4.462739167 container cleanup 7422db8583215f803276163acacf116013fdeac5f219c67f885549974a54783a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-21T09:21:16.064 INFO:journalctl@ceph.mds.g.smithi066.stdout:Apr 21 09:21:15 smithi066 bash[97322]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-g 2024-04-21T09:21:16.196 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.g.service' 2024-04-21T09:21:16.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:16.229 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-21T09:21:16.229 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-21T09:21:16.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.j 2024-04-21T09:21:16.369 INFO:journalctl@ceph.mds.j.smithi066.stdout:Apr 21 09:21:16 smithi066 systemd[1]: Stopping Ceph mds.j for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:16.869 INFO:journalctl@ceph.mds.j.smithi066.stdout:Apr 21 09:21:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-j[65895]: 2024-04-21T09:21:16.549+0000 7f93cd271700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:16.870 INFO:journalctl@ceph.mds.j.smithi066.stdout:Apr 21 09:21:16 smithi066 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-j[65895]: 2024-04-21T09:21:16.549+0000 7f93cd271700 -1 mds.j *** got signal Terminated *** 2024-04-21T09:21:19.759 INFO:journalctl@ceph.mds.j.smithi066.stdout:Apr 21 09:21:19 smithi066 podman[97443]: 2024-04-21 09:21:19.476362531 +0000 UTC m=+3.026586144 container died 5ef9adaba20add3205eceb558fbcdd4bdc09970e44c8f49b4e042b40aae22a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-j, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-21T09:21:20.084 INFO:journalctl@ceph.mds.j.smithi066.stdout:Apr 21 09:21:19 smithi066 podman[97443]: 2024-04-21 09:21:19.842165656 +0000 UTC m=+3.392389278 container cleanup 5ef9adaba20add3205eceb558fbcdd4bdc09970e44c8f49b4e042b40aae22a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-21T09:21:20.084 INFO:journalctl@ceph.mds.j.smithi066.stdout:Apr 21 09:21:19 smithi066 bash[97443]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-j 2024-04-21T09:21:20.230 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.j.service' 2024-04-21T09:21:20.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:20.262 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-21T09:21:20.262 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-21T09:21:20.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.b 2024-04-21T09:21:20.559 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:21:20 smithi083 systemd[1]: Stopping Ceph mds.b for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:20.859 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:21:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: 2024-04-21T09:21:20.559+0000 7feceb6d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:20.860 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:21:20 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b[62455]: 2024-04-21T09:21:20.559+0000 7feceb6d8700 -1 mds.b *** got signal Terminated *** 2024-04-21T09:21:21.850 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:21:21 smithi083 podman[68958]: 2024-04-21 09:21:21.542792186 +0000 UTC m=+1.088629632 container died 9079479c6bb07150c990bb349f4dcd7bdd6e7ee81f22fd8c768a363222a076c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-21T09:21:22.109 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:21:21 smithi083 podman[68958]: 2024-04-21 09:21:21.851856516 +0000 UTC m=+1.397693993 container cleanup 9079479c6bb07150c990bb349f4dcd7bdd6e7ee81f22fd8c768a363222a076c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T09:21:22.109 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:21:21 smithi083 bash[68958]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b 2024-04-21T09:21:22.110 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 21 09:21:22 smithi083 podman[68972]: 2024-04-21 09:21:22.061061506 +0000 UTC m=+0.519344291 container remove 9079479c6bb07150c990bb349f4dcd7bdd6e7ee81f22fd8c768a363222a076c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-21T09:21:22.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.b.service' 2024-04-21T09:21:22.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:22.322 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-21T09:21:22.322 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-21T09:21:22.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.e 2024-04-21T09:21:22.666 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:21:22 smithi083 systemd[1]: Stopping Ceph mds.e for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:23.110 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:21:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e[61836]: 2024-04-21T09:21:22.666+0000 7f02d2501700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:23.110 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:21:22 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e[61836]: 2024-04-21T09:21:22.666+0000 7f02d2501700 -1 mds.e *** got signal Terminated *** 2024-04-21T09:21:25.859 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:21:25 smithi083 podman[69078]: 2024-04-21 09:21:25.533696347 +0000 UTC m=+2.999533372 container died e42708ebc0bf8876a777daf907232ee4ed118244915ec2b44391652f296e58e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:21:26.181 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:21:25 smithi083 podman[69078]: 2024-04-21 09:21:25.955389036 +0000 UTC m=+3.421226069 container cleanup e42708ebc0bf8876a777daf907232ee4ed118244915ec2b44391652f296e58e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-21T09:21:26.181 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 21 09:21:25 smithi083 bash[69078]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-e 2024-04-21T09:21:26.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.e.service' 2024-04-21T09:21:26.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:26.420 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-21T09:21:26.420 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-21T09:21:26.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.h 2024-04-21T09:21:26.734 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:21:26 smithi083 systemd[1]: Stopping Ceph mds.h for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:27.109 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:21:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h[62149]: 2024-04-21T09:21:26.734+0000 7f820d8fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:27.109 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:21:26 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h[62149]: 2024-04-21T09:21:26.734+0000 7f820d8fb700 -1 mds.h *** got signal Terminated *** 2024-04-21T09:21:29.841 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:21:29 smithi083 podman[69211]: 2024-04-21 09:21:29.532184746 +0000 UTC m=+2.892369141 container died 66486971edd068516916cf9874aaf72b1cf8513377e7e90fc74b3e22b8a41514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True) 2024-04-21T09:21:30.110 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:21:29 smithi083 podman[69211]: 2024-04-21 09:21:29.842774326 +0000 UTC m=+3.202958721 container cleanup 66486971edd068516916cf9874aaf72b1cf8513377e7e90fc74b3e22b8a41514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-21T09:21:30.110 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:21:29 smithi083 bash[69211]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h 2024-04-21T09:21:30.110 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 21 09:21:30 smithi083 podman[69225]: 2024-04-21 09:21:30.018822563 +0000 UTC m=+0.487193590 container remove 66486971edd068516916cf9874aaf72b1cf8513377e7e90fc74b3e22b8a41514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T09:21:30.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.h.service' 2024-04-21T09:21:30.298 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:30.298 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-21T09:21:30.298 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-21T09:21:30.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.k 2024-04-21T09:21:30.596 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:30 smithi083 systemd[1]: Stopping Ceph mds.k for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:30.859 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:30 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k[61520]: 2024-04-21T09:21:30.596+0000 7ff0938ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:30.859 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:30 smithi083 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k[61520]: 2024-04-21T09:21:30.596+0000 7ff0938ab700 -1 mds.k *** got signal Terminated *** 2024-04-21T09:21:32.730 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:32 smithi083 podman[69333]: 2024-04-21 09:21:32.46156415 +0000 UTC m=+1.950651397 container died 7c2f6bee9ced4395a5e13b9039b90f6ad587d2dbb52ba6cff5022bcbd45488d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T09:21:32.988 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:32 smithi083 podman[69333]: 2024-04-21 09:21:32.804609982 +0000 UTC m=+2.293697219 container cleanup 7c2f6bee9ced4395a5e13b9039b90f6ad587d2dbb52ba6cff5022bcbd45488d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True) 2024-04-21T09:21:32.988 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:32 smithi083 bash[69333]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k 2024-04-21T09:21:33.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.k.service' 2024-04-21T09:21:33.256 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:32 smithi083 podman[69347]: 2024-04-21 09:21:32.989331394 +0000 UTC m=+0.523996690 container remove 7c2f6bee9ced4395a5e13b9039b90f6ad587d2dbb52ba6cff5022bcbd45488d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-21T09:21:33.256 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:33 smithi083 systemd[1]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-21T09:21:33.256 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:33 smithi083 systemd[1]: Stopped Ceph mds.k for 0cca84f6-ffbd-11ee-bc93-c7b262605968. 2024-04-21T09:21:33.257 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 21 09:21:33 smithi083 systemd[1]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.k.service: Consumed 1.079s CPU time. 2024-04-21T09:21:33.267 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:33.267 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-21T09:21:33.267 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-21T09:21:33.268 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.c 2024-04-21T09:21:33.587 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:21:33 smithi186 systemd[1]: Stopping Ceph mds.c for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:33.850 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:21:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c[62802]: 2024-04-21T09:21:33.610+0000 7fd51d8be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:33.850 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:21:33 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c[62802]: 2024-04-21T09:21:33.610+0000 7fd51d8be700 -1 mds.c *** got signal Terminated *** 2024-04-21T09:21:37.828 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:21:37 smithi186 podman[69721]: 2024-04-21 09:21:37.53587642 +0000 UTC m=+4.074615175 container died a7641693372a8e982e24d343985bb74f7c1080dfd577051b8aac366f15cf55fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:21:38.087 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:21:37 smithi186 podman[69721]: 2024-04-21 09:21:37.903188012 +0000 UTC m=+4.441926779 container cleanup a7641693372a8e982e24d343985bb74f7c1080dfd577051b8aac366f15cf55fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T09:21:38.088 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 21 09:21:37 smithi186 bash[69721]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-c 2024-04-21T09:21:38.318 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.c.service' 2024-04-21T09:21:38.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:38.353 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-21T09:21:38.353 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-21T09:21:38.354 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.f 2024-04-21T09:21:38.588 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:21:38 smithi186 systemd[1]: Stopping Ceph mds.f for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:39.087 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:21:38 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f[62492]: 2024-04-21T09:21:38.698+0000 7fbb2cca5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:39.088 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:21:38 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f[62492]: 2024-04-21T09:21:38.698+0000 7fbb2cca5700 -1 mds.f *** got signal Terminated *** 2024-04-21T09:21:41.816 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:21:41 smithi186 podman[69853]: 2024-04-21 09:21:41.538563505 +0000 UTC m=+2.970231103 container died deac478869e68e406000b3123513b7c751ea415493b662310c55894e92eb0599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T09:21:42.087 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:21:41 smithi186 podman[69853]: 2024-04-21 09:21:41.908088536 +0000 UTC m=+3.339756134 container cleanup deac478869e68e406000b3123513b7c751ea415493b662310c55894e92eb0599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-21T09:21:42.088 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 21 09:21:41 smithi186 bash[69853]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-f 2024-04-21T09:21:42.350 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.f.service' 2024-04-21T09:21:42.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:42.384 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-21T09:21:42.384 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-21T09:21:42.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.i 2024-04-21T09:21:42.837 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:21:42 smithi186 systemd[1]: Stopping Ceph mds.i for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:42.837 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:21:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i[63118]: 2024-04-21T09:21:42.694+0000 7fdcf5843700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:42.838 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:21:42 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i[63118]: 2024-04-21T09:21:42.694+0000 7fdcf5843700 -1 mds.i *** got signal Terminated *** 2024-04-21T09:21:45.837 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:21:45 smithi186 podman[69976]: 2024-04-21 09:21:45.531706294 +0000 UTC m=+2.937655074 container died 51d59c97461494b91c2e93bad739f172fdabec7b48ad552ca0e61244e33fdc98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T09:21:45.837 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:21:45 smithi186 podman[69976]: 2024-04-21 09:21:45.83800476 +0000 UTC m=+3.243953582 container cleanup 51d59c97461494b91c2e93bad739f172fdabec7b48ad552ca0e61244e33fdc98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-21T09:21:46.087 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:21:45 smithi186 bash[69976]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i 2024-04-21T09:21:46.088 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 21 09:21:46 smithi186 podman[69990]: 2024-04-21 09:21:46.039841152 +0000 UTC m=+0.509042392 container remove 51d59c97461494b91c2e93bad739f172fdabec7b48ad552ca0e61244e33fdc98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-21T09:21:46.155 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.i.service' 2024-04-21T09:21:46.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:46.190 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-21T09:21:46.190 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-21T09:21:46.190 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.l 2024-04-21T09:21:46.508 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:21:46 smithi186 systemd[1]: Stopping Ceph mds.l for 0cca84f6-ffbd-11ee-bc93-c7b262605968... 2024-04-21T09:21:46.838 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:21:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l[62168]: 2024-04-21T09:21:46.508+0000 7ff532479700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:21:46.838 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:21:46 smithi186 ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l[62168]: 2024-04-21T09:21:46.508+0000 7ff532479700 -1 mds.l *** got signal Terminated *** 2024-04-21T09:21:49.710 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:21:49 smithi186 podman[70097]: 2024-04-21 09:21:49.392904679 +0000 UTC m=+2.997380471 container died 1bfc3d587b3c79e1fd1668441ca808f10464584b1070f5c9e4a495d0f5f6194f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-21T09:21:50.054 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:21:49 smithi186 podman[70097]: 2024-04-21 09:21:49.784822525 +0000 UTC m=+3.389298316 container cleanup 1bfc3d587b3c79e1fd1668441ca808f10464584b1070f5c9e4a495d0f5f6194f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-21T09:21:50.054 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 21 09:21:49 smithi186 bash[70097]: ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968-mds-l 2024-04-21T09:21:50.204 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0cca84f6-ffbd-11ee-bc93-c7b262605968@mds.l.service' 2024-04-21T09:21:50.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:21:50.236 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-21T09:21:50.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T09:21:50.433 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T09:21:53.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T09:21:53.202 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T09:21:55.511 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T09:21:55.699 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T09:21:58.042 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:21:58.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:21:58.107 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:21:58.132 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-21T09:21:58.134 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi066/crash 2024-04-21T09:21:58.135 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash -- . 2024-04-21T09:21:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T09:21:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2024-04-21T09:21:58.172 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi083/crash 2024-04-21T09:21:58.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash -- . 2024-04-21T09:21:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T09:21:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-04-21T09:21:58.243 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi186/crash 2024-04-21T09:21:58.244 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash -- . 2024-04-21T09:21:58.275 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T09:21:58.275 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-04-21T09:21:58.276 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-21T09:21:58.276 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-21T09:21:58.315 INFO:tasks.cephadm:Compressing logs... 2024-04-21T09:21:58.316 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T09:21:58.359 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T09:21:58.362 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T09:21:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T09:21:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:: No such file or directory 2024-04-21T09:21:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log 2024-04-21T09:21:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.g.log 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.j.log 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.3.log 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.0.log 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.x.log 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.a.log 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.x.log: 87.6% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T09:21:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.6.log 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr: 94.7% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.a.log 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr: 84.1% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.a.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.d.log 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.9.log 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-21T09:21:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-21T09:21:58.406 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:: No such file or directory 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose --/var/log/ceph/cephadm.log: /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.2.log 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.f.log 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.5.log 2024-04-21T09:21:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.c.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.5.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.c.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.z.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.i.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr: 87.6% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.11.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.8.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T09:21:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.l.log 2024-04-21T09:21:58.411 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T09:21:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose -- 2024-04-21T09:21:58.412 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/cephadm.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.7.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.b.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.4.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.7.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.1.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.y.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.h.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.y.log: gzip 87.6% -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.10.log 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.h.log: 90.3% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T09:21:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T09:21:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.e.log 2024-04-21T09:21:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T09:21:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T09:21:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-21T09:21:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.b.log 2024-04-21T09:21:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.k.log 2024-04-21T09:21:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T09:21:58.428 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.l.log: 94.7% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-21T09:21:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-21T09:21:58.442 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T09:21:58.450 INFO:teuthology.orchestra.run.smithi186.stderr: 94.6% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-21T09:21:58.453 INFO:teuthology.orchestra.run.smithi186.stderr: 92.4% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-21T09:21:58.460 INFO:teuthology.orchestra.run.smithi083.stderr: 92.5% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-21T09:21:58.460 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-21T09:21:58.467 INFO:teuthology.orchestra.run.smithi186.stderr: 93.9% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-21T09:21:58.695 INFO:teuthology.orchestra.run.smithi083.stderr: 92.9% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-21T09:21:58.729 INFO:teuthology.orchestra.run.smithi083.stderr: 92.0% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-21T09:21:58.760 INFO:teuthology.orchestra.run.smithi186.stderr: 91.9% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-21T09:21:58.762 INFO:teuthology.orchestra.run.smithi066.stderr: 90.3% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-21T09:21:59.391 INFO:teuthology.orchestra.run.smithi066.stderr: 90.6% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-21T09:21:59.734 INFO:teuthology.orchestra.run.smithi186.stderr: 92.1% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-21T09:21:59.908 INFO:teuthology.orchestra.run.smithi083.stderr: 92.1% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-21T09:22:00.048 INFO:teuthology.orchestra.run.smithi186.stderr: 92.8% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-21T09:22:00.118 INFO:teuthology.orchestra.run.smithi066.stderr: 92.0% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-21T09:22:00.143 INFO:teuthology.orchestra.run.smithi186.stderr: 91.9% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-21T09:22:00.338 INFO:teuthology.orchestra.run.smithi083.stderr: 91.8% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-21T09:22:00.383 INFO:teuthology.orchestra.run.smithi066.stderr: 92.6% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-21T09:22:00.407 INFO:teuthology.orchestra.run.smithi083.stderr: 92.7% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-21T09:22:00.538 INFO:teuthology.orchestra.run.smithi083.stderr: 92.5% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-21T09:22:00.539 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-21T09:22:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m2.164s 2024-04-21T09:22:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m8.150s 2024-04-21T09:22:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m0.287s 2024-04-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi066.stderr: 91.9% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-21T09:22:00.664 INFO:teuthology.orchestra.run.smithi066.stderr: 91.7% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-21T09:22:00.666 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-04-21T09:22:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m2.293s 2024-04-21T09:22:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m9.357s 2024-04-21T09:22:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.311s 2024-04-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi186.stderr: 92.3% -- replaced with /var/log/ceph/0cca84f6-ffbd-11ee-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m2.523s 2024-04-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi186.stderr:user 0m7.602s 2024-04-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m0.270s 2024-04-21T09:22:00.899 INFO:tasks.cephadm:Archiving logs... 2024-04-21T09:22:00.899 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi066/log 2024-04-21T09:22:00.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T09:22:02.318 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi083/log 2024-04-21T09:22:02.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T09:22:03.647 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi186/log 2024-04-21T09:22:03.648 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T09:22:04.878 INFO:tasks.cephadm:Removing cluster... 2024-04-21T09:22:04.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 --force 2024-04-21T09:22:05.078 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T09:22:05.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 --force 2024-04-21T09:22:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T09:22:05.929 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0cca84f6-ffbd-11ee-bc93-c7b262605968 --force 2024-04-21T09:22:06.115 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 0cca84f6-ffbd-11ee-bc93-c7b262605968 2024-04-21T09:22:06.373 INFO:tasks.cephadm:Removing cephadm ... 2024-04-21T09:22:06.374 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T09:22:06.394 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T09:22:06.410 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T09:22:06.427 INFO:tasks.cephadm:Teardown complete 2024-04-21T09:22:06.427 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-21T09:22:06.456 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-21T09:22:06.456 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T09:22:06.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T09:22:06.465 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T09:22:06.577 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-21T09:22:06.604 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-21T09:22:06.605 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T09:22:06.612 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T09:22:06.619 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T09:22:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-21T09:22:06.631 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-21T09:22:06.634 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T09:22:06.634 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-21T09:22:06.634 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 -258us[ -258us] +/- 75ms 2024-04-21T09:22:06.634 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -457us[ -471us] +/- 66ms 2024-04-21T09:22:06.635 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +771us[ +771us] +/- 75ms 2024-04-21T09:22:06.635 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T09:22:06.635 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T09:22:06.636 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-21T09:22:06.636 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 -267us[ -267us] +/- 75ms 2024-04-21T09:22:06.636 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -451us[ -448us] +/- 66ms 2024-04-21T09:22:06.636 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +773us[ +773us] +/- 75ms 2024-04-21T09:22:06.636 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T09:22:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-21T09:22:06.644 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T09:22:06.644 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-21T09:22:06.644 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 -282us[ -282us] +/- 75ms 2024-04-21T09:22:06.644 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -480us[ -481us] +/- 66ms 2024-04-21T09:22:06.644 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +737us[ +737us] +/- 75ms 2024-04-21T09:22:06.645 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T09:22:06.645 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-21T09:22:06.657 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-21T09:22:06.658 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-21T09:22:06.684 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T09:22:06.770 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T09:22:06.837 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T09:22:06.902 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T09:22:06.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:22:06.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T09:22:06.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:22:06.965 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T09:22:06.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:22:06.999 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-21T09:22:07.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-21T09:22:07.057 INFO:teuthology.task.internal:Duration was 1928.889304 seconds 2024-04-21T09:22:07.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-21T09:22:07.088 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-21T09:22:07.089 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T09:22:07.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T09:22:07.101 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T09:22:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T09:22:07.136 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T09:22:07.138 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T09:22:07.647 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-21T09:22:07.647 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-04-21T09:22:07.648 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T09:22:07.669 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-21T09:22:07.669 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T09:22:07.713 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-04-21T09:22:07.714 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T09:22:07.744 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-21T09:22:07.744 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T09:22:07.747 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T09:22:07.756 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T09:22:07.811 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-21T09:22:07.811 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T09:22:07.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T09:22:07.822 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T09:22:08.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-21T09:22:08.218 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-21T09:22:08.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T09:22:08.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T09:22:08.266 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T09:22:08.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-21T09:22:08.303 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:22:08.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:22:08.308 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:22:08.327 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-04-21T09:22:08.328 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-21T09:22:08.353 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-04-21T09:22:08.375 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:22:08.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:22:08.394 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:22:08.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:22:08.409 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:22:08.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:22:08.429 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-21T09:22:08.443 INFO:teuthology.task.internal:Transferring archived files... 2024-04-21T09:22:08.443 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi066 2024-04-21T09:22:08.444 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T09:22:08.488 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi083 2024-04-21T09:22:08.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T09:22:08.524 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538/remote/smithi186 2024-04-21T09:22:08.524 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T09:22:08.559 INFO:teuthology.task.internal:Removing archive directory... 2024-04-21T09:22:08.559 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T09:22:08.562 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T09:22:08.564 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T09:22:08.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-21T09:22:08.624 INFO:teuthology.task.internal:Not uploading archives. 2024-04-21T09:22:08.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-21T09:22:08.658 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-21T09:22:08.658 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T09:22:08.661 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T09:22:08.663 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T09:22:08.673 INFO:teuthology.orchestra.run.smithi066.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 09:22 /home/ubuntu/cephtest 2024-04-21T09:22:08.675 INFO:teuthology.orchestra.run.smithi083.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 09:22 /home/ubuntu/cephtest 2024-04-21T09:22:08.677 INFO:teuthology.orchestra.run.smithi186.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 09:22 /home/ubuntu/cephtest 2024-04-21T09:22:08.679 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-21T09:22:08.691 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-21T09:22:08.759 INFO:teuthology.nuke:Checking targets against current locks 2024-04-21T09:22:08.780 DEBUG:teuthology.nuke:shortname: smithi066 2024-04-21T09:22:08.780 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T09:22:08.802 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:43:38.123882', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNWDD6NXJaM0r1cQ6hz827U7hnauLGZej8cHcsYfO/sxrBHiZ9jxxYlemwuvuQv9qtzhAQTb7TEc3IAUMSKVc9A='} 2024-04-21T09:22:08.807 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-21T09:22:08.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T09:22:08.828 DEBUG:teuthology.nuke:shortname: smithi186 2024-04-21T09:22:08.828 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T09:22:08.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:43:38.121229', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUXBiHT4qAE/mo7YvPRoA788R9+B6KJzvCWntbMNXc854vuuq3/mC66GojIy4Xr/+nZZZ/zicQHArNkb40NPjY='} 2024-04-21T09:22:08.850 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666538', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:43:38.122744', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtX7ERHlIMZyJNMv7GKaPSMdQwgvQGrc8HgYRDdEoDc7GklfjHkGaBoM7r4nbTBsnHB87xg+chHSCL+9eM6fFE='} 2024-04-21T09:22:08.860 INFO:teuthology.orchestra.console.smithi066:Power off 2024-04-21T09:22:08.860 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T09:22:08.887 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off 2024-04-21T09:22:08.888 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:08.905 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-21T09:22:08.905 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T09:22:08.920 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-04-21T09:22:08.930 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-21T09:22:08.931 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:08.952 INFO:teuthology.orchestra.console.smithi186:Power off 2024-04-21T09:22:08.953 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T09:22:08.965 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-21T09:22:08.977 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2024-04-21T09:22:08.977 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:09.001 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-21T09:22:12.921 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:12.966 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:13.002 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:13.032 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-04-21T09:22:13.078 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-21T09:22:13.113 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-21T09:22:17.034 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:17.079 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:17.114 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:17.145 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off 2024-04-21T09:22:17.189 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-21T09:22:17.224 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-21T09:22:17.246 INFO:teuthology.orchestra.console.smithi066:Power off completed 2024-04-21T09:22:17.290 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-21T09:22:21.226 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:22:21.337 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2024-04-21T09:22:21.438 INFO:teuthology.orchestra.console.smithi186:Power off completed 2024-04-21T09:22:21.543 INFO:teuthology.run:Summary data: description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} duration: 1928.8893041610718 failure_reason: 'error during scrub thrashing: ' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-21T09:22:21.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T09:22:21.637 INFO:teuthology.run:FAIL